Your Web News in One Place

Help Webnuz

Referal links:

Sign up for GreenGeeks web hosting
August 19, 2022 06:30 pm GMT

Building infrastructure for an open-source programmable zapier

Hi community! Looking forward to hearing your thoughts about this.

Inngest was started out of the pain of building products that required complex integrations and background jobs especially those driven by user interactivity. Weve built the infrastructure so many times that we decided to open-source it for other people so that everyone can have access to world class tooling, instead of just MANGA.

Weve created a single API that lets you record whats happening in your app, then lets you run serverless functions in response to these events. Its a developers Zapier.

Its more than a job queue or serverless

Example of infra required

From our past experience, we know that building out reliable products requires more than a job queue, Kafka, or Lambda, and off-the-shelf products don't cut it for the customization us developers need.

We believe that it should be accessible to everyone not $just 1B+ companies who have hundreds of people to configure and maintain infra.

Even for something as basic as reliably handling a webhook youll need some sort of queue, retries and logging which can take weeks to build, even with off the shelf systems in clouds. Heres our plan to give this to everyone, from jamstack APIs all the way to realtime elixir webapps:

Event-driven infrastructure foundations

Using a single API to record whats happening in your app provides rich functionality like:

  • Running step functions in real time, whenever specific things happen, automatically. This includes off-the-shelf, repeatable functions, like sending emails, forwarding events to analytics, or common APIs.
  • Historic replay and retries by saving events we allow you to replay and retry past data locally on your own machine if there are errors, or retry your functions if there are errors.
  • Personalized debugging when your users do things in your app, you can assign the events that are generated to that user. This lets you see every function that ran for each of your users, making debugging and insight super easy
  • Coding best practices. Sending an event describing whats happening then reacting to the event reduces coupling and makes your critical path faster.
  • No servers or job queues. You can make everything serverless by scheduling functions to run in the future when an event comes in, making your app entirely serverless
  • Monitoring and visibility, allowing you to see whats happening in your app and the functions that are running
  • Complex coordinated functionality, such as on cart added, wait 24 hours for the checkout event and only run the next step if the user didnt check out. This is annoying to build manually, and usually requires complex cron jobs, messy database state, or external (expensive) services like braze.com

Local development first

We also think that local development should be a first class part of the process, without spending days installing and configuring services locally. The open source CLI already does the following:

  • Makes it easy to create functions in any language, either from a blank slate or using quick starts for common use cases
  • Locally run functions with a single command using the same execution engine deployed to your cloud, with a single command. This makes sure you have local <> production parity
  • Start a fully functional local test environment in a single command which accepts events then runs functions locally
  • CI/CD out of the box, using things like github actions
  • Replay real production events locally for debugging & testing.

We want everyone to have world-class infrastructure to build the complex parts of their app the stuff thats beyond basic API routes. This includes things youd only dream of at fortune 500s: serverless functions, event schema generation, event and function versioning, blue-green deploys, throttling, idempotency, and local debugging.

Even for basic use cases like handling webhooks, this makes your life far easier. And we want this for everyone, available locally, with the source freely available to hack on.

Status & next steps

Weve already got our execution engine and CLI that does the above. Were planning webassembly support, step-over debuggers, and a ton more. What would you like to see? Do you have job queues or events in your system? What would your ideal programmable zapier do?

Feel free to comment here or on Github at https://github.com/inngest/inngest.


Original Link: https://dev.to/tonyhb/building-infrastructure-for-an-open-source-programmable-zapier-36cf

Share this article:    Share on Facebook
View Full Article

Dev To

An online community for sharing and discovering great ideas, having debates, and making friends

More About this Source Visit Dev To