Tools you love vs tools you use at work

Something weird is going on. On the one hand, there are amazing tools out there that allow you to build applications quickly without thinking of infrastructure. Tools like Vercel, Heroku, Firebase, Netlify. On the other hand, most teams keep building on AWS or GCP, seemingly ignoring these amazing tools for all but the smallest of projects. How come? Arent we all supposed to be serverless by now?

The cost of great developer experience

What makes Vercel and Heroku amazing is simplicity. Just connect a repo (or push your code) and see it live in a few seconds. Perfection! It just works. How? Youre not supposed to think about that, its been handled by the platform.

Thats all great, until you need a bit more than a PaaS can offer. Things like block storage for example, or private networking, or pub-sub, or a specific kind of a database. There are many of them, you know - not just relational. And there is a reason why each exists.

But building such a stack now presents a problem. Your apps work just fine on a PaaS; you only need one extra piece that can be built say on AWS. So you end up spreading your stack across multiple providers. Frontends are running on Netlify, backends on Heroku, databases on AWS. All over public network of course. And then there is a problem of latencies and multi-region setups. Things got to be close to each other to be fast, but there arent that many regions to choose from on a PaaS. The only option now seems to be to move to AWS (or GCP).

The cost of great flexibility

This migration of course could have been avoided altogether if you had started on AWS right away. Maybe the developer experience is not that great, but then at least you can be sure that your future needs are covered. In fact most startups do just that and Amazon as well as Google and Microsoft are happy to extend generous free credits to them.

At this point startups are split in two distinct groups: those who have DevOps expertise, and those who dont. The former are doing alright, perhaps moving a bit slower, but just a bit. They get their developer infrastructure in place rather quickly, set up CI / CD pipelines, and start shipping. That group is a minority though, roughly 10-15% of all startups. The remaining 90% dont have in-house DevOps expertise. And thats where it becomes really painful.

AWS has over 200 services. Say youre building a simple web app with a backend, frontend, and database. Should you deploy it into EC2? Or ECS? Or EKS? Or Lambda? Or Fargate? Or Beanstalk? Whats a VPC? How do I make my service public? And so on. Its not clear at all. And so those unfortunate teams without DevOps expertise can either move slow by trial and error, or hire a DevOps engineer. Which to a startup is a tough decision with consequences. This essentially means hiring one less engineer to work on the product

But what about infrastructure as code?

Its great. Terraform is a de-facto standard for DevOps work, other tools like are either niche (Pulumi) or badly designed (CloudFormation). But infra-as-code does not solve this dilemma. But you still need to think in the exact same terms as without tools. So they arent making it any simpler they are solving a different problem (infrastructure consistency). Present-day IaC tools are a bit like assembly language: total precision, zero abstraction.

There must be a better way

Just imagine if building mobile apps was like that. To start building an app you use one combination of IDE, language and toolchain. Then when it matures you rebuild the whole thing in a different language, with a different IDE, using a different toolchain. Ridiculous, isnt it?

So can we not somehow combine the developer experience of tools like Vercel with flexibility of major cloud providers? So you could start small, move fast with reasonable defaults, then add customisation without the need to rebuild from scratch? Wouldnt that be normal?

Thats what we thought and built Digger.dev. It automatically generates infrastructure for your code in your AWS account. It has a nice web UI so you can deploy your apps and services in a couple clicks. But under the hood is Terraform. So when time comes you can customise or even completely replace it. But only if you need to.

We have just launched and hungry for feedback - check it out!