Paradox of scale: why Netflix couldn't build Netflix

TL;DR notes from articles I read today.

The paradox of scale

Galls Law: A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.

• What works for small simple systems is significantly different for what works for large complicated systems.
• Things that seem crucial for building high-quality production systems, like the ones large tech companies use, are mostly not required when you are starting out building new products.
• To build large systems, you need to start with small steps to build small systems. Think big, start small, find a foothold and start journeying is how any large system starts getting built.
• Looking at what the giants do and trying to copy it is only a good way to fail. Instead, learn lessons from how they started, what they did along their journeys (and why they did it)

Full post here, 4 mins read

The cracking monolith: the forces that call for microservices

• Overweight monoliths exhibit degrading system performance and stability, or slow development cycles, or both.
• Single points of failure are typical of large monolithic apps and when they come under pressure, your team spends too much time solving technical issues instead of on development. For example - outages in non-critical data processing that bring down the whole system. Or all time-intensive tasks grouped into the background and becoming so unstable as to need a dedicated team. Or changing one part of the system affects others that should logically be unrelated.
• If shipping a hotfix takes weeks or months, you will have a problem with slow development. To know when it is a good time to break the monolith, you should watch out for:
  • CI builds that take longer than 10 minutes (though a good CI tool should help it by splitting or auto-sequencing tasks).
  • Slow deployment, due to many dependencies and multiple app instances, especially when containerized.
  • Slow onboarding of new staff as it may take them months to get comfortable enough to make a non-trivial change to the codebase. Veteran team members becoming a bottleneck as reviewers because too many developers are waiting for their inputs.
  • New use cases and problems are not easily addressed with existing tools, and software updates are being put off, indicating you are dependent on outdated technology.

Full post here, 6 mins read

Get these notes directly in your inbox every weekday by signing up for my newsletter, in.snippets().