Vertical teams at scale a.k.a how to scale frontend work in growing organisation

What we were trying to achieve?

We want to scale our work between multiple independent teams each with its product owner, designer, and multiple engineers. To do that we have a few solutions that well discuss and Ill try to explain our logic behind making our final decision.

What are our options?

Separate applications with a host.

Creating independent applications that would live in separate repositories is one of the most popular way of building frontend. Each team has its own technologies, tools, and buildchain which some of them really value. There is unfortunately one hefty problem - versioning. In this setup, after making a change in our application we would have to update version in package registry and then bump version inside our host application. And then there is our UI library that each app is using.

Monorepo to the rescue?

Our current application is a monorepo, rather large monorepo.
At the time of writing this article, we have 19290 files with 3580303 lines of code with 89 authors in the last few months.
To create new applications we dont have to think about build configurations. Linters, unit tests, e2e tests are all already set up and ready for development. Its as simple as adding a new directory and path to our app routing. It comes at a cost of being forced to use specific technologies and tools. Changing them would need to be approved and developed by each individual team and its a nightmare to coordinate.
Additionally, our pipelines duration is already ranging between tiresome and infinite (our last worked for 52 minutes). Merge requests are happening on average every hour so we have a constant stream of workersehmworking.
Unfortunately, deployment is shared across all teams so even the smallest of changes needs to be verified through multiple people in code review and needs to pass our pipelines two times (one before making a merge and one after on master branch).

Microfrontend to the rescue?

A microfrontend is a microservice that exists within a browser. Each microfrontend has its own repository, its own build configuration and process, and is being able to be deployed individually. There is a lot of implementation of this concept. One of the most popular tool that helps is single-spa - a framework for bringing together multiple JavaScript microfrontends in a frontend application. It is an incredible tool and should be considered for greenfield projects. It gives a lot of tools and features, such as being able to use different frameworks in the same application.

These additional features however would mean increased initial payload and memory allocation. Although performance overhead is minor, when we dont use these additional functionalities its a waste of resources, especially when setting up single-spa would be costly to implement in our existing setup.

Module federation to the rescue?

Finally, we decided to integrate new applications using Webpacks latest feature - module federation. It integrates nicely with our webpack configuration, has a tiny boilerplate, and is straightforward to read (after understanding the complexity of webpack itself).

Multiple separate builds should form a single application. These separate builds should not have dependencies between each other, so they can be developed and deployed individually. - Webpack team

We distinguish between local and remote modules. Local modules are normal modules that are part of the current application. Remote modules are modules that are being loaded at the runtime.

The idea is simple. An application references a remote using a configured name that is not known at compile time. That reference is only resolved at runtime by the so-called remote entry point. Its a minimal script that provides actual external.

In its simplest form, the code looks like this:

// webpack.config.jsmodule.exports = {  ...    plugins: [        new ModuleFederationPlugin({            name: 'mother',            remotes: {                "remote": "remote@http://localhost:3001/remoteEntry.js"            },        }),    ]}// src/index.jsimport RemoteApp from 'remote/App'

Our remote application will be imported from an external URL instead of our local repository and loaded at runtime.

What we gained by adopting microservice architecture?

Microfrontend gave us a lot of benefits and resolved a lot of issues we had. Well walk through in a bit more details.

Independent teams - independent applications

Our vertical teams can work on their own in separate repositories and are free to choose the technologies they need to create the best user experience.

Autonomous deployments

Our team can now deploy features without being dependent on the mother app. We were able to set up our pipelines that on average last about 8 minutes.

Code trimming

We are not adding additional code to the already humongous codebase of our monorepo.

Onboarding new people

Onboarding can be overwhelming for new developers, especially juniors that join our teams. We eased the process and new friends were able to contribute even on their first day with confidence.

Developer experience

Its often overlooked, but developer experience is crucial for every successful project. Because we created a new project and were independent of our monorepo application, we were able to integrate Snowpack into our day-to-day work. It gave us instant startup time with a fast refresh and cleaner configuration.

What problems we've encountered?

On a road to production, we had a few blockades that none of us had met before. We had to be a little bit more creative.

Singleton libraries

In libraries such as React, we cannot run multiple versions of the same library at once if they dont share the same version. We updated to the latest version in both applications which was a lengthy process. After that, we added our react library to shared dependencies in Wepback configuration.

new ModuleFederationPlugin({    shared: {        "react": { singleton: true }    }})

Preview environment

Our monorepo is using preview deployments to be able to test changes both manually and using e2e tests. By using module federation, we are not creating branches in our mother app - code is dynamically run directly on the client and server-side.
The way we were able to get around that was by dynamically injecting the correct remote based on the parameter in the URL. It was not as easy as we thought. To achieve that we had to:

1. Deploy our remote application to be available through some dynamic URL on each pull request. We created a deploy preview step in our CI that created dynamic storage using Amazons Simple Storage Service.

https://$bucketName.s3.eu-central-1.amazonaws.com/federated/remoteEntry.js

1. Inject this dynamic remote into our living staging environment.

// https://website.com?remoteApp1=https://$bucketName.s3.eu-central-1.amazonaws.com/federated/remoteEntry.jsconst remote = new URLSearchParams().get('remoteApp1')

1. Insert script tag with this remote.

const element = document.createElement('script');element.src = remote;document.head.appendChild(element);

1. Load actual component to be used in our code.

const Component = React.lazy(loadComponent(remote, module));return <Component {...props} />

Learning curve

Our setup has a steep learning curve. There is a lot to learn and understand to get a grasp for some of the low-level concepts and webpack documentation isnt much easier to read with its building blocks defined as ContainerPlugin, ContainerReferencePlugin, and ModuleFederationPlugin.

Conclusion

Module federation filled an enormous gap in the frontend world. Lessons learned can help us extract some of the self-contained applications currently living inside monorepo to speed our development and give a lot of freedom to autonomous teams.

Whats next?

Our current setup is impressive for us. With our fast pipelines, separate deployments, and independent teams we are more agile than ever.
But we must not rest on our laurels. There is a new version of React coming and we need to figure out a way of introducing backward-incompatible changes such as this. And we have our eyes on the new cool kids on the block - Javascripts native module system (ESM) and non-JS bundlers such as esbuild written in Go.