Building a Flutter application for Web, iOS and Android using a CI/CD pipeline on CodeBuild cdk4j

This post is a follow up to the last one where I showed a CDK project that can be used to build a Flutter application for Web.

In this post, we are going to expand our existing project on Github to be able to build an apk file for Android and a zip file for iOS. Before I can show you how this is possible, lets start with some challenges that Ive faced

The aim of this CI/CD pipeline is (not yet) to be able to push the apps into the AppStore / PlayStore for testing. Thats something we can add later

Challenges on the way to a full CI/CD pipeline for Flutter on CodeBuild

While preparing this post I unfortunately faced more problems building up the pipeline than expected. Several problems.

AWS CodePipeline does not support M1 / macOS build images

Currently, AWS CodePipeline unfortunately does not provide the possibility to use the famous M1 minis on AWS as CodeBuild images. Tis his a real problem, as is makes it impossible to use CodeBuild for building iOS apps.
Running XCode on macOS is a requirementfor building a Flutter app for iOS.
The M1 minis on AWS are currently pricey as hell and even worse starting one costs you 24h! You need to actual get a dedicated instance, not usable for our use case of quickly building something for a side-project.
So we needed to find an alternative read below!

The current AWS CodePipeline standard runtimes are not able to build (modern) Android applications

The runtimes available and exposed by CodePipeline support Android runtime 29 and the Docker images are provisioned using Java 8. Unfortunately, as of July 2021, the Android gradle tools (used by Flutter) require Java 11. I have created an issue in the corresponding Github (see here) but needed to find a workaround to move on I think Ive found one, but I hope that anyone reading this might have a better way or idea?

TypeScript dependencies on AWS Lambda can take your sleep

When implementing the trigger for the iOS App build (see more details below) I decided to quickly implement the HTTPS POST call using TypeScript which turned out to be a bad decision
I had trouble getting the axios dependency that I am using installed correctly. I asked around, especially my fellow AWS Community Builders and got a lot of great tips and ideas (kudos to Martin and Matt). Martin had the right stomach feeling I was missing a npm install.

Matt enlightened me with the three different possibilities of making Typescript Lambda functions understand their dependencies:

1. Bundle dependency with your source code (can be achieved using esbuild)
2. Add a package.json and node_modules to the lambda function source - only a good idea if dependencies cannot be minified
3. Put the dependencies in a lambda layer

At the end, this challenge was especially difficult because I needed to add the required "npm install" in two places: In the "installCommands" for the CodePipeline itself and in the "installCommands" for the Flutter build step.

CodeBuild is slow, misses conditional steps and misses integrations - and does not easily allow multi-branch pipelines

While implementing the pipeline and solving the different challenges mentioned above, I lost some time because of CodeBuild being "slow" (>1min wait time during provisioining of the build containers). Thats understandable given the nature of the service, however it would be cool to have something like a "warm start" for a pipeline where the containers are re-used instead of re-provisioned.
There are no conditional steps - no chance to run a job only based on environment variables or anything similar. That made me implement a workaround. It would be cool to be able to use something like "branch-conditions" in the way Jenkins offers it.

CodeBuild offers only basic integration to SNS, but you cannot integrated a "lambda build step" to run the CodeMagic integration i nparallel to the flutter build job, but that is not possible, so I needed to run this "at the end" of the pipeline.

Another thing I'd love to have: multibranch pipelines. I needed to merge everything to main directly in order to test, because I couldnt figure out how my CDKPipeline would be able toe support multiple branches.

Reaching the goal: a full CI/CD pipeline running on AWS CodeBuild to build a Flutter app for Web, Android and iOS

Here is a diagram of the final result that I am presenting today:

The output artifacts of our pipeline are:
Flutter Web application (located on S3 and reachable through HTTP call)
Flutter Android APK (that can be side-loaded on Android phones, located on S3 bucket)
Flutter iOS App (that can be side-loaded on iOS phones, located within CodeMagic)

As the diagram shows, we needed to fall back to an 3rd-party, non-AWS service to be able to package the iOS application. After doing a quick vendor selection and a shortlist that included Bitrise and CodeMagic I decided to integrate CodeMagic in this example because I liked the API more and it offers more free build credits/minutes. Setting it up took less then 5 minutes it connects natively to Github and the set up of the Flutter pipeline is very easy.
The integration is set up using a Lambda function that calls the start build API.

How did I solve the challenges mentioned above?

The problem building the iOS image was resolved by integrating the external Service CodeMagic.

The Android Runtime Dependencies problems with Java 11 was resolved by switching to a custom docker container (Open source) and then installing the requirements on top of it (npm/node, awscli, etc.).

What did you learn in this post?

In this post you have learned on how to expand the implementation of our CI/CD pipeline for an example Flutter application to not only building a web application, but also building an Android APK and an iOS zip file.
You have also seen an extension and integration of the Codepipeline with SNS for notifications and those events being picked up by a lambda function to trigger an external HTTPs API.
This is a major step with this pipeline we are able to publish our application for three different platforms without a single code change and it will all happen completely automatically!

Id be glad to get your inputs into my Github repository as a pull request or just as comments on the project itself.

Next steps

Further expansions needed to this project:
CodePipeline already has a SNS topic that it reports to but right now the build iOS / Android App packages are not exposed anywhere the idea would be to publish to an SQS queue the name of the APK file and the CodeMagic Build Id and have a lambda function that is triggered by the queue update a link on the example application to download the newest version of the app Today, we need to retrieve both from S3 / CodeMagic itself
use the CloudFormation Exports of the Lambda functions in the Flutter application instead of hardcoding the URLs for the Lambda Function URLs- enhance security for Lambda Function URLs
add CloudFront in front of S3 to allow HTTPs connections to the Flutter App
enhance CI/CD pipeline to package Windows App using Flutter
enhance CI/CD pipeline to push created apps to App Store / Play Store

Feel free to contribute and add your contributions to this project into my Github repository.