CI/CD for .net 6, using GitHub actions

With the publishing of Orleans.SyncWork, Ive had the opportunity to explore GitHub actions - which is a way to automate workflows. Heres some of my first experience into the action (groan).

Automated workflow

First things first, what even is a workflow, and what does it mean to automate one? Well dear potential reader, a workflow is nothing more than a set of steps taken to complete a task.

From Wikipedia:

A workflow consists of an orchestrated and repeatable pattern of activity, enabled by the systematic organization of resources into processes that transform materials, provide services, or process information. It can be depicted as a sequence of operations, the work of a person or group, the work of an organization of staff, or one or more simple or complex mechanisms.

You can think of a workflow as the steps taken to accomplish something. That something can be any number of things, related to any number of subjects. In the context of this post, well be mostly covering workflows as it relates to a build and release pipeline, also commonly referred to as continuous integration (CI) and continuous delivery (CD).

Id like to cover both the CI and CD aspects of the Orleans.SyncWork, so lets get started.

What youll (probably) need

• Experience working with a CLI
• Unit tests
• An idea of the steps that you need to take in order to build, test, and deploy your code. If these steps are already in your head in the form of CLI commands, then youre already most of the way there!
• Some patience in getting your workflow properly laid out

Continuous Integration

Before youre able to deploy code through a work flow (continuous delivery), you need to be able to integrate it safely into your main/trunk. For dotnet, through a handful of CLI commands, the building and testing of code is pretty straight forward. Doing CI has the added benefit of bringing up a brand new environment for builds, each and every time, a similar idea to why Ive been a proponent of build servers for so long.

Build

dotnet build

The above command is the minimum you need to build either a solution file or project file on the dotnet side of things. From a continuous integration perspective, you may want to throw a few flags onto the command, such as:

dotnet build --configuration release

and things of that nature, see whats available to you with the dotnet build documentation.

That could look like this from the CLI:

Test

Next is testing. Ive probably already said it too many times, but test your code! Especially if youre building libraries! Tests help ensure that the code youre writing does what you say it does. Additionally tests be used as documentation in a way, if the tests are named well, and are invoking the code in a similar manner to how your consumer will use it, theyll be in a better place to get started using what youve delivered.

Like the build command, the test command is quite straightforward as well:

dotnet test

Of course the above is the absolute bare minimum command, there are lots of parameters that can be passed to dotnet test as well.

A test run could look like this from the CLI:

CI Action

With the above dotnet build and dotnet test commands, we have most of what well need to put together an action to build and test our code, automatically!

There is lots of good information, even some specific to .net testing on the documentation. I pretty much used the documentation as a starting point, and ended up with this

.github/workflows/ci.yml:

name: Build and teston:  pull_request:    branches: [ main ]jobs:  build:    runs-on: ubuntu-latest    steps:    - uses: actions/checkout@v2      with:        fetch-depth: 0    - name: Setup .NET      uses: actions/setup-dotnet@v1      with:        dotnet-version: 6.0.x    - name: Restore dependencies      run: dotnet restore    - name: Build      run: dotnet build -c Release --no-restore    - name: Test      run: dotnet test -c Release --no-build --verbosity normal --filter "Category!=LongRunning"

The above file should be mostly straightforward, first we give a name to our workflow with name, specify the triggers for the workflow, in this case on pull requests against the main branch. The file then goes on to define the job build, which specifies an OS to run on, then steps. The steps do a few things of note:

• Checkout the code, placing it into the ubuntu instance that is being utilized from the step previous
• Setup .NET with a prebuilt action
• restore dependencies
  • This is an explicit step, rather than implicit from our previous dotnet build command, as if this explicit restore step were to fail, wed more quickly know at what point of the build there is a failure
• build the solution file at the project root
• finally test the code

We have a few new flags in our build and test commands, namely specifying a configuration of release, and dont restore/build on steps after those steps having already occurred. One final note is the --filter "Category!=LongRunning" - I was having trouble with the test runner getting through the tests I had laid out. They took 3 minutes to run locally, but ran for over 25 minutes on the build agent. Due to this fact, I added some classifications of category to the longer running tests, and excluded them from the test run in the above ci.yml file.

Continuous Delivery

Continuous delivery is a lot like continuous integration, and builds on top of it. Im of the thinking that CD should do everything CI should do, or perhaps even better, actually rely on the CI, rather than redefining the steps in your CICD like I ended up doing. That was a bit rambly, but CD should do everything CI should do, except with the additional step of actually delivering (deploying/pushing) the code as a part of its workflow.

Delivery Complexities

That delivery part can have a lot of nuance to it that ups the complexity by a significant amount when compared to just CI. What does it mean to actually deliver code? Well, that could depend a lot on what type of code youre actually delivering. In my case, Im delivering a NuGet package, which has its own complexities, but what else is there? Well the other obvious thing that comes to mind is a web site / web api, one which could potentially have database changes to roll out in addition to the code. This to me, has the potential to be worlds more complex than just pushing a NuGet package up. How do you not only handle failures, but detect them and roll back, in the case of something going wrong with either you web push or database push? Perhaps Ill be able to explore that one day, but for now, lets get back to the NuGet package.

So, is there a complexity with delivering a NuGet package? Yes. NuGet package versioning can be a big undertaking when it comes to manual deployments, much less CD; as there is a requirement of NuGet packages being immutable. Does this mean that for every check in, on every potential branch that will be pushed to NuGet, you need to update some text file or code to indicate the next built version? That was my initial thinking, but thankfully that is not the case with the help of Nerdbank.GitVersioning.

I dont think I have my CICD set up exactly how Ill end up having it, but for right now it works. I installed the NerdBank.GitVersioning tool and package, and now for each build, I get a unique version number for the NuGet package upon build. I can toggle between prerelease or release packages, and can even publish nightly builds that contain a commit hash on them, all in the name of uniquely identifiable NuGet packages.

There was a fair amount of setup, that in some ways Im still working through, but this article is already getting long enough, take a look at the PR(s) if youre curious: https://github.com/OrleansContrib/Orleans.SyncWork/pull/8 and https://github.com/OrleansContrib/Orleans.SyncWork/pull/13. The tldr of it is, nbgv tooling uses the git history to rev the version number being used during builds, allowing for unique build numbers each time the CI/CD fires.

CD Action

Theres been a fair amount of information so far, but between our CI action and the information about GitVersioning, we have everything we need to put together a first pass cicd.yml. For CI, we were doing builds/tests against PRs to main. For CD, well want to do delivery when code is pushed to main, as well as branches that begin with RELEASE/v*. My thinking here is that since well be integrating often into main (theoretically), we dont necessarily want to create full new release packages for every commit to main. We could however, create prerelease NuGet packages to main for each commit, making those changes available to the NuGet feed, but them not being labeled as a release version. Otherwise I have Nerdbank.GitVersioning set up to release release versions of packages from the RELEASE/v* branches.

The CD action file itself will look very similar to the CI one, just with the few additions of dotnet nuget... commands, shown below:

.github/workflows/cicd.yml

name: Build, test, and deployon:  push:    branches:      - 'main'      - 'RELEASE/v**'jobs:  build:    runs-on: ubuntu-latest    steps:    - uses: actions/checkout@v2      with:        fetch-depth: 0    - name: Setup .NET      uses: actions/setup-dotnet@v1      with:        dotnet-version: 6.0.x    - name: Restore dependencies      run: dotnet restore    - name: Build      run: dotnet build -c Release --no-restore    - name: Test      run: dotnet test -c Release --no-restore --no-build --verbosity normal --filter "Category!=LongRunning"    - name: Pack      run: dotnet pack src/Orleans.SyncWork/Orleans.SyncWork.csproj -c Release --no-restore --no-build --include-symbols -p:SymbolPackageFormat=snupkg -o .    - name: Push to NuGet      run: dotnet nuget push *.nupkg --skip-duplicate -s https://api.nuget.org/v3/index.json -k ${{secrets.NUGET_API_KEY}}

In the above, youll notice that its more than 50% the same file as CI. I was going to potentially look into composite actions as some point, so see if I could instead chain CI and CD, rather than redefining the CI within the CD file; but Ive not yet had a chance to explore that.

Aside from the change to the on event (pull_request -> push), there are two new commands at the bottom dotnet pack and dotnet nuget push. The dotnet pack command is used to package up the project specified into a .nupkg file (and snupkg in this case for symbols). Finally the dotnet nuget push command is used to push those newly packed NuGet package(s) to the feed specified of nuget.org. On this command youll also see the {{secrets.NUGET_API_KEY}} portion of the command, this is defined as a repository secret and it can be used to pass secret information to things like workflows, in this case its my NuGet API key. These secrets can be set from the repository Settings -> Secrets:

Still to do

The putting out a release branch is still a bit of a manual step for me. I need to run nbgv prepare-release from my local environment, then push up the subsequently created RELEASE/v* branch and updated new pre-release version that is created under main.

That may not have made sense.

If Im working in main with a prerelease version of 1.0-prerelease, when I nbgv prepare-release, main will be (as an example) updated to 1.1-prerelease with a branch called RELEASE/v1.0 created having a release version of 1.0. The push of these two changes will currently build a new prerelease package of 1.1-prerelease and release package of 1.0, both of which will contain the same content at the time of being pushed.

Im not sure how I feel about the above. I like the automatic build and deploying of packages, but I dont like having to create the release locally. I could conceivably create a manually dispatched workflow that did this release preparation for me, but then thered still be the slight strangeness around immediately pushing out a prerelease package with no changes in comparison to the previous pre-release package built, and the new release package being built. Im not sure what the right flow is quite yet, what I have right now does work, it just seems a bit messy.

Perhaps Ill eventually look into workflows more like this:

• CI - continues to be run on PRs to main
• CICD - can be executed against the main branch and RELEASE/v* branches, but is not done automatically as it currently is
  • Im not sure about this part, main and the release should theoretically always be deployable, but do I really want to deploy every time theres a change to main?
• Prepare release workflow - with this workflow, Id want to do the local steps I currently take for preparing a release, but do it through a github action.

My Workflow

I wasn't sure if the literal sections were required, so here they are, "My Workflow" I think was pretty well described above, hopefully!

Submission Category:

• Maintainer Must-Haves
• DIY Deployments

Yaml File or Link to Code

Additional Resources / Info

• Orleans.SyncWork on GitHub
• Wikipedia
• CLI
• And its like whats the deal with build servers?
• Getting started with XUnit
• dotnet build
• dotnet test
• Building and Testing with .NET using GitHub actions
• Nerdbank.GitVersioning
• GitHub Composite Actions
• GitHub Encrypted Secrets