How to use Redis and Lua Scripts in a C ASP.NET Core Microservice Architecture

Use StackExchange.Redis and run Redis in Docker to Cache Aggregated Database Data for a Scaled C# Application

Redis has many great use-cases like session or full page caching, queues, pub/sub and leaderboards/counting, etc. Usable in your applications and microservice architectures.

In this article, I show you how to use StackExchange.Redis in ASP.NET Core to access a Redis server running in Docker.

The example application lets users write posts in categories. It uses Redis to cache aggregated data for top categories and users while the database stores the item/line data as source of truth.

This article starts with a simple first use-case and then advances to a more complex one with Lua scripting and the inbox pattern.

Contents

1. Setup Redis in Docker and Prepare the .NET Core Project

2. Implement the Top Categories

3. Top Users, the Inbox Pattern, and Lua Scripting for Atomicity

4. Final Thoughts and Outlook

1. Setup Redis in Docker and Prepare the .NET Core Project

Install Docker Desktop

Create the Redis container:

C:\dev>docker run --name redis -d redis

Use Visual Studio Community (its free) with the ASP.NET and web development workload. Open an ASP.NET Core 5 Web API project with EntityFramework.

Install the following NuGet packages:

• Microsoft.EntityFrameworkCore.Tools

• StackExchange.Redis

Create the following entities:

2. Implement the Top Categories

Add Top Categories

Increment the count for a category when inserting a new post.

• Line 1: Connect a Redis multiplexer

• Lines 711: Insert the post and update the category count in a transaction

• Lines 1315: Read the latest count for the category and add/update the entry in the Redis sorted set CategoriesByPostCount

The ZADD command has an optional GT parameter: The value is only updated if it is greater than the current value. This prevents race conditions where another thread also increased and updated the count.

Read Top Categories

SortedSetRangeByRankWithScore is a wrapper for the Redis ZRANGE command. It reads the category and the count. Use the start and end parameters for paging.

3. Top Users, the Inbox Pattern, and Lua Scripting for Atomicity

For the users, I show you how to count the posts in Redis instead of the DB. This is e.g. needed in a sharded environment where the users posts are scattered over all shards.

Atomically Add Posts with Lua Scripting

The PublishOutstandingPosts method sends new posts to Redis. It uses the inbox pattern and a Lua script to make the operation idempotent.

Further reading: Outbox, Inbox patterns and delivery guarantees explained

• Line 38: Prepare the Lua script

• Line 13: Read the last sent post ID from Redis

• Line 15: Load the unsent posts from the DB

• Line 1721: Execute the Lua script to add the post to the user

• Line 22: Set the post count for the user in UsersByPostCount

• Line 25: Update the last sent ID after all operations are finished

The Lua script tries to add the post ID and timestamp to the PostsByTimestamp sorted set of the user.

If it can add the key, it also increments the PostCount of the user. The script makes the ZADD and ZINCRBY commands atomic. If the post ID already exists, it returns the existing PostCount.

An extra counter per user is needed so that all key parameters map to the same Redis hash tag. And therefore would be placed on the same Redis shard. The curly braces like in the key {User:5}:PostsByTimestamp are signifiers for a Redis hash tag.

Read the Top Users

Use SortedSetRangeByRankWithScore to read the IDs of the top users. Then read the names:

4. Final Thoughts and Outlook

You used StackExchange.Redis to access Redis from C#. See my previous article if you want to manually access Redis via redis-cli or want more information about the example data model and use-cases.

The code in this article focuses on the Redis-related part. The basis for the code can be found in the solution from my article about database sharding if you want to build and run the service.

There are many other use-cases for you to use Redis in your applications!

Please contact me if you have any questions, ideas, or suggestions.