Running an HTTP Server with AWS Nitro Enclaves

AWS Nitro enclaves are isolated execution environments. They allow us to compute on sensitive and/or private data.

In this blog we will look at how you can run an HTTP server in Go on a Nitro Enclave. Final code is available on Github.

Creating the Server

We will start with an HTTP server running outside of an enclave.

package mainimport (    "io"    "log"    "net/http")const PORT = ":8888"func main() {    handler := func(w http.ResponseWriter, req *http.Request) {        io.WriteString(w, "Hello, regular server!\\n")    }    http.HandleFunc("/", handler)    log.Println("listening on", PORT)    log.Fatal(http.ListenAndServe(PORT, nil))}

We will run and test our enclave in Docker. That will be the easiest way to get our code running in the enclave later. We can use the following Dockerfile.

FROM golang:1.18-alpineWORKDIR /appCOPY go.mod ./COPY go.sum ./RUN go mod downloadCOPY *.go ./RUN go build -o /enclave-serverEXPOSE 8888CMD [ "/enclave-server" ]

Set up an Enclave Capable EC2 Instance

Next up, lets run our server inside of an enclave. To do this, we need a Nitro Enclave capable EC2 instance. You can follow the instructions from AWS to get an image, and installing the Nitro Enclaves CLI

Building our Enclave Image File (EIF)

The Nitro CLI can turn Dockerfiles into Enclave Image Files. We will then pass the Enclave Image file to the run-enclave command.

We will first build our docker image

docker build -t hello-nitro .

Then create our EIF

nitro-cli build-enclave --docker-uri hello-nitro --output-file hello-nitro.eif

You should see an output that looks something like

Start building the Enclave Image...Using the locally available Docker image...Enclave Image successfully created.{  "Measurements": {    "HashAlgorithm": "Sha384 { ... }",    "PCR0": "bf9b5743dc00c63972e7cc06da44bda4a18e40a80173c5a5203651853509f37c1e4c6794fa028e29e60081b007175b09",    "PCR1": "bcdf05fefccaa8e55bf2c8d6dee9e79bbff31e34bf28a99aa19e6b29c37ee80b214a414b7607236edf26fcb78654e63f",    "PCR2": "ad8dfc09b5415f3a1d5586d7de878e87785669371a590aa4e392e14f2fc70aeba5bd53d04ef0043447e9ad04cd0c4893"  }}

Then we run our EIF

nitro-cli run-enclave --eif-path hello-nitro.eif --memory 2000 --cpu-count 2 --enclave-cid 16 --debug-mode

Note that we will run in debug mode for now. Debug mode lets us see console output. The enclave is not secure when we use debug mode, but it is very helpful with prototyping. We also ran the enclave with context identifier (CID) 16. This is not necessary but ensures that you can copy-paste some commands that refer to this CID below.

We can see that our enclave is running with the describe-enclaves command

$ nitro-cli describe-enclaves[  {    "EnclaveName": "hello-nitro",    "EnclaveID": "i-02d0ff88a0e47b51d-enc183d6a008ea4b9a",    "ProcessID": 26734,    "EnclaveCID": 16,    "NumberOfCPUs": 2,    "CPUIDs": [      1,      17    ],    "MemoryMiB": 2048,    "State": "RUNNING",    "Flags": "DEBUG_MODE",    "Measurements": {      "HashAlgorithm": "Sha384 { ... }",      "PCR0": "bf9b5743dc00c63972e7cc06da44bda4a18e40a80173c5a5203651853509f37c1e4c6794fa028e29e60081b007175b09",      "PCR1": "bcdf05fefccaa8e55bf2c8d6dee9e79bbff31e34bf28a99aa19e6b29c37ee80b214a414b7607236edf26fcb78654e63f",      "PCR2": "ad8dfc09b5415f3a1d5586d7de878e87785669371a590aa4e392e14f2fc70aeba5bd53d04ef0043447e9ad04cd0c4893"    }  }]

Note that the PCR 0, 1, and 2 output is the same from when we built the enclave. These values can be used during Attestation.

Finally, since we are using debug mode, we can see the console output of our enclave with the console command.

$ nitro-cli console --enclave-name hello-enclave<enclave setup output redacted>2022/10/14 13:12:41 listening on :8888

Seems easy enough, right? However, we have no way to request this endpoint. The only way to connect to a running enclave is through vsock. Great! How do we do that?

To connect be able to connect to our HTTP server, we will make two modifications to our setup:

• Our go server needs to listen on a vsock address. By default, http.ListenAndServe will listen on the provided TCP network address, which will be non-functional inside the enclave.
• We would like to be able to make requests to our server with standard tools. So far we have used cURL, but we would like any http client to work without modification. We will use [socat[(https://linux.die.net/man/1/socat) to maintain this compatibility.

Listen on vsock

We can use https://github.com/mdlayher/vsock to get a vsock implementation of net.Listener that we can pass to http.Serve.

package mainimport (    "io"    "log"    "net/http"    "github.com/mdlayher/vsock")func main() {    handler := func(w http.ResponseWriter, req *http.Request) {        io.WriteString(w, "Hello, Enclave!\\n")    }    listener, err := vsock.Listen(8888, nil)    if err != nil {        log.Fatal(err)    }    log.Fatal(http.Serve(listener, http.HandlerFunc(handler)))}

Proxy HTTP to vsock

Using socat, we can allow clients to continue to make HTTP requests, and socat will proxy them to vsock for us. An image is available on Docker Hub.

docker run -d -p 8888:8888 --name socat alpine/socat tcp-listen:8888,fork,reuseaddr vsock-connect:16:8888

If you are unfamiliar with socat, what this command is saying is to listen for tcp connections on port 8888, and forward them to vsock. The fork option makes this happen in a new child process, while the parent goes back to waiting for new connections (useful if you wish to handle concurrent requests). reuseaddr allows other sockets to bind to an address if parts of it are already in use by socat.

The second argument describes how to connect to vsock. 8888 matches up with the port we listened to in our http server. The 16 is the context identifier which identifies the enclave (we explicitly set this above).

You can find the address of your socat server by inspecting the bridge docker network. Mine was at 172.17.0.2.

Thats it! we can now request to our enclave

$ curl 172.17.0.2:9090Hello, Enclave!

Conclusion & Next Steps

We now have an HTTP server running inside of an enclave. We can now put confidential workloads behind these HTTP endpoints.

In future blogs, we will explore running our server on HTTPS, so callers have a secure connection directly into the enclave. We will also look at running our enclave capable EC2 instances on Kubernetes for a more production ready environment.

Shameless plug

Thanks for reading! I work full time on enclaves at https://capeprivacy.com/. We are trying to make enclaves easy to access and use for all developers and data scientists. Cape Privacy is currently in open beta. I would love to hear any feedback, comments, or questions you have on the product. Check it out! https://docs.capeprivacy.com/getting-started/