An Interest In:
Web News this Week
- April 2, 2024
- April 1, 2024
- March 31, 2024
- March 30, 2024
- March 29, 2024
- March 28, 2024
- March 27, 2024
Using AWS S3 to Store DevStream State
In our latest release v0.6.0, using AWS S3 to store DevStream state is supported.
In this blog, we are going to demonstrate the usage of AWS S3 to store DevStream's state files.
Terminology
State: if you don't already know about DevStream state, please read this doc and this blog helps, too.
Backend: where to actually store the state file. It can be either local
or s3
at the moment.
Configuration
In the main config file, we have a section to configure the state. Currently, local and S3 are supported.
Local example:
varFile: variables-gitops.yamltoolFile: tools-gitops.yamlstate: backend: local options: stateFile: devstream.state
S3 example:
varFile: variables-gitops.yamltoolFile: tools-gitops.yamlstate: backend: s3 options: bucket: devstream-remote-state region: ap-southeast-1 key: devstream.state
More on configuring state here.
In short, we can use the "backend" keyword to specify where to store the state: either locally or in an S3 bucket. If S3 is used, we need to specify the bucket, region, and the S3 key as well.
Config File Examples
In this demo, we use the following configs:
config.yaml
:
varFile: variables-gitops.yamltoolFile: tools-gitops.yamlstate: backend: s3 options: bucket: devstream-test-remote-state region: ap-southeast-1 key: devstream.state
variables-gitops.yaml
:
githubUsername: IronCore864repoName: dtm-test-godefaultBranch: maindockerhubUsername: ironcore864argocdNameSpace: argocdargocdDeployTimeout: 5m
tools-gitops.yaml
:
tools:- name: github-repo-scaffolding-golang instanceID: default options: owner: [[ githubUsername ]] org: "" repo: [[ repoName ]] branch: [[ defaultBranch ]] image_repo: [[ dockerhubUsername ]]/[[ repoName ]]
Getting Started
Before reading on, now is a good time to check if you have configured your AWS related environment variables correctly or not.
For macOS/Linux users, do:
export AWS_ACCESS_KEY_ID=ID_HEREexport AWS_SECRET_ACCESS_KEY=SECRET_HEREexport AWS_DEFAULT_REGION=REGION_HERE
For more information, see the official document here.
Apply
Then let's run dtm apply
:
tiexin@mbp ~/work/devstream-io/devstream $ ./dtm apply2022-05-30 17:07:59 [INFO] Apply started.2022-05-30 17:07:59 [INFO] Using dir <.devstream> to store plugins.2022-05-30 17:07:59 [INFO] Using s3 backend. Bucket: devstream-test-remote-state, region: ap-southeast-1, key: devstream.state.2022-05-30 17:08:00 [INFO] Tool (github-repo-scaffolding-golang/default) found in config but doesn't exist in the state, will be created.Continue? [y/n]Enter a value (Default is n): y2022-05-30 17:08:08 [INFO] Start executing the plan.2022-05-30 17:08:08 [INFO] Changes count: 1.2022-05-30 17:08:08 [INFO] -------------------- [ Processing progress: 1/1. ] --------------------2022-05-30 17:08:08 [INFO] Processing: (github-repo-scaffolding-golang/default) -> Create ...2022-05-30 17:08:12 [INFO] The repo dtm-test-go has been created.2022-05-30 17:08:29 [SUCCESS] Tool (github-repo-scaffolding-golang/default) Create done.2022-05-30 17:08:29 [INFO] -------------------- [ Processing done. ] --------------------2022-05-30 17:08:29 [SUCCESS] All plugins applied successfully.2022-05-30 17:08:29 [SUCCESS] Apply finished.
As we can see from the output, the S3 backend is used, and it also shows the bucket and key you are using, and in which region this bucket lives.
Checking the State File
After apply
, let's download the state file from S3 and check it out:
tiexin@mbp ~/work/devstream-io/devstream $ aws s3 cp s3://devstream-test-remote-state/devstream.state .
And if we open the downloaded file, we will see something similar to the following:
github-repo-scaffolding-golang_default: name: github-repo-scaffolding-golang instanceid: default dependson: [] options: branch: main image_repo: ironcore864/dtm-test-go org: "" owner: IronCore864 repo: dtm-test-go resource: org: "" outputs: org: "" owner: IronCore864 repo: dtm-test-go repoURL: https://github.com/IronCore864/dtm-test-go.git owner: IronCore864 repoName: dtm-test-go
which is exactly the same as if we were using the local backend to store state.
Like this quick tutorial? Then I suggest to read more of our latest DevOps blogs:
Original Link: https://dev.to/devstream/using-aws-s3-to-store-devstream-state-1dgj
Dev To
An online community for sharing and discovering great ideas, having debates, and making friendsMore About this Source Visit Dev To