An Interest In:
Web News this Week
- April 23, 2024
- April 22, 2024
- April 21, 2024
- April 20, 2024
- April 19, 2024
- April 18, 2024
- April 17, 2024
January 28, 2022 05:40 pm GMT
Original Link: https://dev.to/ductnn/mongodb-cluster-on-docker-5ghh
MongoDB cluster on Docker
## MongoDB Cluster on Docker
A simple implement MongoDB Cluster on Docker with PSS Style (Primary - Secondary - Secondary). Inspired by this article
Source code in here
Components
- 2 Mongos (router): The mongos acts as query routers, providing an interfacebetween client applications and the sharded cluster.
router01
router02
- 3 Config Servers: Store metadata and configuration settings for the cluster
configsvr01
configsvr02
configsvr03
- 3 Shards (each a 3 member replica set): Each shard contains a subset ofthe sharded data. Each shard can be deployed as a replica set.
shard01-a
,shard01-b
,shard01-c
shard02-a
,shard02-b
,shard02-c
shard03-a
,shard03-b
,shard03-c
Installation
Clone this repo:
git clone https://github.com/ductnn/mongodb-cluster-docker.gitcd mongodb-cluster-docker
And run docker-compose.yaml
:
mongodb-cluster-docker git:(master) docker-compose up -dCreating network "mongodb-cluster-docker_default" with the default driverPulling router01 (mongo:5.0.3)...5.0.3: Pulling from library/mongo7b1a6ab2e44d: Already exists90eb44ebc60b: Already exists5085b59f2efb: Already existsc7499923d022: Already exists019496b6c44a: Already existsc0df4f407f69: Already exists351daa315b6c: Already exists557b07ecd9d7: Already existsa2dff157a5e3: Already exists07d83e88231b: Already existsDigest: sha256:07212fb304ea36b8c5a9e5694527f16deeb0b99f87fc60162dc15ab260bf8a2aStatus: Downloaded newer image for mongo:5.0.3Creating shard-02-node-b ... doneCreating shard-03-node-b ... doneCreating shard-03-node-c ... doneCreating shard-02-node-c ... doneCreating router-01 ... doneCreating shard-01-node-b ... doneCreating shard-01-node-c ... doneCreating shard-03-node-a ... doneCreating shard-01-node-a ... doneCreating shard-02-node-a ... doneCreating router-02 ... doneCreating mongo-config-01 ... doneCreating mongo-config-02 ... doneCreating mongo-config-03 ... done
Result:
mongodb-cluster-docker git:(master) docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES302243ef45a3 mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 15 minutes 0.0.0.0:27121->27017/tcp, :::27121->27017/tcp mongo-config-034599103daf74 mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27120->27017/tcp, :::27120->27017/tcp mongo-config-025be686736dd8 mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27119->27017/tcp, :::27119->27017/tcp mongo-config-014eff0c607d3c mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27118->27017/tcp, :::27118->27017/tcp router-02b28d39a0a262 mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27125->27017/tcp, :::27125->27017/tcp shard-02-node-aad9c4985f630 mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27122->27017/tcp, :::27122->27017/tcp shard-01-node-a5a43e22ff9ae mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27128->27017/tcp, :::27128->27017/tcp shard-03-node-a52520b1b29a9 mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27124->27017/tcp, :::27124->27017/tcp shard-01-node-c9591f85f2dc0 mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27130->27017/tcp, :::27130->27017/tcp shard-03-node-c1b745802b88a mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27126->27017/tcp, :::27126->27017/tcp shard-02-node-b161baac14972 mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27117->27017/tcp, :::27117->27017/tcp router-014468e7f23da1 mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27123->27017/tcp, :::27123->27017/tcp shard-01-node-b1b99d3d5ea54 mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27129->27017/tcp, :::27129->27017/tcp shard-03-node-b0480f8b2247d mongo:5.0.3 "docker-entrypoint.s" 16 minutes ago Up 16 minutes 0.0.0.0:27127->27017/tcp, :::27127->27017/tcp shard-02-node-c
Then, we need to init config-servers
, shards
and router
:
Config servers
# Initilizing the config server mongodb-cluster-docker git:(master) bash bash/init-configserver.sh MongoDB shell version v5.0.3connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodbImplicit session: session { "id" : UUID("555e23b7-f1bd-4232-9947-8f5dd0db6c3b") }MongoDB server version: 5.0.3================Warning: the "mongo" shell has been superseded by "mongosh",which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed inan upcoming release.We recommend you begin using "mongosh".For installation instructions, seehttps://docs.mongodb.com/mongodb-shell/install/================{ "ok" : 1, "$gleStats" : { "lastOpTime" : Timestamp(1636305337, 1), "electionId" : ObjectId("000000000000000000000000") }, "lastCommittedOpTime" : Timestamp(0, 0)}bye
Shards
# Initilzing the shards mongodb-cluster-docker git:(master) bash bash/init-replicaset.sh MongoDB shell version v5.0.3connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodbImplicit session: session { "id" : UUID("815f9a5c-997b-48a6-bba0-8847b0890521") }MongoDB server version: 5.0.3================Warning: the "mongo" shell has been superseded by "mongosh",which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed inan upcoming release.We recommend you begin using "mongosh".For installation instructions, seehttps://docs.mongodb.com/mongodb-shell/install/================{ "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1636305394, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1636305394, 1)}byeMongoDB shell version v5.0.3connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodbImplicit session: session { "id" : UUID("16424801-e037-450b-b4db-4d9dfe19752b") }MongoDB server version: 5.0.3================Warning: the "mongo" shell has been superseded by "mongosh",which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed inan upcoming release.We recommend you begin using "mongosh".For installation instructions, seehttps://docs.mongodb.com/mongodb-shell/install/================{ "ok" : 1 }byeMongoDB shell version v5.0.3connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodbImplicit session: session { "id" : UUID("6f621971-1d81-45c6-9f74-23eacc9cc4a1") }MongoDB server version: 5.0.3================Warning: the "mongo" shell has been superseded by "mongosh",which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed inan upcoming release.We recommend you begin using "mongosh".For installation instructions, seehttps://docs.mongodb.com/mongodb-shell/install/================{ "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1636305395, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1636305395, 1)}bye
Routers
After initilizing config-server
and shards
, wait a bit to to elect primaries
before initilizing router
:
mongodb-cluster-docker git:(master) bash bash/init-router.sh MongoDB shell version v5.0.3connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodbImplicit session: session { "id" : UUID("25a1c2a2-135c-4550-b732-713e018eb0a2") }MongoDB server version: 5.0.3================Warning: the "mongo" shell has been superseded by "mongosh",which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed inan upcoming release.We recommend you begin using "mongosh".For installation instructions, seehttps://docs.mongodb.com/mongodb-shell/install/================{ "shardAdded" : "rs-shard-01", "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1636305656, 6), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1636305656, 5)}{ "shardAdded" : "rs-shard-02", "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1636305660, 2), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1636305658, 4)}{ "shardAdded" : "rs-shard-03", "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1636305662, 9), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1636305662, 3)}bye
Verify status
We can check the status of replicasets:
# Exec to the shard mongodb-cluster-docker git:(master) docker exec -it shard-01-node-a bash root@ad9c4985f630:/# mongors-shard-01:PRIMARY> rs.status(){ "set" : "rs-shard-01", "date" : ISODate("2021-11-07T17:31:01.326Z"), "myState" : 1, "term" : NumberLong(1), "syncSourceHost" : "", "syncSourceId" : -1, "heartbeatIntervalMillis" : NumberLong(2000), "majorityVoteCount" : 2, "writeMajorityCount" : 2, "votingMembersCount" : 3, "writableVotingMembersCount" : 3, "optimes" : { "lastCommittedOpTime" : { "ts" : Timestamp(1636306261, 9), "t" : NumberLong(1) }, "lastCommittedWallTime" : ISODate("2021-11-07T17:31:01.283Z"), "readConcernMajorityOpTime" : { "ts" : Timestamp(1636306261, 9), "t" : NumberLong(1) }, "appliedOpTime" : { "ts" : Timestamp(1636306261, 9), "t" : NumberLong(1) }, "durableOpTime" : { "ts" : Timestamp(1636306261, 8), "t" : NumberLong(1) }, "lastAppliedWallTime" : ISODate("2021-11-07T17:31:01.283Z"), "lastDurableWallTime" : ISODate("2021-11-07T17:31:01.278Z") }, "lastStableRecoveryTimestamp" : Timestamp(1636306245, 14), "electionCandidateMetrics" : { "lastElectionReason" : "electionTimeout", "lastElectionDate" : ISODate("2021-11-07T17:16:45.266Z"), "electionTerm" : NumberLong(1), "lastCommittedOpTimeAtElection" : { "ts" : Timestamp(0, 0), "t" : NumberLong(-1) }, "lastSeenOpTimeAtElection" : { "ts" : Timestamp(1636305394, 1), "t" : NumberLong(-1) }, "numVotesNeeded" : 2, "priorityAtElection" : 1, "electionTimeoutMillis" : NumberLong(10000), "numCatchUpOps" : NumberLong(0), "newTermStartDate" : ISODate("2021-11-07T17:16:45.299Z"), "wMajorityWriteAvailabilityDate" : ISODate("2021-11-07T17:16:45.957Z") }, "members" : [ { "_id" : 0, "name" : "shard01-a:27017", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 1211, "optime" : { "ts" : Timestamp(1636306261, 9), "t" : NumberLong(1) }, "optimeDate" : ISODate("2021-11-07T17:31:01Z"), "syncSourceHost" : "", "syncSourceId" : -1, "infoMessage" : "", "electionTime" : Timestamp(1636305405, 1), "electionDate" : ISODate("2021-11-07T17:16:45Z"), "configVersion" : 2, "configTerm" : 1, "self" : true, "lastHeartbeatMessage" : "" }, { "_id" : 1, "name" : "shard01-b:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 867, "optime" : { "ts" : Timestamp(1636306260, 9), "t" : NumberLong(1) }, "optimeDurable" : { "ts" : Timestamp(1636306260, 9), "t" : NumberLong(1) }, "optimeDate" : ISODate("2021-11-07T17:31:00Z"), "optimeDurableDate" : ISODate("2021-11-07T17:31:00Z"), "lastHeartbeat" : ISODate("2021-11-07T17:31:01.263Z"), "lastHeartbeatRecv" : ISODate("2021-11-07T17:31:01.252Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncSourceHost" : "shard01-a:27017", "syncSourceId" : 0, "infoMessage" : "", "configVersion" : 2, "configTerm" : 1 }, { "_id" : 2, "name" : "shard01-c:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 867, "optime" : { "ts" : Timestamp(1636306260, 9), "t" : NumberLong(1) }, "optimeDurable" : { "ts" : Timestamp(1636306260, 9), "t" : NumberLong(1) }, "optimeDate" : ISODate("2021-11-07T17:31:00Z"), "optimeDurableDate" : ISODate("2021-11-07T17:31:00Z"), "lastHeartbeat" : ISODate("2021-11-07T17:31:01.263Z"), "lastHeartbeatRecv" : ISODate("2021-11-07T17:31:01.248Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncSourceHost" : "shard01-a:27017", "syncSourceId" : 0, "infoMessage" : "", "configVersion" : 2, "configTerm" : 1 } ], "ok" : 1, "$gleStats" : { "lastOpTime" : Timestamp(0, 0), "electionId" : ObjectId("7fffffff0000000000000001") }, "lastCommittedOpTime" : Timestamp(1636306261, 9), "$configServerState" : { "opTime" : { "ts" : Timestamp(1636306261, 11), "t" : NumberLong(-1) } }, "$clusterTime" : { "clusterTime" : Timestamp(1636306261, 11), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1636306261, 9)}
Similar with another shards
. Check the status of router
:
mongodb-cluster-docker git:(master) docker-compose exec router01 mongo --port 27017MongoDB shell version v5.0.3connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodbImplicit session: session { "id" : UUID("69408b5a-d313-44ae-ae64-81e9d3d21588") }MongoDB server version: 5.0.3================Warning: the "mongo" shell has been superseded by "mongosh",which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed inan upcoming release.We recommend you begin using "mongosh".For installation instructions, seehttps://docs.mongodb.com/mongodb-shell/install/================---The server generated these startup warnings when booting: 2021-11-07T17:10:47.048+00:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted---mongos> sh.status()--- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("618809c41517a9e726602e13") } shards: { "_id" : "rs-shard-01", "host" : "rs-shard-01/shard01-a:27017,shard01-b:27017,shard01-c:27017", "state" : 1, "topologyTime" : Timestamp(1636305656, 2) } { "_id" : "rs-shard-02", "host" : "rs-shard-02/shard02-a:27017,shard02-b:27017,shard02-c:27017", "state" : 1, "topologyTime" : Timestamp(1636305658, 2) } { "_id" : "rs-shard-03", "host" : "rs-shard-03/shard03-a:27017,shard03-b:27017,shard03-c:27017", "state" : 1, "topologyTime" : Timestamp(1636305662, 1) } active mongoses: "5.0.3" : 2 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: yes Collections with active migrations: config.system.sessions started at Sun Nov 07 2021 17:33:54 GMT+0000 (UTC) Failed balancer rounds in last 5 attempts: 0 Migration results for the last 24 hours: 237 : Success databases: { "_id" : "config", "primary" : "config", "partitioned" : true } config.system.sessions shard key: { "_id" : 1 } unique: false balancing: true chunks: rs-shard-01 787 rs-shard-02 118 rs-shard-03 119 too many chunks to print, use verbose if you want to force print
Done ... wait. You can remove all:
mongodb-cluster-docker git:(master) bash clean_all.sh Stopping mongo-config-03 ... doneStopping mongo-config-02 ... doneStopping mongo-config-01 ... doneStopping router-02 ... doneStopping shard-02-node-a ... doneStopping shard-01-node-a ... doneStopping shard-03-node-a ... doneStopping shard-01-node-c ... doneStopping shard-03-node-c ... doneStopping shard-02-node-b ... doneStopping router-01 ... doneStopping shard-01-node-b ... doneStopping shard-03-node-b ... doneStopping shard-02-node-c ... doneRemoving mongo-config-03 ... doneRemoving mongo-config-02 ... doneRemoving mongo-config-01 ... doneRemoving router-02 ... doneRemoving shard-02-node-a ... doneRemoving shard-01-node-a ... doneRemoving shard-03-node-a ... doneRemoving shard-01-node-c ... doneRemoving shard-03-node-c ... doneRemoving shard-02-node-b ... doneRemoving router-01 ... doneRemoving shard-01-node-b ... doneRemoving shard-03-node-b ... doneRemoving shard-02-node-c ... doneRemoving network mongodb-cluster-docker_defaultRemoving image mongo:5.0.3
And so real done =))))
Show your support
Give a if you like this application
Contribution
All contributions are welcomed in this project!
License
The MIT License (MIT). Please see LICENSE for more information.
Original Link: https://dev.to/ductnn/mongodb-cluster-on-docker-5ghh
Share this article:
Tweet
View Full Article
Dev To
An online community for sharing and discovering great ideas, having debates, and making friendsMore About this Source Visit Dev To