An overview
In this blog post on MongoDB Replication on multiple AWS EC2 instances : Step by Step Guide, you will find the details about what and why of MongoDB replication, MongoDB replication internal and concepts, the detailed step by step process to setup and configure various settings over AWS EC2 instances for replication to work. You will be able to follow along the steps and do it yourself. If you don’t have any idea of how to setup AWS free tier account, you can watch my YouTube video on how to setup AWS free tier account. This can be followed by how to create AWS EC2 instances and connecting to AWS EC2 instances.
What is MongoDB Replication?
The MongoDB replication is the process of copying data from one MongoDB node to another MongoDB node. This process of replication is an asynchronous process i.e. the data is copied from primary node to the secondary node.
MongoDB Replication Benefits
The following section lists down the advantages of the MongoDB replication.
- The biggest advantage is the availability of the data almost all the time. This happens due to the automated failover capability of MongoDB i.e. it has the capability to trigger the election to select the new primary in case the existing primary goes down or unreachable due to any reason.
- Automated backup of the data
- The read capacity is increased in case of any use cases like reporting or reading data from nearest to the client.
Strategies for MongoDB Replica Set Deployment Architecture
- Decide the #members and maximum #voting members
- A replica set can have up to 50 members
- Max 7 voting members
- Deploy odd number of voting members
- Use hidden and delayed members for dedicated functions
- Add capacity ahead of demand
- Distribute members geographically
- Use journaling to protect against power failures
- Load balance on read-heavy deployments (use read preference)
Step by Step Guide to MongoDB Replication setup ( Complete concepts of MongoDB replication along with installation over multiple AWS EC2 instances steps are available over my YouTube channel as well. )
Step 1 : Launch EC2 instances (watch my YouTube video for launching EC2 instance ). The suggested count of instance is one EC2 instance each for every replica set member node. An instance for mongos process which will connect our client ( MongoDB compass to our sharded MongoDB cluster)
Step 2 : Configure security group to all the inbound traffic/connection on the MongoDB server port. The details of security group configuration can be found here.
Step 3 : Install MongoDB (all AWS EC2 Ubuntu instances) ( watch my YouTube video for installing MongoDB on Ubuntu )
Step 4: Replication setup (Ubuntu AWS EC2 instance):
The following command is very handy to find out any existing mongo process running on the connected EC2 instance (Ubuntu).
ps -aef | grep mongo
4.a create directories on all the EC2 instances
mkdir -p replicaset/member
4.b Start MongoDB with the following command on every EC2 instance. The following command starts MongoDB. The bold text in the following commands shows the keys followed by values. Please note that these keys must be specified while running the MongoDB server. nohup is used to keep the processes running after the terminal window is closed. “&” in the end of the command is used to ensure that the process runs in the background. Please replace the EC2 instances IP and public DNS entries with your own values.
nohup mongod –port 28041 –bind_ip localhost,ec2-3-86-210-11.compute-1.amazonaws.com –replSet replica_demo –dbpath replicaset/member &
nohup mongod –port 28042 –bind_ip localhost,ec2-3-82-236-231.compute-1.amazonaws.com –replSet replica_demo –dbpath replicaset/member &
nohup mongod –port 28043 –bind_ip localhost,ec2-44-201-173-85.compute-1.amazonaws.com –replSet replica_demo –dbpath replicaset/member &
4.c Connect to the any of the above mentioned instance. In this example, we connect to the first one.
mongosh –host 3.86.210.11 –port 28041
4.d Once connected successfully, create the configuration as in the following step.
rsconf = {
_id: “replica_demo”,
members: [
{
_id: 0,
host: “3.86.210.11:28041”
},
{
_id: 1,
host: “3.82.236.231:28042”
},
{
_id: 2,
host: “44.201.173.85:28043”
}
]
}
4.e Initiate the configuration, check the status and view the configuration.
rs.initiate(rsconf)
rs.status()
rs.conf()
4.f Add an arbiter (if required). An arbiter doesn’t hold the data. It can participate in the election. It is used sometimes due to cost constraints as there is no budget to provision the required number of nodes for MongoDB replication. All such concepts have been covered in my YouTube video on this subject. Create a new EC2 instance. This can be setup over application server or some other server as well.
Create directory:
mkdir -p replicaset/member
Start MongoDB server
nohup mongod –port 27017 –bind_ip localhost,ec2-44-202-164-113.compute-1.amazonaws.com –replSet replica_demo –dbpath replicaset/member &
Add/Remove the arbiter with the following commands:
rs.addArb(“ec2-44-202-164-113.compute-1.amazonaws.com:27017”)
rs.remove(“ec2-44-202-164-113.compute-1.amazonaws.com:27017”)
OR
rs.addArb(“44.202.164.113:27017”)
rs.remove(“44.202.164.113:27017”)
If required, follow the following commands to add/ remove a new replica set member to/from the cluster respectively.
rs.add( { host: “ec2-44-202-164-113.compute-1.amazonaws.com:27017” } )
rs.remove(“ec2-44-202-164-113.compute-1.amazonaws.com:27017”)
You can ingest the data by connecting to the primary node from either MongoDB shell or MongoDB compass or your application. If everything is followed correctly, your data will be replicated to other nodes as well.
Hope, you have found this article useful. You can share this article or my YouTube video with your friends as well.
You can keep your MongoDB learning journey going and enhance your skills by following my other posts and YouTube videos.
MongoDB Sharding on AWS
MongoDB Atlas – running MongoDB on cloud
Learn about MongoDB Schema design best practices here
My other videos on AWS cloud include:
How to create a free AWS account and what are the services offered by AWS free account
How to launch EC2 instance in AWS
How to connect AWS EC2 instances from desktop or laptop
You can connect with me on following social platforms as well.
https://www.youtube.com/c/LearnwithNeerajGarg
https://www.linkedin.com/in/neerajgarg5/
https://www.facebook.com/neerajgarg5/
