Prometheus Server Setup,Adding Node for Monitoring that OS & Also Monitor Docker metrics if it is running on Any Server
So we will go step by step so steps are:
1. First we will setup the Prometheus Server
2. Then we setup the node & adding into the prom server
3. After that we will go for the collecting docker metrics with the help of Prom
Prometheus URL(Linux):- https://github.com/prometheus/prometheus/releases/download/v2.37.0-rc.1/prometheus-2.37.0-rc.1.linux-amd64.tar.gz
Node_Exporter URL(Linux):- https://github.com/prometheus/node_exporter/releases/download/v1.3.1/node_exporter-1.3.1.linux-amd64.tar.gz
So let’s start
1. For setup the promethues server we need a OS where we will install or do the all the setup, so in my case i used a Linux OS which is running over AWS.
So first we will connect & then run a simple command to download the .tar file of prometheus, so for finding the URL of prometheus you simply type on google that is prometheus download and open first link then select the OS where you want to setup your prometheus server
After selecting your OS you will see there is two URL come up in the form of versions so you can pick-up any one of them and copy the URL based on your choice
After copying the URL go to your Instance or OS and put the URL with wget it will download the .tar file of Prometheus
After downloading complete you need to unzip that file so run the command
After that you will see the unzip file come up likewise this
Then go to inside that file with the help of “cd” command then do “ls” you will see a executable file of Prometheus you need to run that file with help of this command “./prometheus” and if you want to run in background you can use this command “./prometheus &”
After that if you will see the prometheus server running on port 9090 so for connecting the prometheus you need to run the IP with the port 9090 then you will see the actual dashboard of prometheus.
After that if you will click on Status you will see there is one option coming that is Target click on that
After clicking you will see there one single target that is prometheus itself
2. Now we are going to add the another OS for monitoring that OS, so in the prometheus world that node is called node exporter. So first install the node exporter into that OS(node) for this you need to go again on the same page where you were find the prometheus URL so go to down of the page you will see the node exporter copy that URL
Go to your OS(node) use the same command that is wget and paste the URL
After downloading complete use same as you in the prom server unzip the node exporter .tar file & go inside that file
After that use same command to run the node_exporter that is “./node_exporter &”
The node_exporter run on the port 9100 also if wants to see the 9100 port is running or not run the command that is “netstat -tnlp” yu will see 9100 port is running. So now you need to run that OS IP with same port that is 9100 you will find the metrics of that node.
first screen is like this but if you will clicked on metrics you will find all the metrics of that OS or node.
Now final step is to add that node into the prometheus server so basically prometheus has on configuration file that prometheus.yml it will store all the details of node or servernwe can say so first i will show you how it is look like.
If you go to prom server on the same file and do “ls” you will see a file that file so you need to open that file with help of “vi prometheus.yml” command.
So now as you can see there is single job is running that is prometheus on the port of 9090 and prometheus by itslef taking localhost for the master server.So now we need to add one another job that is for node give the job name based on your choice we added that with that IP & port no.
After adding that save the file come out from & run some command to detect that changes so it is “./prometheus — config.file=prometheus.yml &”.
Might be you will facing one issue that is port 9090 is already in use then do one thing need to kill the prom process,so first run one command that is “pgrep prom” you will find that process with some PID no, kill that PID with command “kill -9 PID” or you can also use one another command which is restart your process that is “kill -HUP PID”.
Now you will go to prometheus server dashboard and refresh once you will see node has been come up.
Now if you wants to run the query based on your requirements as you can in the header there is one option come up that is Graph clicked on that you will find two things one is table & graph just above there is search option where you will search the query and run your query. for example i run this
For graph, clicked on the graph and you will see the graph of that query.
3. Now we will go for docker there is simple setup for collect the metrics of docker through the prometheus, for your refrence i am running running some command for confirmation that docker is running that is “systemctl status docker”.
After that docker has one file that is daemon.json so for collecting the metrics need to put some in that file so first open that file use this command that is “vi /etc/docker/daemon.json” and put this in your daemon file {
"metrics-addr" : "0.0.0.0:9323",
"experimental" : true
}
I used 0.0.0.0 because of open connection likewise
save that file and restart the docker using the command that is “systemctl restart docker”. After restarting the docker run one command that is “netstat -tnlp” you will see 9323 port is running. So now next step to add the docker into the prometheus server using same way but remember port is 9323.
After save the file run same command to restart the process “kill -HUP PID” and you can also run the command “./prometheus —config.file=prometheus.yml”. And go to the dashboard of prom server.
And now finally now you can monitor your docker also, collect your metrics of docker. You are seeing one more node is in my dashboard because of i added one more that’s why.
So this was a beautiful journey with You,Thanks for Reading😍
