Having a cluster built out of Raspberry Pi's is not only fun but also eases your work. As we discussed in our latest article in the series you can use the cluster to compile software or make it gather data from various sources on the Internet. There are many uses for such a thing.
However, after compiling for an hour straight you might get curious as of how your nodes perform. How they work under load, if they're not under-powered or if the CPU temperature is not rising above desired levels. How much memory have you got left in each of the nodes while they're doing tasks. The more nodes you have in your rack, the more information you will have to go through. Luckily there is software out there that can help you visualise such information all at once in various ways.In this tutorial you will learn:
- How to install the
- How to configure glances
- How to use
glancesto see cluster information
- How to create a local webpage to display the
Software Requirements and Conventions UsedSoftware Requirements and Linux Command Line Conventions Category Requirements, Conventions or Software Version Used System Raspbian Linux Software Glances Other Privileged access to your Linux system as root or via the sudo command. Conventions # - requires given linux commands to be executed with root privileges either directly as a root user or by use of sudo command $ - requires given linux commands to be executed as a regular non-privileged user
Glances - the must-have in cluster monitoring
Think of glances as a
top for clusters. It's more than that. It's a Swiss army knife for monitoring clusters. It can display everything you need in one terminal window: hardware temperature, fan speed, running processes, RAM and CPU usage, node IP and several other interesting, smaller gems. The software is based on
ncurses and it will also let you browse your cluster nodes easily, configure what the information window displays and it even has a webserver mode for you to output that information in a webpage.
The software is based on
Python. There are several ways to install it. Most Linux distributions already have it in their repositories so in Ubuntu you would simply do a
$ sudo apt install glances
and all the package dependencies would be resolved. If you would rather do it the Python way, you can use
$ sudo apt install pip $ pip install glances
Glances has a ton of modules such as RAID information, WiFi network information, web interface or Docker integration. To be sure you get them all, should you use
pip, install glances with
$ pip install 'glances[action,browser,cloud,cpuinfo,docker,export,folders,gpu,graph,ip,raid,snmp,web,wifi]'
You can also use a setup script that installs
glances on your system regardless of what distribution you use. It downloads the latest master branch from GitHub and sets it up on your system. For this use
$ curl -L https://bit.ly/glances | /bin/bash
Also be sure to install
glances on all the cluster nodes.
glances creates a configuration file in
glances.conf. In this file you can fine-tune what and how the ncurses interface will show you. For example, if you want to use a white background instead of a black background in the system information screen you can modify the line
The ncurses interface uses colors so if you want to be visually allerted by an overloading CPU you can define at what percent the CPU usage bar will turn from green to red. For this modify the value of
cpu_critical=90 to something between
99. RAM, SWAP, Load thresholds all have the same parameters:
careful=50 warning=70 critical=90
You can modify their values so that their respective bars will change colors only then that limit defined by the value is reached. Uncomment the
Sensors alias lines in the
[sensors] section to get information about CPU and motherboard temperature. For this you will also need to have the
psutil packages installed on each of the Raspbian cluster nodes:
$ sudo apt install hddtemp python-psutil
The list of configurable elements is extensive but the
/etc/glances/glances.conf file is well documented and self-explanatory and would require an article of its own. But let us see what you can do with
There are several ways you can use glances. If you want to see information about a particular cluster node and you happen to be connected to it via SSH you can simply invoke the command with
If you want to start
glances in server mode on a cluster node you append the
$ glances -s
Then on another machine you can start
glances as a simple client application and connect to the
glances server you just started with
$ glances -c 192.168.x.x
192.168.x.x is the Glances server's IP. Alternatively you can use hostnames if they are defined in your
$ glances -c rpi1
You can define your cluster nodes in the configuration file. Look for the
Client/server section in
glances.conf and add your nodes:
[serverlist] # Define the static servers list server_1_name=192.168.1.126 server_1_alias=rpi1 server_1_port=61209 server_2_name=192.168.1.252 server_2_alias=rpi2 server_2_port=61209 server_3_name=192.168.1.150 server_3_alias=rpi3 server_3_port=61209 server_4_name=192.168.1.124 server_4_alias=rpi4 server_4_port=61209
As you can see, each server has a number, a name, an alias and a port. The port should remain
61209 for all nodes while the alias should correspond to the hostname of each cluster node IP.
When using multiple nodes, to avoid entering their login password each time you connect to a
glances server you can define that password in plaintex in the
glances configuration file. Edit
/etc/glances/glances.conf and add your cluster nodes to the
rpi1=yourpassword rpi2=yourpassword rpi3=yourpassword rpi4=yourpassword
yourpassword with the password you set in part three of this series and save the file.
Now you can call
glances from a laptop with
$ glances --browser
and see all your cluster nodes in a browsable ncurses list. You can enter each of them, see what each one is doing with its resources and processes then move on to the next.
But what if you wanted to view all of your cluster nodes at once? Would that be nicer? And wouldn't it be even better to display that information in a webpage? Glances has a webserver mode that you can start with:
$ glances -w
Once a Glances server is activated as a webserver you can use any web browser to access a webpage detailing the same information you can view in the ncurses interface. Just input the node's IP address or hostname in the URL bar of your browser and use
61208 as a port.
http://192.168.1.252:61208 will, in our LAN, display the Glances webserver's information pertaining the second node of the cluster. The problem is you will have to use different pages and addresses for each node. So why not make a single webpage that displays all the four nodes at once using a simple HTML page with
Creating a Glances webpage monitoring station
Install Apache on the first node of the cluster. We will use
rpi1 as the node that displays this HTML file. You can do this with
$ sudo apt install apache2
The root of the Apache webserver is the
/var/www/html/ folder. Create a new file here and name it
# touch /var/www/html/index.html
Now edit this file:
# nano /var/www/html/index.html
Add this to its contents:
out.println("<!DOCTYPE html>"); out.println("<html>"); out.println("<body>"); out.println("<h2>My cluster info"</h2>"); out.println("<br>"); out.println("<center>"); out.println("<iframe src="http://rpi1:61208" height="465" width="326"></iframe>"); out.println("<iframe src="http://rpi2:61208" height="465" width="326"></iframe>"); out.println("<iframe src="http://rpi3:61208" height="465" width="326"></iframe>"); out.println("<iframe src="http://rpi4:61208" height="465" width="326"></iframe></center>"); out.println("</br>"); out.println("</body>"); out.println("</html>");
Save the file with
ctrl+x. As you can see we are using hostnames instead of IPs and the Glances webserver port -
61208. There are four HTML
IFRAMES, one for each cluster node. Change the
width values so that the iframes will fill your screen.
Now just one more thing to do. Add the following line to each
/etc/rc.local file of your cluster nodes:
glances -w --theme-white --process-short-name --hide-kernel-threads --fs-free-space --disable-irq --disable-raid --disable-folder --disable-swap
This tells Glances to start at boot as a webserver, to use the white theme, trim the process list width it displays and hide a few things that a Raspberry Pi does not need or that you would rarely use: RAID information, IRQ information, certain folder information, SWAP and kernel thread information. This is so that after you reboot your cluster and access the first node's address through a web browser, the
IFRAMES would be able to fit all relevant information in their respective windows.
You can further decide how the Glances webserver webpages display information by editing their respective CSS file located at
/usr/lib/python3/dist-packages/glances/outputs/static/public/css/style.css. Font size, font type, colors - all can be customized here to make the cluster information webpage prettier.
Glances is a great way to get relevant information out of your cluster. If you use the
IFRAME webpage method you can visit the cluster information page at any time and see how all the nodes are performing in one browser window. Or you can use the
ncurses interface in a Terminal screen to view the same information. Glances is highly customizable and highly useful when you are using a cluster, be it a small one made out of Raspberry Pi's.
This is a companion discussion topic for the original entry at https://linuxconfig.org/building-a-raspberry-pi-cluster-part-iv-monitoring