Clusterfox: the raspberry pi cluster

by

in

The Raspberry Pi, a small yet powerful device, has become a popular choice for various DIY projects, including building web servers. This blog post explores how one can create a Raspberry Pi cluster using spare Raspberry Pi you might have laying around. The end result became the very server hosting this entire blog!

Motivation:

After pouring money into hosting services, I decided it was high time to switch to self-hosting and cut costs. Coincidentally, I had a bunch of Raspberry Pis lying around unused. So, I rolled up my sleeves and got to work on setting them up.

Step 1: Gathering the Hardware

The first step in creating a Raspberry Pi cluster is gathering all the necessary hardware. For this project, multiple Raspberry Pi 2 units were used, an old 5-port gigabit ethernet switch was salvaged, and a 5v power supply. Other things were also used like wires and standoffs.

Next, to put it all together, a 3D design was made to encapsulate the power supply and mount the PCB of the switch along with the tower of Pis.

Step 2: Setting Up the Cluster

Following a guide from the official Raspberry Pi website, the cluster was set up. This involved configuring each Raspberry Pi to work in tandem with the others, distributing the workload among them.

Step 3: Installing Docker Swarm

Once the cluster was operational, Docker Swarm was installed on each node. Docker Swarm is an orchestration tool that allows for the management of a group of Docker hosts, and it’s ideal for deploying applications across a cluster.

Step 4: Deploying Services

With Docker Swarm in place, services could be deployed to the cluster. WordPress, a popular content management system, was chosen for the blog site. It was installed as a service within the Docker Swarm environment, allowing for easy scaling and management.

Since the WordPress instance requires the use of images stored on file disk, I needed to come up with a way to share the same content across multiple nodes. I spent hours trying to get glusterfs to work in these pis, but unfortunately, the performance wasn’t great. The website would take around 5 seconds to load per refresh, which I consider unbearable.

At the end of the day, I managed to setup a simple file sync using union, where files get synchronized every minute across all nodes in case there are changes. Which only happen when a new image or plugins are updated.

Step 5: Managing Traffic with Traefik

To handle incoming traffic efficiently, Traefik was used. Traefik is a modern HTTP reverse proxy and load balancer that integrates with Docker Swarm to manage network traffic to the services. It also handles TLS termination and Certificate management.

Conclusion

The result is a robust, scalable web server cluster capable of hosting a blog site, with the flexibility to add more services as needed. This project demonstrates the versatility of Raspberry Pi clusters for web hosting and the power of containerization with Docker Swarm.

By following a detailed guide and utilizing tools like Docker Swarm and Traefik, even spare Raspberry Pi 2 units can be transformed into a sophisticated web server cluster. This setup not only provides a cost-effective solution but also offers a great learning experience for those interested in cluster computing and web hosting.

One can go even further, and setup something like K3s in place of Swarm and start learning Kubernetes from your own lab.

Remember, the key to success in such a project is careful planning, following best practices, and a willingness to experiment. Happy clustering!


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *