Glusterfs 3.2 Updates
At our office, we’ve been using Glusterfs in an unconventional way. Instead of merely using it in a distributed or replicated cluster, we’re using it as central storage for our user home directories.
As part of our office-wide upgrade to Ubuntu 12.04LTS, we have had to upgrade our Glusterfs from 3.0 to 3.2 and it was not immediately evident how we could accomplish the same setup as we had previously. Now, it’s been sorted out and we’d like to share our setup.
Let’s assume that there are four workstations with one file server. The four workstations have the IP address 18.104.22.168-104 and the file server has the IP 192.168.1.100
On the file server, do the following:
# gluster peer probe 192.168.1.101
# gluster peer probe 192.168.1.102
# gluster peer probe 192.168.1.103
# gluster peer probe 192.168.1.104
# gluster peer status
You should see that there are 5 machines in the cluster. On each workstation and server, we will use the /data/export directory for storage. However, we need to make a few adjustments to the server to make our scheme work.
# ln -s /data /data101
# ln -s /data /data102
# ln -s /data /data103
# ln -s /data /data104
From here on, things are going to get a little weird but these instructions can be replicated on each of the workstations without any problems.
# gluster volume create vol101 replica 2 192.168.1.101:/data/export 192.168.1.100:/data101/export
# gluster volume start vol101
# mount -t glusterfs 192.168.101:/vol101 /home
That’s it. That will mount the gluster file-system to the home directory.
Doing things this way allows the server to act as a mirror for the data in the home directory. By replicating the setup across different workstations, the server also acts as a shared storage for the cluster.
This gives us the power of automatic replication and recovery using the Gluster AFR mechanism with self-healing features. It also gives us the advantage of a shared storage that allows a user’s data to be accessed from any workstation. It also gives us the performance boost of having to only read/write to the local machine instead of over the network.
Any change that is made on one workstation, will be distributed to the other workstations via the central server.
This scheme may be heresy for some but it’s served us well for more than a year.