OCFS2

The Oracle Clustered File System is a file system which understand being mounted on, and written to by, more than one computer at the same time. It's a bit like NFS, except it's local to a machine rather than mounted over a network.

I use it with DRBD to get a network-replicated file system mounted on two machines at the same time, which either can read from and write to.

1. install the package ocfs2-tools
2. create the cluster definition file

   /etc/ocfs2/cluster.conf

   node:
     ip_port = 7777
     ip_address = 203.0.113.20
     number = 20
     name = Pinky
     cluster = SharedFS
   
   node:
     ip_port = 7777
     ip_address = 203.0.113.16
     number = 16
     name = Perky
     cluster = SharedFS
   
   cluster:
     node_count = 2
     name = SharedFS

3. edit the configuration file by running dpkg-reconfigure ocfs2-tools
4. check the status by running /etc/init.d/o2cb status

This should show something like:

Driver for "configfs": Loaded
Filesystem "configfs": Mounted
Stack glue driver: Loaded
Stack plugin "o2cb": Loaded
Driver for "ocfs2_dlmfs": Loaded
Filesystem "ocfs2_dlmfs": Mounted
Checking O2CB cluster "SharedFS": Online
  Network idle timeout: 30000
  Network keepalive delay: 2000
  Network reconnect delay: 2000
  Heartbeat mode: Local
Checking O2CB heartbeat: Not active
Debug file system at /sys/kernel/debug: mounted

Note that the hostnames in the configuration file (here Pinky and Perky) must match the actual hostnames of the machines.

Now you can format the filesystem on one of the nodes, and DRBD will synchronise this to the other node:

mkfs.ocfs2 -N 2 -L Shared /dev/drbd0

While this is doing its job, you can see the other node replicating the data:

cat /proc/drbd

version: 8.4.10 (api:1/proto:86-101)
srcversion: D7CA86B1B124FE79CE17C50 
 0: cs:Connected ro:Primary/Primary ds:UpToDate/UpToDate C r-----
    ns:0 nr:294324 dw:294324 dr:0 al:0 bm:0 lo:1 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

Note that it will take considerably longer to format the partition that it would do on a standalone machine, depending on the speed of the network connection between the two nodes.

Be careful if you plan to use OCFS2 for a partition mounted on a virtual machine.

You can't do the OCFS2 part on the virtualisation host, and then mount the resulting file system inside the guest. You need to have OCFS2 installed and configured on the guest itself.

At first I tried to do:

• DRBD on the virtualisation host
• OCFS2 on the virtualisation host
• mount the resulting FS on the virtualised guest

This does not work (the FS simply refuses to mount, with an error about there being no heartbeat).

Therefore I see two possible alternatives:

1:

• DRBD on the virtualisation host
• OCFS2 on the virtualised guest
• mount the resulting FS on the virtualised guest

2:

• DRBD on the virtualised guest
• OCFS2 on the virtualised guest
• mount the resulting FS on the virtualised guest

Of the two, I prefer the first, because it seems neater to me to have DRBD running once on the host for as many shared devices as are needed for all the VMs, instead of installing and configuring DRBD on every VM.

You can't avoid having to have OCFS2 on each VM, so that to me feels like a good compromise.

Go up
Return to main index.