ZFS on read-only devices

On my last post I described my experiments to get along the best way to archive ROM sets with the biggest space saving. In a nutshell, ZFS got a best-for-money status, except for the RAM usage. My idea after the tests was to check if it was possible to mount a ZFS pool recorded in a read-only device, like a CD/DVD/BD.

Why here?

They’re cheap, small, and usually, more durable (quite depends on disc quality, but most of the failures are bitwise not full disc, and this can be solved with error recovery algorithms). But ZFS was not designed with these discs in mind, so some magic was expected. All of this is done on Linux. Should be doable on BSD, Solaris and Mac OS X with not many differences. First of all, I did not test ZFS on packet writing, if it works at all optical drives seeking time and speed will be a hell. So, first of all, I created a dummy file to act as the writable zpool before recording it to disc, with the following command:

dd if=/dev/zero of=zpool.bin bs=2048 conv=sparse count=<sectors>

<sectors> being size as following:

• 333,000 for 74-min CD-Rs and CD-RWs
• 360,000 for 80-min CD-Rs and CD-RWs
• 2,298,496 for single-layer DVD-Rs
• 4,171,712 for double-layer DVD-Rs
• 2,295,104 for single-layer DVD+Rs
• 4,173,824 for double-layer DVD+Rs
• 12,219,392 for single-layer Blu-rays
• 24,438,784 for double-layer Blu-rays
• 48,878,592 for triple-layer Blu-rays
• 62,500,864 for quad-layer Blu-rays

Once the dummy file is created you need to create the zpool. While you could create a RAIDZ pool for error recovery, it would require you to have 3 optical drives with 3 discs inserted, at the same time, at least, so I suggest for another solution (I’m still searching one, you can comment your ideas). For the zpool creation:

zpool create -o comment="Put some disc identification here" -o ashift=11 \
-o failmode=continue -O mountpoint=/mnt/myzfsdvd -O checksum=sha256 \
-O compression=gzip-9 -O dedup=on -O atime=off -O devices=off -O exec=off \
-O setuid=off myzfsdvd zpool.bin

Explaining each options:

• comment=“Put some disc identification here”, as it’s for a removable disc you need some description of what you’re going to store on it. You can set it up later
• ashift=11, this means that the underlying block device (that is, the optical disc) uses 2048 bytes/sector
• failmode=continue, not a good idea for the system to panic or stop when something happens to the optical disc!
• mountpoint=/mnt/myzfsdvd, good to tell a mountpoint. You can change this on mount, but setting a default prevents security risks
• checksum=sha256, this is just personal taste
• compression=gzip-9, optical discs are only written once, so let that writing time be used the best
• dedup=on, same as above, deduplicate blocks
• atime=off, no sense to store access time on read-only discs
• devices=off, to disable support of device nodes
• exec=off, to prevent executing code from the disc, you may want to do, but I didn’t
• setuid=off, to prevent setuid

Once this is done, ZFS will automatically mount the pool on the specified mountpoint, so you can start copying things.

You may be tempted to change recordsize (block size) to equal hardware sector for optical discs (2048 bytes), DO NOT DO IT, it will make ZFS be marginally slower, and compression and deduplication work worse.

And for RAM usage on deduplication, don’t worry, with the disc out RAM will get freed, and even using quad-layer Blu-rays, the deduplication table will never be bigger than 1Gb.

Time to unmount the ZFS file so we can record it to disc

zpool export myzfsdvd

Now recording it. If you’re going to record on CD (really? sure? ok, as you wish) be sure that it is recorded single-session, mode 1, data, finished. Following line works for CD, DVD and BD:

cdrecord -v driveropts=burnfree -dao -data zpool.bin

Once it is recorded, you’ll be tempted to mount it, and if you do, you’ll see the failure:

It took me a couple of minutes to find what the problem of that unspecific error was, as it clearly was trying to mount it. It is trying to write!

I used /dev because sometimes it decided to not mount it from the DVD-ROM drive (/dev/sr0).

Now the pool works perfectly, except for some secondary effects:

• zpool scrub denies to work, because pool is read-only, instead of checking without correcting.
• zdb is unable to find the pool at all.