Tag: ZFS on Linux

Posted on

zpool: Symbol `spa_feature_table’ has different size in shared object, consider re-linking

If you see the following error message when executing ‘zpool status’ after “upgrading” Proxmox to the (currently) latest version with ZoL 0.6.4 instead of 0.6.3:

zpool: Symbol `spa_feature_table' has different size in shared object, consider re-linking

Explanation:

“Re-linking” doesn’t have a special meaning in ZFS/ZoL terminology, it rather just describes the usual linking step after compiling some source code.

Solution:

The most likely cause is that some new, updated and required packages have not been installed (e.g. with ‘uname -a’, you’ll see that the old kernel is still running, even after rebooting). Usually, this happens if one executes “apt-get update && apt-get upgrade” (as usual) rather than “apt-get update && apt-get dist-upgrade“. So, in order to solve this problem, do:

# apt-get update && apt-get dist-upgrade
# reboot

In this case, ZoL 0.6.4 also includes a couple of new features which can be enabled for all local ZFS pools by executing:

# zpool upgrade -a
This system supports ZFS pool feature flags.

Enabled the following features on 'rpool':
 spacemap_histogram
 enabled_txg
 hole_birth
 extensible_dataset
 embedded_data
 bookmarks
Posted on

Proxmox VE 3.4 and ZFS: How to create an ISO to use lz4 compression by default

EDIT 20150607: Meanwhile, this fix has been included by Proxmox – the current official Proxmox VE 3.4 Installer ISO finally enables lz4 compression by default at installation time. I thus recommend using an official Proxmox VE ISO image instead of creating an ISO image yourself, although creating your own ISO might still be interesting from an educational point of view or to include fixes that haven’t made it into the official Proxmox ISO yet.

Proxmox VE is a truly great, Debian based Linux distribution to host both KVM virtual machines and OpenVZ containers, using a RedHat based kernel (numlock.ch runs as a KVM guest on Proxmox VE, BTW ;).

With the recent release of Proxmox VE 3.4, the whole package got even better, now also allowing to install Proxmox on top of ZFS (which is the best enterprise file system available to date), specifically its native port for Linux, ZFS on Linux (ZoL).

Unfortunately, Proxmox VE 3.4 doesn’t allow to use lz4 as a compression algorithm at installation time yet: Using lz4 would be strongly recommended however instead of using ZoL’s default lzjb (or much worse: gzip). There are some known workarounds for this, but those are rather tedious, either involving extensive file copying or a manual setup of the ZFS pool and file systems.

The most elegant solution at the moment is creating a bootable ISO image (based on the original bootable Proxmox VE 3.4 ISO image) with a patched /usr/bin/proxinstall script that uses “compression=lz4” instead of “compression=on” by default.

And here’s how to create the patched ISO:

1. Loop-mount the original ISO (can only be mounted read-only):

# mount -o loop /path/to/proxmox-ve_3.4-3f2d890e-1.iso /mnt/cdrom

2. Copy /usr/bin/proxinstall from the mounted ISO image to a read-writeable directory (create directories as necessary):

# cp -a /mnt/cdrom/usr/bin/proxinstall /mnt/cdrom_patched/usr/bin/proxinstall

3. Edit the /mnt/cdrom_patched/usr/bin/proxinstall script as follows:

# diff -u ../cdrom/usr/bin/proxinstall usr/bin/proxinstall 
--- ../cdrom/usr/bin/proxinstall	2015-02-12 17:52:50.000000000 +0100
+++ usr/bin/proxinstall	2015-03-17 21:50:07.662031284 +0100
@@ -592,7 +592,7 @@
     # disable atime during insatll
     syscmd ("zfs set atime=off $zfspoolname") == 0 ||
 	die "unable to set zfs properties\n";
-    syscmd ("zfs set compression=on $zfspoolname") == 0 ||
+    syscmd ("zfs set compression=lz4 $zfspoolname") == 0 ||
 	die "unable to set zfs properties\n";
 }

4. Use the complicated but very powerful tool xorriso (install it, if necessary) to create a bootable ISO image based on the original ISO, but with /usr/bin/proxinstall “overwritten” with the edited proxinstall script:

# xorriso -boot_image grub patch -indev proxmox-ve_3.4-3f2d890e-1.iso -overwrite on -outdev proxmox-ve_3.4-3f2d890e-1-with-lz4-patch.iso -blank as_needed -pathspecs on -add /usr/bin/proxinstall=/mnt/cdrom_patched/usr/bin/proxinstall -- -commit

5. Use this patched ISO to install Proxmox VE 3.4 as usual

After rebooting the installed Proxmox VE 3.4 host/server, use the following command to verify that lz4 has been used by default:

# zfs get compression
NAME              PROPERTY     VALUE     SOURCE
rpool             compression  lz4       local
rpool/ROOT        compression  lz4       inherited from rpool
rpool/ROOT/pve-1  compression  lz4       inherited from rpool
rpool/swap        compression  lz4       inherited from rpool


For your convenience, here’s a patched ISO of Proxmox VE 3.4 for download (Note: Use it at your own risk!)

proxmox-ve_3.4-3f2d890e-1-with-lz4-patch.iso_.gz (571 MB, md5sum: 2abba5445133c011aadb1808237202b0)

Download it, gunzip it and rename it to proxmox-ve_3.4-3f2d890e-1-with-lz4-patch.iso to get rid of the silly underscore (“_”) WordPress appended when I uploaded the file. The resulting ISO image will be 725 MB, md5sum: e09e5d250d16fa182129c72be88a5aa2.

EDIT 20150607: Meanwhile, this fix has been included by Proxmox – the current official Proxmox VE 3.4 Installer ISO finally enables lz4 compression by default at installation time. I thus recommend using an official Proxmox VE ISO image instead of creating an ISO yourself.

Have fun!