How to fix a failed apt-clone dist-upgrade on Nexenta unstable

The current Nexenta NCP 2.0 hardy-unstable repository has some broken bits that prevent a clean apt-clone dist-upgrade from working. A common question on the IRC channel is how to work around this, and I’ve answered it enough times that I thought it best to write up some basic instructions.

Here’s an example of what happens:


# apt-clone -v dist-upgrade
This operation will upgrade your system using ZFS capabilities. Proceed ? (y/n) y

Updating APT sources …
apt-get -V update
Hit http://apt.nexenta.org hardy-unstable Release.gpg
Hit http://apt.nexenta.org hardy-unstable Release
Hit http://apt.nexenta.org hardy-unstable/main Packages
Hit http://apt.nexenta.org hardy-unstable/contrib Packages
Hit http://apt.nexenta.org hardy-unstable/non-free Packages
Hit http://apt.nexenta.org hardy-unstable/main Sources
Hit http://apt.nexenta.org hardy-unstable/contrib Sources
Hit http://apt.nexenta.org hardy-unstable/non-free Sources
Reading package lists… Done
Initiating Nexenta upgrade procedure. Please wait…
Success. This upgrade will download approximately 38.90MB
This upgrade will require REBOOT. Proceed? (y/n) y

Upgrade is in progress. Please DO NOT interrupt…
Creating Upgrade Checkpoint…
syspool/rootfs-nmu-002: entry not found in menu.lst

Upgrade Checkpoint has been created: rootfs-nmu-002

Use ‘zfs list -r syspool’ command to list all available
upgrade/rollback checkpoints

apt-get -V -y install dpkg apt
Reading package lists… Done
Building dependency tree
Reading state information… Done
dpkg is already the newest version.
apt is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 16 not upgraded.
rmdir /tmp/upgrade-attempt.21883 2>/dev/null
mkdir /tmp/upgrade-attempt.21883
mount -F zfs syspool/rootfs-nmu-002 /tmp/upgrade-attempt.21883 2>/dev/null
chroot /tmp/upgrade-attempt.21883 mount /proc 2>/dev/null
mount -F lofs -O /etc/mnttab /tmp/upgrade-attempt.21883/etc/mnttab 2>/dev/null
mount -F lofs -O /devices /tmp/upgrade-attempt.21883/devices 2>/dev/null
APT_CLONE_ENV=1 chroot /tmp/upgrade-attempt.21883 apt-get -V -o Dpkg::Options::=”–force-confdef” -o Dpkg::Options::=”–force-confold” dist-upgrade
Reading package lists… Done
Building dependency tree
Reading state information… Done
Calculating upgrade… Done
The following packages will be upgraded:
man-db (2.5.2-2nexenta1 => 2.5.2-2nexenta2)
sunwcakr (5.11.104-2 => 5.11.104-3)
sunwckr (5.11.104-4 => 5.11.104-5)
sunwcsl (5.11.104-5 => 5.11.104-6)
sunwemlxs (5.11.104-1 => 5.11.104-2)
sunwiscsidmr (5.11.104-1 => 5.11.104-2)
sunwiscsidmu (5.11.104-1 => 5.11.104-2)
sunwiscsir (5.11.104-5 => 5.11.104-6)
sunwiscsitr (5.11.104-1 => 5.11.104-2)
sunwiscsitu (5.11.104-1 => 5.11.104-2)
sunwkrbu (5.11.104-1 => 5.11.104-2)
sunwstmf (5.11.104-1 => 5.11.104-2)
sunwstmfu (5.11.104-1 => 5.11.104-2)
sunwzfskr (5.11.104-5 => 5.11.104-7)
sunwzfsr (5.11.104-4 => 5.11.104-6)
sunwzfsu (5.11.104-4 => 5.11.104-6)
16 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 20.8MB/38.9MB of archives.
After this operation, 3048kB disk space will be freed.
Do you want to continue [Y/n]? y
Get:1 http://apt.nexenta.org hardy-unstable/main sunwcakr 5.11.104-3 [2462kB]
Get:2 http://apt.nexenta.org hardy-unstable/main sunwckr 5.11.104-5 [14.5MB]
Get:3 http://apt.nexenta.org hardy-unstable/main sunwzfsr 5.11.104-6 [453kB]
Get:4 http://apt.nexenta.org hardy-unstable/main man-db 2.5.2-2nexenta2 [1050kB]
Get:5 http://apt.nexenta.org hardy-unstable/main sunwiscsir 5.11.104-6 [236kB]
Get:6 http://apt.nexenta.org hardy-unstable/main sunwzfskr 5.11.104-7 [851kB]
Get:7 http://apt.nexenta.org hardy-unstable/main sunwzfsu 5.11.104-6 [1253kB]
Fetched 20.8MB in 49s (420kB/s)
nl.1: sh: fatal: relocation error: file /lib/libncurses.so.5: symbol main: referenced symbol not found
E: Sub-process /usr/bin/apt-clone -c exited unexpectedly
E: Failure running script /usr/bin/apt-clone -c
chroot /tmp/upgrade-attempt.21883 umount /proc 2>/dev/null
umount /tmp/upgrade-attempt.21883/etc/mnttab 2>/dev/null
umount /tmp/upgrade-attempt.21883/devices 2>/dev/null
rm -rf /tmp/nl*
umount /tmp/upgrade-attempt.21883 2>/dev/null
rmdir /tmp/upgrade-attempt.21883
Upgrade failed. Would you like to rollback changes now? (y/n)

In order to fix this, you would answer no to the rollback and then execute something like this:


# mount -F zfs syspool/rootfs-nmu-002 /mnt
# chroot /mnt mount /proc
# mount -F lofs -O /etc/mnttab /mnt/etc/mnttab
# mount -F lofs -O /devices /mnt/devices
# chroot /mnt


Now from within the chroot, execute:


# rm -r /tmp/nl*
# APT_CLONE_ENV=1 apt-get -V -o Dpkg::Options::=”–force-confdef” -o Dpkg::Options::=”–force-confold” dist-upgrade
# bootadm update-archive
# exit


This should get all the packages within the clone upgraded properly, update the boot archive, and back you out of the clone. Now you need to install a new grub, unmount the clone, and reboot:


# installgrub /mnt/boot/grub/stage1 /mnt/boot/grub/stage2 /dev/rdsk/
# installgrub /mnt/boot/grub/stage1 /mnt/boot/grub/stage2 /dev/rdsk/ (if you have one)
# chroot /mnt umount /proc
# umount /mnt/devices
# umount /mnt/etc/mnttab
# umount /mnt


Obviously you’ll need to replace with the correct device(s), such as c0d0s0 and c1d0s0. If you don’t know which devices to use, run zpool status syspool and it should tell you. Note above that we did not activate the new clone as the default bootfs. It is best now to reboot, select the new entry from the grub menu, and verify that the clone is working properly. Once you’re booted into the clone, you can run apt-clone -a rootfs-nmu-002 to make it the default.

Hopefully this helps some of you out there having problems. Feel free to drop by ##nexenta on irc.freenode.net if you require more assistance.