complex lvm on an alternative install of ubuntu (debian-installer)

I’ve been meaning to post my notes on this for a while but I wanted to post some code which needs to be stripped of the proprietary work. At my job, we have a need to do multiple installs a day of ubuntu via the network and require a fairly complex lvm configuration. When done on feisty, first you need to deal with the three minutes it takes to create the lvm devices. This is a bug, check out this post about it.

Partman-auto and partman-lvm-auto configuration recipes are pretty complex to begin with. If you’re not simply formatting the whole disk, it can get really confusing, let alone figuring out the difference between partman and partman-lvm. There’s some sample code out there, but it’s hard to tell when era it’s from. Upon learning that partman-auto wasn’t going to support multiple disks, I started looking for alternatives. Thanks to cjwatson and fjp for pointing me in the right directions at times.

Finding the right hooks was the hard part. debian-installer (or d-i) is the alternative and network (pxe based) installer for ubuntu. It’s very modular and uses anna (the lightweight version of apt) specific udeb (anna’s version of debs) configurations to let d-i know in what order a specific module should be run. Basically what I did was force partman not to run, and have a shell script run instead.

Assuming that you’ve already got a pxe install going with a preseed file, use the following:


d-i preseed/early_command string wget http://server.example.com/ubuntu/config/lvm.sh -P /tmp ; chmod 755 /tmp/lvm.sh ; echo /tmp/lvm.sh installer >> /var/lib/dpkg/info/download-installer.postinst

This will download your script and set it to be executed after the base installer is downloaded. This is important because some of the install system is in the initrd, but a lot of it, partman included, is installed from a udeb after the initrd is loaded. So you can’t just hack partman from inside the initrd.

lvm.sh is a two phase script. the first is run when the base installer is setup, by the earlier call in download-installer.postinst. This phase then sets itself to be run instead of partman, allowing you to create and mount your partitions at the correct time.

Note that you should have your disk mounted as /target by the end of phase 2 as well as have your fstab configured. I’d also recommend running swapon against your swap, but keep in mind that the mkswap currently shipping will only prepare 2GB of swapspace, see bug 119900.

Theres some extra stuff in this script you won’t need. It’s also untested with my companies stuff removed. If having a bunch of lvms named ‘stuffN’ seems stupid, it’s because that’s my way of obfuscating the code so I won’t have someone from work complain. As always, YMMV.


#!/bin/sh
# Manually create LVM configuration
# Partman does not currently support multi-disk lvm
# Designed to be run after download-installer but before partman-base
# This allows us to modify partman-base.postinst after it's been dropped in by anna
# Partman appears to be entirely an external program, removing the call to partman from partman-base.postinst prevents it from running.

case "$1" in
installer)
# we should have d-i downloaded by now.
# partman comes in a udeb from the network so we have to hook here
# and replace the partman-base.postinst file
sed -i 's/partman/\/tmp\/lvm.sh partman/' /var/lib/dpkg/info/partman-base.postinst
logger lvm.sh modified partman-base.postinst
;;
partman)
# do filesystem stuff: detect our config, fdisk, lvms, mount /target
logger lvm.sh partition configuration starting
modprobe dm_mod

# FIXME: This is going to be really dirty to handle our configurations. More work will need to be done later.
# case1: sda: 1171842048 hda: 125056
# case2: sda: 976519168 hda: 58605120
# case3: sda: 732389376 hda:

SIZE_SDA=`sed -n 's/.* \([0-9]*\) sda$/\1/p' < /proc/partitions`
SIZE_HDA=`sed -n 's/.* \([0-9]*\) hda$/\1/p' < /proc/partitions`

echo sda: $SIZE_SDA hda: $SIZE_HDA

# pvcreate filters (ignored by filtering) if the there's a partition table
dd if=/dev/zero of=/dev/sda bs=512 count=1

# check for separate physical boot drive
if [ $SIZE_HDA ] ; then
# we have the boot disk create a primary partition
echo ",,83" | sfdisk /dev/hda


pvcreate -ff -y /dev/sda
BOOT=/dev/hda1
LVM=/dev/sda
else
# no separate boot drive
echo -e ",256,83\n,,8e" | sfdisk -uM /dev/sda

pvcreate -ff -y /dev/sda2
BOOT=/dev/sda1
LVM=/dev/sda2
fi

mke2fs -q $BOOT
vgcreate -s 256M system $LVM

if [ $SIZE_SDA -gt 700000000 ] ; then
COMPLEXFS=1
lvcreate -L 20G -n stuff1 system
lvcreate -L 20G -n stuff2 system
lvcreate -L 8G -n swap system
lvcreate -L 20G -n stuff3 system
lvcreate -L 200G -n stuff4 system
lvcreate -L 200G -n stuff5 system
lvcreate -L 200G -n stuff6 system

for fs in stuff1 stuff2 stuff3 stuff4 stuff5 stuff6 ; do mkfs.reiserfs -q /dev/system/$fs 1>/dev/null; done

else
# FIXME: swap too big for vmware
lvcreate -L 8G -n swap system
lvcreate -l `pvdisplay | sed -n 's/Free PE \([0-9]*\)/\1/p'` -n config1 system

mkfs.reiserfs -q /dev/system/stuff1 1>/dev/null
fi

# setup common swap
mkswap /dev/system/swap
swapon /dev/system/swap

# Create directory structure
mkdir /target
mount /dev/system/stuff11 /target -treiserfs
mkdir /target/boot
mount $BOOT /target/boot -text2
if [ $COMPLEXFS ] ; then
mkdir -p /target/stuff2
mkdir -p /target/stuff3
mkdir -p /target/stuff4
mount /dev/system/stuff2 /target/stuff2
fi

# Create fstab
mkdir /target/etc
echo \# /etc/fstab: static file system information. > /target/etc/fstab
echo \# >> /target/etc/fstab
echo "# " >> /target/etc/fstab
echo $BOOT /boot ext2 defaults 1 2 >> /target/etc/fstab
echo /dev/system/stuff1 / reiserfs acl,user_xattr 1 1 >> /target/etc/fstab
if [ $COMPLEXFS ] ; then
echo /dev/system/stuff2 /stuff2 reiserfs acl,user_xattr 1 2 >> /target/etc/fstab
echo /dev/system/stuff3 /stuff3 reiserfs acl,user_xattr 1 2 >> /target/etc/fstab
echo /dev/system/stuff4 /stuff4 reiserfs acl,user_xattr 1 2 >> /target/etc/fstab
fi
echo /dev/system/swap none swap sw 0 0 >> /target/etc/fstab
echo proc /proc proc defaults 0 0 >> /target/etc/fstab

# Secret udev rules hack for network cards
mkdir -p /target/etc/udev/rules.d
echo \# on board e100 > /target/etc/udev/rules.d/50-network.rules
echo KERNELS==\"0000:00:06.0\", NAME=\"eth2\" >> /target/etc/udev/rules.d/50-network.rules
echo \# on board tg3 \(2x1000\) >> /target/etc/udev/rules.d/50-network.rules
echo KERNELS==\"0000:02:09.0\", NAME=\"eth0\" >> /target/etc/udev/rules.d/50-network.rules
echo KERNELS==\"0000:02:09.1\", NAME=\"eth1\" >> /target/etc/udev/rules.d/50-network.rules

;;
*)
echo $0: This script is destructive and should only be run as part of the debian-installer process
;;
esac

5 thoughts on “complex lvm on an alternative install of ubuntu (debian-installer)

  1. phillip

    Nice, I’ve been looking for some way to override the brain-dead partman. Trying to use it a for a cd-install system though, adding raid under the lvms. no luck so far

  2. btm

    I’ve got more code up now under this post now. I realize looking that there are some bits that overlap. Anyways, feel free to post comment questions or email me.

    I’d imagine something similar could be done with a cd install. Point at a preseed file from the syslinux bootloader on the cd should work, although i’ve never tried it. Keeping in mind that a few things run in different orders between a cd and net install. I don’t remember anything that should bother you.

    Standard shell script debugging can help. If you modify the preseed file so it’s not an automatic headless install and modify the shell script so it gets put in place, you can pause the install and then run the script yourself to see how it all works out rather than spending the time building debugging in the d-i way.

  3. Glenn Aaldering

    Epic. With this its possible to work around the braindead partman and use your own custom partitioning scheme.

    This will also work if you dont use LVM and just want to partition multiple disks like /dev/vda and /dev/vdb. But more important: with this you can use the whole raw block disk without partitions/partitioning. I finally came up with an install on /dev/vda instead of /dev/vda1

    Thanks a lot!

  4. fogcat5

    just happened to find this article when looking for how to find a list of attached drives on debian so I can partition,format,and mount them.

    this will be really handy building the servers with ubuntu, thanks for the good writeup!

  5. David

    Your insight proved very handy in helping me supplant partman’s retardation with my own for a number of baremetal nodes that must be Ubuntu with several VGs/LVs across several logical disks. Thank you.

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload the CAPTCHA.