The trend with init replacements

When you write a number of service resource providers for a configuration management system, you get some intimate experience with the quirks of init systems. A slew of new ones are working their way into stable releases lately which seem primarily motivated by decreasing system startup time by allowing services to be started in parallel. For instance, Ubuntu has been moving to upstart, the latest release of Debian uses insserv, and OS X uses launchd. There is overlap in design, and certainly parallel service execution isn’t the only significant improvement. Since init is a basic building block of our systems, small changes can cause large ripples. In the end we will have some great new functionality, but we’re in a rough patch of transition right now and need to ensure the functionality we rely upon doesn’t get passed over.

Disabling services with Upstart

If you want a service to not start on system startup, but still want to be able to start it, you have to comment out a line in the configuration file. Programmatically editing configuration files, from a script or a configuration management system is difficult to do cleanly. In general you want to avoid minor changes to configuration files because then you have to reconcile the differences when you upgrade the package. There are plans to add support for an override file wherein you can specify that the service is manual, but clearly Ubuntu server users are taking a backseat to desktop users inside Canonical where Upstart is developed.

Restarting services with Upstart

Which is interesting, as Ubuntu server related packages are being migrated to use Upstart. We start to run into additional quirks, such as when you restart a service that isn’t running, Upstart does not start it. We plan to work around this behavior in Chef but others have clearly taken notice.

$ status mysql
mysql start/running, process 548
$ sudo restart mysql
mysql start/running, process 649
$ sudo stop mysql
mysql stop/waiting
$ sudo restart mysql
restart: Unknown instance: 

Insserv changes how you specify runlevels

On Debian lenny you could specify service runlevels and priorities as such:

$ sudo update-rc.d apache2 start 20 3 4 5 . stop 80 0 1 .
 Adding system startup for /etc/init.d/apache2 ...
   /etc/rc0.d/K80apache2 -> ../init.d/apache2
   /etc/rc1.d/K80apache2 -> ../init.d/apache2
   /etc/rc3.d/S20apache2 -> ../init.d/apache2
   /etc/rc4.d/S20apache2 -> ../init.d/apache2
   /etc/rc5.d/S20apache2 -> ../init.d/apache2

However on squeeze, update-rc.d is wrapped by insserv, which ignores your request and acts on the LSB headers.

$ sudo update-rc.d apache2 start 20 3 4 5 . stop 80 0 1 2 6 .
update-rc.d: using dependency based boot sequencing
update-rc.d: warning: apache2 start runlevel arguments (3 4 5) do not match LSB Default-Start values (2 3 4 5)
update-rc.d: warning: apache2 stop runlevel arguments (0 1 2 6) do not match LSB Default-Stop values (0 1 6)
$ find /etc/rc* -name '*apache*'
/etc/rc0.d/K01apache2
/etc/rc1.d/K01apache2
/etc/rc2.d/S18apache2
/etc/rc3.d/S18apache2
/etc/rc4.d/S18apache2
/etc/rc5.d/S18apache2
/etc/rc6.d/K01apache2

Insserv does have an option to override the LSB headers, but the update-rc.d wrapper doesn’t use it and you have to be very careful as it fails silently if you use it wrong.

$ sudo insserv -r apache2
$ sudo insserv apache2,start=3,4,5,stop=0,1,2,6
$ find /etc/rc* -name '*apache*'
/etc/rc0.d/K01apache2
/etc/rc1.d/K01apache2
/etc/rc2.d/K01apache2
/etc/rc2.d/S18apache2
/etc/rc3.d/S18apache2
/etc/rc4.d/S18apache2
/etc/rc5.d/S18apache2
/etc/rc6.d/K01apache2

Additional behavior to work around in Chef.

Moving forward

Distributions continue to change the way we interact with init with every release. This is clearly a reasons to use a configuration management tool. You know that you want mysql to never start automatically because your cluster resource manager controls it, but how you achieve that has been changing lately with regularity. You can let your configuration management tool abstract that from you. Still, we need to stay involved in the discussions in the open source communities whose software we use and be proactive citizens.