There’s good info out there here and here on how to do this, I’m not going to repeat them.

A few notes though:

1) You do not need password syncronization installed when using Identity management for Unix (what Services for Unix is called in R2. Note that R2 x64 supports this stuff while SFU won’t install on on 2003 SP2 x64). You just need the NIS Server for Unix module and the admin tools, which is what you get if you just click Identity Management for Unix in add/remove programs under something like Active Directory something.

Interestingly I’ve been crashing another DC trying to get that working and while I’ve done this before it’d been a while and I forgot this key point: pam_ldap does NOT look for the users password in LDAP. It looks for the user in LDAP using the binddn to search, then BINDS as the user using the provided password to see if it succeeds or fails.

2) If when you log in as user ‘getent passwd’ stops providing useful information, and w, ls -l, etc can’t resolve the uid to a name, or “I have no name!” is part of your shell prompt, it’s because the pam stuff isn’t running as root and the user doesn’t have access to the right ldap bind credentials. I had rootbinddn set and passwords in /etc/*.secret which were 600. But AD doesn’t allow anonymous binds, so I had to move all this to the conf files and 644 them, setting binddn and bindpw instead of rootbinddn.

Story is that you could run nscd (name service caching daemon) as root and then hide this stuff from the user, I don’t have the time do deal with this at the moment though.

3) Changing passwords. I was getting this error at first:

Enter login(LDAP) password:
New password:
passwd: Authentication information cannot be recovered
passwd: password unchanged

Putting my “password sufficient pam_ldap.so” line before the pam_unix.so line in /etc/pam.d/common-password fixed that. Then:

LDAP password information update failed: Unknown error
00002098: SecErr: DSID-03150A45, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0

Watching the LDAP transaction I saw that we were binding as the user (not the binddn) and sending a crypt password. I read in the man page about pam_password ad which I set and then I started getting this instead:

LDAP password information update failed: Unknown error
00002077: SvcErr: DSID-03190DC9, problem 5003 (WILL_NOT_PERFORM), data 0

Because I hadn’t set up SSL yet. turning on ‘ssl start_tls’ didn’t work but setting “ssl on” and “tls_checkpeer no” which makes it not verify the certs worked alright.

I put the certs in and tried to turn on tls_checkpeer but had failures and saw a lot of traffic that looked like “SASL GSS-API Integrity: searchResDone(15886) insufficientAccessRights (00002105: LdapErr: DSID-0C0907C9, comment: Error processing control, data 0, vece) [0 results]”. At first I assumed this was because I wasn’t running krb5. Everyone seems to set up winbind and I’m trying not to. Anyways, password changes still fail with:

LDAP password information update failed: Unknown error
0000052D: AtrErr: DSID-03190F00, #1:
0: 0000052D: DSID-03190F00, problem 1005 (CONSTRAINT_ATT_TYPE), data 0, Att 9005a (unicodePwd)

I assumed at first this might be the “you’re not using a hard enough password” error, but it wasn’t the case.

I spent hours flipping different options trying to get tls to work without setting “tls_checkpeer no”. using tls (ldaps:// for the uri) or starttls with the “sasl_secprops maxssf=0” AD hack didn’t work. Eventually I stumbled over this and changed my ldaps:// URI from the IP address of the DC to the FQDN of the DC and now ssl works without tls_checkpeer no. !!!! Just put the hostname and IP in /etc/hosts if you care.

Unfortunately I can’t find a newer version of this kb article that describes how unicodepwd access control works in Windows 2000.

I grabbed the source with apt-get and it looks sane:

case PASSWORD_AD:
/*
* Patch from Norbert Klasen :
*
* To be able to change a password in AD via LDAP, an SSL connection
* with a cipher strength of at least 128 bit must be established.
* http://support.microsoft.com/support/kb/articles/q264/4/80.ASP
* http://support.microsoft.com/support/kb/articles/Q247/0/78.ASP
*
* The password attribute used by AD is unicodePwd. Its syntax is octect
* string. The actual value is the password surrounded by quotes in
* Unicode (LSBFirst).
*
* NT passwords can have max. 14 characters.
*
* FIXME:
* The conversion to Unicode only works if the locale is
* ISO-8859-1 (aka Latin-1) [of which ASCII is a subset].
*/

snprintf (new_password_with_quotes, sizeof (new_password_with_quotes),
“\”%s\””, new_password);
memset (new_unicode_password, 0, sizeof (new_unicode_password));
for (i = 0; i < bv_val =" new_unicode_password;" bv_len =" strlen" modv_bvals =" bvalsnew;" mod_type =" (char">conf->rootbinddn || getuid () != 0)
{
/* user must supply old password */
snprintf (old_password_with_quotes,
sizeof (old_password_with_quotes), “\”%s\””,
old_password);
memset (old_unicode_password, 0, sizeof (old_unicode_password));
for (i = 0; i < bv_val =" old_unicode_password;" bv_len =" strlen" modv_bvals =" bvalsold;" mod_type =" (char" mod_op =" LDAP_MOD_DELETE" mod_op =" LDAP_MOD_ADD" mod_op =" LDAP_MOD_REPLACE">

Interestingly, I went and added rootbinddn and the /etc/pam_ldap.secret file and when changing the password for the ldap user as root it works. Oddly it seems to keep allowing all the past passwords for a while as well. I’m going to leave it as this, as all users will be root users for now and can sudo passwd username.

4) (update) I had problems using openssl 0.9.8c-4etch1 and libnss-ldap 251-7.5etch1 on debian etch where when I turned on ssl ‘getent passwd’ wouldn’t fully return. turning ssl off (and changing the uri back to ldap:// from ldaps://) would make things fine again. Wireshark looked like the stream was getting mucked up somewhere, as the windows server was sending back duplicate acks. A consultant said this is a libnss-ldap bug, or at least that padl’s libnss-ldap stuff is pretty buggy expecially when you get beyond a basic ldap configuration. it looks like we’re headed for a full blown winbind configuration instead of libnss.

I talked about this a bit in an earlier post but wanted to talk about it a bit more as it comes to multiple architectures. We run vmware vms on debian hosts with gobs of memory so generally they’re amd64/x64 but often the guests are i386/32bit debian installs to not have to deal with the ickyness of amd64 and compatibility like libraries.

apt-mirror supports multiple architectures but normally searching for information about it just pulls up tons of pages advertising as much. I finally found an example here off which I based my configs.

After installing apt-mirror (‘apt-get install apt-mirror’ on etch), a config file (/etc/apt/mirror.list) like:

## The default configuration options (uncomment and change to override)
##
#
# set base_path /var/spool/apt-mirror
# set mirror_path $base_path/mirror
# set skel_path $base_path/skel
# set var_path $base_path/var
#
# set defaultarch
# set nthreads 20
#
set _tilde 0

deb http://ftp.us.debian.org/debian/ etch main main/debian-installer
deb-amd64 http://ftp.us.debian.org/debian etch main main/debian-installer
deb-src http://ftp.us.debian.org/debian/ etch main

deb http://security.us.debian.org/ etch/updates main contrib
deb-amd64 http://security.us.debian.org/ etch/updates main contrib
deb-src http://security.us.debian.org/ etch/updates main contrib

The above was about 29GB before the deb-amd64 lines were added and apt-mirror is throwing in another 6.5GB right now for the deb-amd64 lines.

This howto says “In my tests I mirrored the main, contrib, and non-free sections of Debian Sarge and the main, restricted, and universe sections of Ubuntu Edgy Eft which took about 25GB of hard disk space and about 6 hours of download time on a 16MBit DSL line.” Note that does not included deb-src: “Also, I don’t want to mirror the source packages because in 95% of all installations you don’t need source packages, and they need a lot of space on the hard disk.”

Also they say “I will not mirror security updates in this tutorial – I think it’s a good idea to always download security updates directly from the internet. That way you can be sure you get the latest ones.” however I’ve seen when mirroring gutsy that important and large packages sometimes end up in the security updates. It’s really time consuming and annoying when a base package gets a security update and suddenly all your network installs slow down because you have to wait for it.

Running ‘su – apt-mirror -c apt-mirror’ will get you a tree in /var/spool/apt-mirror that looks like (ignoring the pool tree, where the debs are actually kept:

# tree -d -L 6 . -I pool
.
|– ftp.us.debian.org
| `– debian
| `– dists
| `– etch
| `– main
| |– binary-amd64
| |– binary-i386
| |– debian-installer
| `– source
`– security.us.debian.org
`– dists
`– etch
`– updates
|– contrib
| |– binary-amd64
| |– binary-i386
| `– source
`– main
|– binary-amd64
|– binary-i386
`– source

You’ll need to install apache and configure a tree to match a repository. Mines a little hacked with symlinks because I needed a link from stable to etch for the network install. In /var/www:

.
|– debian
| |– dists
| | |– etch -> /var/spool/apt-mirror/mirror/ftp.us.debian.org/debian/dists/etch/
| | `– stable -> etch
| `– pool -> /var/spool/apt-mirror/mirror/ftp.us.debian.org/debian/pool/
|– preseed
`– security -> /var/spool/apt-mirror/mirror/security.us.debian.org

Pay particular note to the line wrapping depending on your screen width. For instance /var/www/debian/dists/etch is actually a symlink to /var/spool/apt-mirror/mirror/ftp.us.debian.org/debian/dists/etch while /var/www/debian/dists/stable is a symlink to /var/www/debian/dists/etch. Preseed is just a folder I keep my preseed config files in.

The lines in your /etc/apt/sources.list should look something like:

deb http://yourhost.local/debian/ etch main
deb-src deb http://yourhost.local/debian/ etch main
deb http://yourhost.local/security etch/updates main

it’s somewhat intelligent about knowing what architecture it is. it’ll need more tuning that that, you’ll find edge cases like security update source packages that you oughtta be able to fix pretty easy. I haven’t tuned it myself yet because it’s not a priority at this point.

Using debian instead of ubuntu at the new job and needed to rebuild my old network installs. Etch takes things a little differently than gutsy used to.

pxelinux.cfg/default:

LABEL etch_i386_install_auto
kernel debian/etch/i386/linux
append vga=normal initrd=debian/etch/i386/initrd.gz preseed/url=http://debian.yourdomain.local/preseed/standard.cfg debian-installer/locale=en_US console-keymaps-at/keymap=us
netcfg/get_hostname=unassigned netcfg/get_domain=yourdomain.local —

Otherwise I’m just pulling bits from my old configs. If you haven’t set this all up before, this is a good howto on pxe booting server installs using syslinux/pxelinux (my dhcp server is Windows, just remember to set DHCP options 66 to the pxe server IP and ‘pxelinux.0’ respectively). Note that I couldn’t get the netcfg lines to work in the preseed file, I assume because it’s technically coming before it grabs the preseed file.

Note that I’ve also used apt-mirror to create a local repository. Don’t forget you’ll need d-i (debian-installer) so you’ll need something like this for a mirror.list:

deb http://ftp.us.debian.org/debian/ etch main main/debian-installer
deb-src http://ftp.us.debian.org/debian/ etch main

deb http://security.us.debian.org/ etch/updates main contrib
deb-src http://security.us.debian.org/ etch/updates main contrib

You could probably do without the deb-src. I wasn’t really thinking at the time and included those lines. My last job all over our boxes had like 2.5TB of space on them so I stopped worrying about such things but at my new job all our storage is in SANs and I haven’t convinced anyone to let me abuse the non-crappy ones yet. Anyways, the above mirror came out to about 29GB.

You’ll need to set up a series of symlinks in apache to get it to serve this all up the way a real mirror would. I wish apt-mirror was smarter about this, especially at my last job where I had lots of local repositories and nests of links to get it all working and right now the SAN where this repo is stored doesn’t support symlinks so I’m still left with a nasty mess of them to get everything where I want it to be. I haven’t gotten around to getting the security mirror working but I’ll warn you that I think it needs to be a separate virtualhost under apache because the “d-i apt-setup/security_host string” preseed command takes a hostname, not a structure like a repository string does (hostname folder release). You can disable security updates on install and just use puppet, cfengine or whatever to take care of this later if you prefer anyways. Or do it like it’s done here, this works for me:

d-i apt-setup/security_host string
d-i apt-setup/local0/repository string http://yourhostserver/debian-security etch/updates main

When configuring passwords with something like:

d-i passwd/root-password-crypted password [md5’d password goes here]
d-i passwd/make-user boolean false

Note that you can create the md5 string with ‘openssl passwd -1’ or ‘grub-md5-crypt’ depending on whats on your box already, and yes keep the word ‘password’ after ‘root-password-crypted’

For the record my preseed ended up looking like this:

# based off of:
# http://loftninjas.org/blog/2007/10/ubuntu-lvm-network-install-part-2.html
# see also: http://www.debian.org/releases/etch/example-preseed.txt

# keymap/language/local/network done on kernel bootline since it’s before we fetch this preseed file

# prevents silly questions like asking about popularity-contest
d-i debconf/priority string critical

# set local tz, set hardware clock to utc
d-i time/zone string US/Pacific
d-i clock-setup/utc boolean true

# Install source
d-i mirror/country string enter information manually
d-i mirror/http/hostname string FIXME
d-i mirror/http/directory string /debian/
d-i mirror/codename string etch
d-i mirror/http/proxy string

d-i apt-setup/local0/repository string http://FIXME debian etch
d-i apt-setup/local0/comment string local debian etch repository
d-i apt-setup/local1/repository string http://FIXME security etch/updates main
d-i apt-setup/local1/comment string local debian etch security repository
d-i apt-setup/security_host string

# this should choose the most minimal install known so far
tasksel tasksel/first multiselect standard

# install additional apps
d-i pkgsel/include string tcpdump vim

d-i grub-installer/only_debian boolean true

# specifc to etch, not the same with sarge
# install to the first scsi disk, automatically, single partition, with swap, no lvm
d-i partman-auto/init_automatically_partition select Guided – use entire disk
d-i partman-auto/disk string /dev/sda
d-i partman-auto/method string regular
d-i partman-auto/choose_recipe select All files in one partition (recommended for new users)
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition select Finish partitioning and write changes to disk
d-i partman/confirm boolean true

# root password in md5, don’t ask to create a normal user
d-i passwd/root-password-crypted password FIXME
d-i passwd/make-user boolean false

# Avoid that last message about the install being complete.
d-i finish-install/reboot_in_progress note

Now just gotta figure out why Windows Server 2003 R2 SP2 crashes because of lsass.exe whenever the new SFU/Services For Unix identity crap updates a changed password.

Many banks/sites are now asking security questions if you log in and don’t have a cookie from a previous login. Worst in my life are:

Key Bank. Requires you enter your credit card number and pin code to verify. How stupid it is to have people entering their pin codes anywhere?

Sallie Mae. Requires you complete five security questions and randomly asks you three. You can only choose from a preselected list. I considered setting all of my answers to ‘1’ or something, because I’ve had to call up and reset my account because I couldn’t remember if the answer to “what street do you live on” was:

26th
26th Ave
26th Ave E

Or what, if any punctuation was used.

This is almost enough bullshit to make me STOP using online banking and go back to the old way of doing things as this is getting to be LESS convenient.

Article in the news today about some Nick Carr dude and his new books prediction that IT will go away.

If Carr’s predictions come true, strong technical skills will still be valued by service providers.

It’s nice that was the last sentence in the article as that was what I was thinking the whole time. I believe there’s an argument hidden in there that service providers will require many less skilled folks that corporations. Software as a Service, heh, what’s the world coming to?

We’ll see I guess, but I have to wonder how much of the style of outsourcing back-lash we’ll see with this. I think outsourcing is just starting to settle and find it’s niches.

A user connecting from Vista 64 with the Cisco AnyConnect client was getting a “The VPN client was unable to successfully verify the IP forwarding table modifications. A VPN connection will not be established.” error when trying to connect. No changes had been made to the concentrator configuration which is an asa5520 running 8.0(3).

Via ASDM, there was a syslog notification of “SVC Message: 17/ERROR: Unable to successfully verify all routing table modifications are correct.”

Also annoyingly, the license only supports 2 clientless ssl vpn connections and the ssl vpn client appears to use a clientless connection initally which fails to shut down then the ssl client fails to connect, which prevents future logins with not error on the client side due to the licensing.

I found this article which linked the proble to Adobe Photoshop. The user had installed the photoshop trial recently and when he disabled bonjour for windows, which was installed by photoshop, the VPN worked fine.

I installed Bonjour on XP 32bit and could not reproduce the problem. Perhaps it’s a Vista 64 issue. It’s a small enough of an edge case that I don’t think I’ll try to reproduce.

User says: “it had a really odd name #1_Service_name###. it was added when I installed Adobe”