So I got open-iscsi working on debian, in so much that I had four disks, two to the ‘preferred controller’ were good but the two to the second controller weren’t. Switching the preferred controller

After installing multipath-tools I started looking at dmsetup but the target types listed in the man page: linear, striped and error, didn’t make sense. When I read the INTRO file included in the debian package I saw there were additional types snapshot and mirror. This thread clued me in to there being a multipath type.

Running ‘multipath -v 3 -ll’ provided some more information that made things click in my head. Running ‘blkid’ produced:

/dev/mapper/36001c23000d59fc600000284478bcdca1: UUID=”8d070de0-403c-4669-9db0-5b17e3aeebc5″ SEC_TYPE=”ext2″ TYPE=”ext3″
/dev/sda1: UUID=”742239f4-b6fe-4422-b1a2-5639e5ab4675″ SEC_TYPE=”ext2″ TYPE=”ext3″
/dev/sda5: TYPE=”swap”

The mapper device was created by running multipath and seemed to figure bits out on it’s own such that running just ‘multipath -ll’ would show the paths that were and were not working (thats a different problem).

sdb: checker msg is “readsector0 checker reports path is down”
sdc: checker msg is “readsector0 checker reports path is down”
36001c23000d59fc600000284478bcdcadm-0 DELL,MD3000i
[size=558G][features=0][hwhandler=0]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:0:0 sdb 8:16 [active][faulty]
\_ round-robin 0 [prio=0][enabled]
\_ 6:0:0:0 sdc 8:32 [active][faulty]
\_ round-robin 0 [prio=1][active]
\_ 4:0:0:0 sdd 8:48 [active][ready]
\_ round-robin 0 [prio=1][enabled]
\_ 5:0:0:0 sde 8:64 [active][ready]

For a while I was getting ‘mount: no such partition found’ when trying to mount by the UUID shown by ‘blkid’. It just stopped while I was researching the problem. The man page for mount indicates it needs access to /proc/partitions but I saw nothing related to UUID’s in there or elsewhere poking around /proc. I noticed there was a correct symlink in /dev/disk/by-uuid, so I rebooted the machine and checked again and it was gone. ‘iscsiadm -m session’ confirmed no sessions but ‘iscsiadm -m node’ had the nodes cached so I ran ‘iscsiadm -m node -L all’ to login again and verified the sessions again. I looked in /dev/disk/by-uuid and the uuid had shown up again. multipathd was running at startup so I figure it got things going again.

Interestingly, ‘multipath -ll’ only showed

sdb: checker msg is “readsector0 checker reports path is down”
sdc: checker msg is “readsector0 checker reports path is down”
36001c23000d59fc600000284478bcdcadm-0 DELL,MD3000i
[size=558G][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
\_ 2:0:0:0 sdd 8:48 [active][ready]
\_ round-robin 0 [prio=1][enabled]
\_ 3:0:0:0 sde 8:64 [active][ready]

Changing the preferred path made the disks go down and the dm faulty:

root@file01:/mnt# multipath -ll
sdd: checker msg is “readsector0 checker reports path is down”
sde: checker msg is “readsector0 checker reports path is down”
36001c23000d59fc600000284478bcdcadm-0 DELL,MD3000i
[size=558G][features=0][hwhandler=0]
\_ round-robin 0 [prio=0][enabled]
\_ 2:0:0:0 sdd 8:48  [failed][faulty]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:0:0 sde 8:64  [failed][faulty]

Running ‘multipath’ added the other two block devices again and I remounted ok.  This time the filesystem stayed happy when I changed the preferred path. I’m willing to suspect that you can only access a virtual disk via one controller at a time, although from either interface on that controller. That is, you can only access it on the second controller when the first one fails or you manually change the preferred path. The work is just getting everything set up so that it works on startup. What’s missing appears to be getting the iscsi login and then the multipath to include all disks, then your normal automount in fstab.

‘iscsiadm -m node -o show’ reports ‘node.startup = manual’ which is also set in /etc/iscsid.conf and /etc/iscsi/iscsid.conf. I ran ‘iscsiadm -m node -o update -n node.startup -v automatic’. Rebooting saw the login automatically firing.

Putting the UUID or /dev/dm-1 in the fstab wasn’t working. Watching the console it was obvious it was trying to mount the partition before the multipath stuff ran. Per ‘/usr/share/doc/multipath-tools-initramfs/README.Debian’ in the ‘multipath-tools-initramfs’ package I ran ‘update-initramfs -t -c -v -k `uname -r`’.

On reboot I saw “FATAL: Module dm_multipath not found.” While multipath may have been part of the problem, it seems like even with _netdev as a mount option the device is trying to be mounted before the open-iscsi daemon runs. I’ll leave that problem and post for another day; tomorrow if I’m lucky and nothing breaks.