SunFire X2200 - Debian Squeeze Installation via eLOM

How to make it happen

Author: Nate Coraor <nate@psu.???>
Version: 0.3
Copyright: Creative Commons Attribution Non-Commercial Share Alike

0. Preface

This document is an attempt at step-by-step instructions for installing Debian Squeeze on a Sun SunFire X2200 using the eLOM management console. It also assumes you're going to install Debian via a PXE netboot. These instructions are valid for Debian's other boot methods, but since you're cool enough to use IPMI, you should be using netboot too.

1. Get it on the network

The X2200's net1 is a shared eLOM/host port. The eLOM will DHCP for an address, but only for 5 seconds, once. If you miss this window, reset the power to initiate a new DHCPDISCOVER.

Once it grabs an address, ssh as root. Password is 'changeme'.

2. Initial eLOM Configuration

Why does eLOM suck so bad compared to iLOM?

  1. Set a password:
/SP -> set User/root Password=foo
Set 'Password' to 'foo'
/SP ->

A '*' in the password is okay for SSH but seems to be invalid for the web interface. Speaking of which, the web interface cannot be disabled =(.

Also, unlike some other BMCs (such as recent ones from Dell and HP), the X2200's eLOM does not have support for 802.1q VLAN Tagging. In the OS itself (Solaris, Linux, or whatever) you can set a VLAN, but the eLOM must use the default (untagged) VLAN.

  1. Up the annoyingly low timeout value:
/SP -> set AgentInfo ConsoleTimeOut=1800
Set 'ConsoleTimeOut' to '1800'
/SP ->

3. BIOS Configuration

  1. Access the BIOS:
/SP -> set SystemInfo/CtrlInfo BootCtrl=BIOSSetup
The System will be Powered On.
Do you wish to Continue..?(y/n)y
Power reset and enter BIOS setup has succeeded
Set 'BootCtrl' to 'BIOSSetup'
/SP ->
  1. Connect to the console:
/SP -> start AgentInfo/console
console activate successful
press ESC ( to terminate session...
  1. Configure IOMMU:
Advanced -> IOMMU Option Menu
IOMMU Mode: 64MB
  1. Configure the serial port (the default speed of 9600 bps is painfully slow):
Advanced -> Remote Access Configuration
Serial Port Mode: 115200 8,n,1
  1. Save:
Esc+0

or

Exit -> Save Changes and Exit
  1. Exit the console:
Esc+(

4. eLOM Serial Port Configuration

  1. Set the port speed
/SP -> set AgentInfo/console speed=115200
Set 'speed' to '115200'
/SP ->

5. Set up DHCP and netboot

netboot is the image used to PXE boot and install Debian. Note that net'inst' is not net'boot'!

  1. Set up a TFTP server somewhere.
  2. Set up a DHCP server somewhere. Doesn't have to be the same physical server as TFTP.
  3. Download netboot.tar.gz from the testing Debian Installer page and unpack it in (probably a subdirectory of) the TFTP server's directory (often /tftpboot or on Debian, /var/lib/tftpboot).
  4. Let's assume I stuck netboot in /tftpboot/debian/squeeze/amd64. Modify the pxelinux config in pxelinux.cfg/default, I generally just wipe out the contents and put what I'm actually going to install in there:
default menu.c32
prompt 0

menu title PXE Boot Menu

label squeeze-eth1-ttys1
    menu label ^Squeeze via eth1/ttyS1
    kernel debian/squeeze/amd64/linux
    append locale=en_US.UTF-8 interface=eth0 console-tools/archs=skip-config console-keymaps-at/keymap=us hostname=x domain=x preseed/url=http://foo.bar.baz/preseed-squeeze.cfg console=ttyS1,115200n8 initrd=debian/squeeze/amd64/initrd.gz -- quiet

Under Lenny, net1 used to correspond to eth3, but this appears to have been fixed under Squeeze.

  1. Configure your DHCP server to respond to the host. I'll only be using one port and net1 at that (so I have less cabling and network hardware). If you want to put the eLOM and host on separate VLANs you'll have to do so after installation as the Debian Installer does not support VLANs at this time. Using ISC DHCP, the following two options are all that's necessary in addition to the usual host stuff:
next-server 1.2.3.4;
filename "/debian/squeeze/amd64/pxelinux.0";

Where "next-server" is the address of your TFTP server and "filename" is the path to netboot's pxelinux.0 from the root of the tftpboot directory. If you need net1's ethernet address, it can be obtained with:

/SP -> show SystemInfo/NIC/NIC0 MACAddress2

  /SP/SystemInfo/NIC/NIC0
    Targets:

    Properties:
        MACAddress2 = 08:00:20:C0:FF:EE

    Target Commands:
        show

/SP ->

6. Boot to the installer

At this point, it'd be great if you could tell the host to use PXE on the next boot, directly from the service processor. Apparently other eLOM implementations support the following:

/SP -> set SystemInfo/CtrlInfo BootCtrl=PXE
Set 'BootCtrl' to 'PXE'
/SP ->

I tried it and the service processor will allow you to set this value. It even causes it to PXE boot, However, shortly after loading the PXELINUX loader, the whole system, including the service processor stopped responding and had to be power cycled by pulling the cord. So unfortunately, you'll have to pay attention to the boot sequence.

  1. Reset the host:
/SP -> set SystemInfo/CtrlInfo PowerCtrl=reset
The System will be Reset.
Do you wish to Continue..?(y/n)y
Set 'PowerCtrl' to 'reset'
/SP ->
  1. Connect to the console:
/SP -> start AgentInfo/console
console activate successful
press ESC ( to terminate session...

If all you see is a bunch of unprintable characters, set the console speed back to 9600 in the eLOM and then double check that it's set properly in the BIOS.

  1. If all is well and there's nothing preinstalled on the hard drive, you should see the boot process, the DHCP server will respond to the PXE request, and the host will load the SYSLINUX loader. Hit Return to select the default installer you defined in pxelinux.cfg/default.

If the install gets messed up and you need to restart, but you've already written some stuff to the disk, the BIOS will no longer default to PXE boot. The easiest way around this is to simply mash <ESC><Shift><2> (<F12>) frantically at boot for the microsecond you're given to choose network boot before the Broadcom firmware loads.

7. Configure Debian

  1. Once booted into the new system, set GRUB_CMDLINE_LINUX and fix the port speed in GRUB_SERIAL_COMMAND in /etc/default/grub to get full console port redirection to the serial port. Don't forget to run update-grub:
# grep GRUB_CMDLINE_LINUX= /etc/default/grub
GRUB_CMDLINE_LINUX="console=ttyS1,115200n8"
# grep GRUB_SERIAL_COMMAND= /etc/default/grub
GRUB_SERIAL_COMMAND="serial --unit=1 --speed=115200 --word=8 --parity=no --stop=1"
# update-grub
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.30-2-amd64
Found initrd image: /boot/initrd.img-2.6.30-2-amd64
done
#
  1. ???
  2. PROFIT!!!