0. Preface

This document is an attempt at step-by-step instructions for installing Debian Squeeze on a Dell PowerEdge 1950 using IPMI's Serial Over LAN console abilities. It also assumes you're going to install Debian via a PXE netboot. These instructions are valid for Debian's other boot methods, but since you're cool enough to use IPMI, you should be using netboot too.

1. Get the BMC on the network

To configure the Baseboard Management Controller (BMC), you're probably going to need a monitor and keyboard. This is because the BMC is disabled from the factory, and I say probably because (if you're really motivated) you could create a USB drive that boots Linux and enables it from within the OS. But we're going to assume you're not going to do that.

The 1950's net0 is a shared BMC/host port.

2. Initial BMC Configuration

1. Boot up and enter BIOS setup and the Remote Access Configuration Utility:

When the hardware boots, you will be presented with the opportunity to enter BIOS Setup:

_                                                                     F2 = Setup
Phoenix ROM BIOS PLUS Version 1.10 2.2.6                      F10 = Utility Mode
Copyright 1985-1988 Phoenix Technologies Ltd.                    F11 = Boot Menu
Copyright 1990-2008 Dell Inc.                                     F12 = PXE Boot
All Rights Reserved

Dell System PowerEdge 1950
www.dell.com
Testing memory.  Please wait.

You'll want to strike <F2> with hasty abandon until F2 = Setup changes to Entering Setup.

As the hardware continues to boot, you will eventually be presented with the option to set up Remote Access:

Press <Ctrl-E> for Remote Access Setup within 5 sec.....

Striking <Ctrl-E> as instructed will bring up the RAC Utility:

┌──────────────────── Remote Access Configuration Utility ─────────────────────┐
│              Copyright 2006 Dell Inc. All Rights Reserved 1.25               │
│                                                                              │
└──────────────────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────────────┐
│ Baseboard Management Controller Revision                       1.77          │
│ Primary Backplane Firmware Revision                            1.05          │
│ ──────────────────────────────────────────────────────────────────────────   │
│                                                                              │
│ IPMI Over LAN ................................................ Off           │
│ NIC Selection ................................................ Shared        │
│ LAN Parameters ............................................... <ENTER>       │
│ LAN User Configuration ....................................... <ENTER>       │
│ Reset To Default ............................................. <ENTER>       │
│ System Event Log Menu ........................................ <ENTER>       │
│                                                                              │
│                                                                              │
│                                                                              │
│                                                                              │
│                                                                              │
│                                                                              │
  Up,Down Arrow to select  │  SPACE,+,- to change  │  ESC to exit  │  F1=Help

2. Enable IPMI Over LAN:

IPMI Over LAN ................................................ On
NIC Selection ................................................ Shared

Failover NIC mode supposedly uses 802.3ad (LACP) to pair both interfaces, which I haven't tested, but it's a neat feature.

3. Configure the network:

Examples, of course, your configuration will be different.

LAN Parameters ............................................... <ENTER>

│ IPMI Ove┌────────────────────────────────────────────────────────┐           │
│ NIC Sele│ ────────────────────────────────────────────────────   │red        │
│ LAN Para│ IP Address Source ............... Static               │TER>       │
│ LAN User│ DHCP Host Name .................. <ENTER>              │TER>       │
│ Reset To│ Ethernet IP Address .............  10. 1 . 1 . 28      │TER>       │
│ System E│ MAC Address ..................... 08:00:20:C0:FF:EE    │TER>       │
│         │ Subnet Mask ..................... 255.255.255. 0       │           │
│         │ Default Gateway .................  10. 1 . 1 . 1     ▒ │           │
│         │ ──────────────────────────────────────────────────── ▒ │           │
│         │ VLAN Enable ..................... On                 ▒ │           │
│         │ VLAN ID ......................... 0001               ▒ │           │
│         │ VLAN ............................ Priority 0           │           │

A nice feature of the PE 1950 (and other newer Dell BMCs) is that they support 802.1q VLAN Tagging. Thus, since the port is shared, you can put the BMC on a management network while the host OS lives on a public network, all with a single network cable.

4. Create an IPMI user:

LAN User Configuration ....................................... <ENTER>

│ LAN Parameters .┌─────────────────────────────────────────────┐<ENTER>       │
│ LAN User Configu│ Account Access ....... Enabled              │<ENTER>       │
│ Reset To Default│ Account Privilege .... Admin                │<ENTER>       │
│ System Event Log│ ─────────────────────────────────────────   │<ENTER>       │
│                 │ Account User Name .... [root            ]   │              │
│                 │ Enter Password ....... [                ]   │              │
│                 │ Confirm Password ..... [                ]   │              │
│                 └─────────────────────────────────────────────┘              │

5. Save and exit:

Strike <ESC> to pop up the Exit menu.

│ NIC Selection ..........┌──────────────────────────┐.......... Shared        │
│ LAN Parameters .........│ Save Changes and Exit    │.......... <ENTER>       │
│ LAN User Configuration .│ Discard Changes and Exit │.......... <ENTER>       │
│ Reset To Default .......│ Return to Setup          │.......... <ENTER>       │
│ System Event Log Menu ..└──────────────────────────┘.......... <ENTER>       │

Choose Save Changes and Exit.

3. BIOS Configuration

1. Access the BIOS:

Assuming you followed step 1 in Initial BMC Configuration, upon exiting the BMC configuration, you'll be presented with the BIOS setup. If not, reboot and strike <F2> vigorously.

The BIOS configuration:

_                  Dell Inc. (www.dell.com) - PowerEdge 1950
                               BIOS Version 2.2.6
┌──────────────────────────────────────┬───────────────────────────────────────┐
│ Service Tag: HAHANO                  │ Asset Tag:                            │
└──────────────────────────────────────┴───────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────────────┐
│ System Time .......................................... 00:00:00              │
│ System Date .......................................... Thu Jan 01, 1970      │
│                                                                              │
│ Memory Information ................................... <ENTER>               │
│ CPU Information ...................................... <ENTER>               │
│                                                                              │
│ SATA Port A .......................................... Off                   │
│                                                                              │
│ Boot Sequence ........................................ <ENTER>               │
│ Boot Sequence Retry .................................. Disabled              │
│                                                                              │
│ Integrated Devices ................................... <ENTER>               │
│ PCI IRQ Assignment ................................... <ENTER>               │
│                                                                            ▒ │
│ Serial Communication ................................. <ENTER>             ▒ │
  Up,Down Arrow to select  │  SPACE,+,- to change  │  ESC to exit  │  F1=Help

2. Configure Console Redirection:

You can play with the Failsafe Baud Rate or Remote Terminal Type if you want, these worked for me:

Serial Communication ................................. <ENTER>

│ Sy┌─────────────────────────────────────────────────────────────────────┐    │
│   │ Serial Communication ....... On with Console Redirection via COM2   │    │
│ Me│ External Serial Connector .. COM1                                   │    │
│ CP│ Failsafe Baud Rate ......... 57600                                  │    │
│   │ Remote Terminal Type ....... VT100/VT220                            │    │
│ SA│ Redirection After Boot ..... Enabled                                │    │
│   └─────────────────────────────────────────────────────────────────────┘    │

3. While you're in there, set the front panel:

Embedded Server Management ........................... <ENTER>

│ CPU In┌───────────────────────────────────────────────────────────────┐      │
│       │ Front-Panel LCD Options ............... User-Defined String   │      │
│ SATA P│ User-Defined LCD String ............... <ENTER>               │      │
│       └───────────────────────────────────────────────────────────────┘      │

User-Defined LCD String ............... <ENTER>

│      ┌─────────────────────────────────────────────────────────────────┐     │
│ SATA │   DER BLINKENLICHTEN                                            │     │
│      └─────────────────────────────────────────────────────────────────┘     │

4. Assuming you're going to PXE boot, enable PXE on NIC1:

Integrated Devices ................................... <ENTER>

│ System┌───────────────────────────────────────────────────────────────┐      │
│       │ Integrated RAID Controller ........ Enabled                   │      │
│ Memory│ Embedded SATA ..................... Off                       │      │
│ CPU In│ IDE CD-ROM Controller ............. Auto                      │      │
│       │ User Accessible USB Ports ......... All Ports On              │      │
│ SATA P│ Embedded Gb NIC1 .................. Enabled with PXE          │      │
│       │   MAC Address ..................... 080020C0FFEE              │      │
│ Boot S│   Capability Detected ............. TOE                       │      │
│ Boot S│ Embedded Gb NIC2 .................. Enabled                 ▒ │      │
│       │   MAC Address ..................... 00E8DEADBEEF            ▒ │      │
│ Integr│   Capability Detected ............. TOE                       │      │
│ PCI IR└───────────────────────────────────────────────────────────────┘      │

And copy down NIC1's MAC Address for entry into your DHCP config later.

5. Save and exit:

Strike <ESC> to pop up the Exit menu.

│                         ┌──────────────────────────┐                         │
│ SATA Port A ............│ Save Changes and Exit    │.. Off                   │
│                         │ Discard Changes and Exit │                         │
│ Boot Sequence ..........│ Return to Setup          │.. <ENTER>               │
│ Boot Sequence Retry ....└──────────────────────────┘.. Disabled              │

Choose Save Changes and Exit. The host reboots.

4. Set up DHCP and netboot

netboot is the image used to PXE boot and install Debian. Note that net'inst' is not net'boot'!

1. Set up a TFTP server somewhere.
2. Set up a DHCP server somewhere. Doesn't have to be the same physical server as TFTP.
3. Download netboot.tar.gz from the testing Debian Installer page and unpack it in (probably a subdirectory of) the TFTP server's directory (often /tftpboot or on Debian, /var/lib/tftpboot).
4. Let's assume I stuck netboot in /tftpboot/debian/squeeze/amd64. Modify the pxelinux config in pxelinux.cfg/default, I generally just wipe out the contents and put what I'm actually going to install in there:

default menu.c32
prompt 0

menu title PXE Boot Menu

label squeeze-eth0-ttys1
    menu label ^Squeeze via eth0/ttyS1
    kernel debian/squeeze/amd64/linux
    append locale=en_US.UTF-8 interface=eth0 console-tools/archs=skip-config console-keymaps-at/keymap=us hostname=x domain=x preseed/url=http://foo.bar.baz/preseed-squeeze.cfg console=ttyS1,57600n8 DEBIAN_FRONTEND=text initrd=debian/squeeze/amd64/initrd.gz -- quiet

DEBIAN_FRONTEND is optional. You may have success with the default curses interface, but if you have all kinds of screen draw problems, switch to text.

5. Add bnx2 firmware to the initrd image as explained here. Otherwise, your installer won't be able to use the network devices. And that's kinda useless.
6. Configure your DHCP server to respond to the host. Using ISC DHCP, the following two options are all that's necessary in addition to the usual host stuff:

next-server 1.2.3.4;
filename "/debian/squeeze/amd64/pxelinux.0";

Where "next-server" is the address of your TFTP server and "filename" is the path to netboot's pxelinux.0 from the root of the tftpboot directory.

5. Boot to the installer

Your BMC should now be responding on the network. Use IPMItool to manipulate it.

You can use the -P <password> argument to ipmitool to avoid typing your password so much, but this will expose it to anyone who views the arguments (like via ps(1)) while the command is executing.

1. Instruct the host to try to boot via PXE:

% ipmitool -I lanplus -H bmc.example.org -U root -P hax chassis bootdev pxe
Set Boot Device to pxe

2. Reboot:

% ipmitool -I lanplus -H bmc.example.org -U root -P hax chassis power reset
Chassis Power Control: Reset

3. Connect to the system's console:

% ipmitool -I lanplus -H bmc.example.org -U root sol activate
Password:
[SOL Session operational.  Use ~? for help]

Your console should begin to show the hardware POST.

4. If all is well you should see the boot process, the DHCP server will respond to the PXE request, and the host will load the SYSLINUX loader. Hit Return to select the default installer you defined in pxelinux.cfg/default.

If the install gets messed up and you need to restart, but you've already written some stuff to the disk, the BIOS will no longer attempt to PXE boot by default. Repeat step 2 if you need to restart the installation.

6. Configure Debian

1. Once booted into the new system, set GRUB_CMDLINE_LINUX and fix the port speed in GRUB_SERIAL_COMMAND in /etc/default/grub to get full console port redirection to the serial port. Don't forget to run update-grub:

# grep GRUB_CMDLINE_LINUX= /etc/default/grub
GRUB_CMDLINE_LINUX="console=ttyS1,57600n8"
# grep GRUB_SERIAL_COMMAND= /etc/default/grub
GRUB_SERIAL_COMMAND="serial --unit=1 --speed=57600 --word=8 --parity=no --stop=1"
# update-grub
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.30-2-amd64
Found initrd image: /boot/initrd.img-2.6.30-2-amd64
done
#

2. Install firmware-bnx2 from non-free so you have drivers in the installed OS. It's a good idea to preseed it, if you're preseeding.

# apt-get install firmware-bnx2
# reboot

2. ???
3. PROFIT!!!