So, as of yesterday, I started running a pretty-close-to-upstream op-build host firmware stack on my Blackbird. Notable yak-shaving has included:
- Update README.md for Fedora 31 build-deps because READMEs are important
- Support Python3 as default /usr/bin/python (so I could compile it at all on Fedora 31)
- RaptorCS Blackbird support the actual minimal firmware port
- Linux v5.4.3-openpower1 because 5.3.7 in the current op-build repo has an NVME driver bug that means it didn’t recognize my Intel NVME drive, and thus prevented me from actually booting an OS.
Apart from that, I was all happy as Larry. Except then I went into the room with the Blackbird in it an went “huh, that’s loud”, and since it was bedtime, I decided it could all wait until the morning.
It is now the morning. Checking fan speeds over IPMI, one fan stood out (fan2, sitting at 4300RPM). This was a bit of a surprise as what’s silkscreened on the board is that the rear case fan is hooked up to ‘fan2″, and if we had a “start from 0/1” mix up, it’d be the front case fan. I had just assumed it’d be maybe OCC firmware dying or something, but this wasn’t the case (I checked – thanks occtoolp9!)
After a bit of digging around, I worked out this mapping:
| IPMI fan0 | Rear Case Fan | Motherboard Fan 2 |
| IPMI fan1 | Front Case Fan | Motherboard Fan 3 |
| IPMI fan2 | CPU Fan | Motherboard Fan 1 |
Which is about as surprising and confusing as you’d think.
After a bunch of digging around the Raptor ports of OpenBMC and Hostboot, it seems that the IPL Observer which is custom to Raptor controls if the BMC decides to do fan control or not.
You can get its view of the world from the BMC via the (incredibly user friendly) poking at DBus:
busctl get-property org.openbmc.status.IPL /org/openbmc/status/IPL org.openbmc.status.IPL current_status; busctl get-property org.openbmc.status.IPL /org/openbmc/status/IPL org.openbmc.status.IPL current_istep
Which if you just have the Hostboot patch in (like I first did) you end up with:
s "IPL_RUNNING"
s "21,3"Which is where Hostboot exits the IPL process (as you see on the screen) and hands over to skiboot. But if you start digging through their op-build tree, you find that there’s a signal_linux_start_complete script which calls pnv-lpc to write two values to LPC ports 0x81 and 0x82. The pnv-lpc utility is the external/lpc/ binary from skiboot, and these two ports are the “extended lpc port 80h” state.
So, to get back fan control? First, build the lpc utility:
git clone git@github.com:open-power/skiboot.git
cd skiboot/external/lpc
makeand then poke the magic values of “IPL complete and linux running”:
$ sudo ./lpc io 0x81.b=254
[io] W 0x00000081.b=0xfe
$ sudo ./lpc io 0x82.b=254
[io] W 0x00000082.b=0xfe
You get a friendly beep, and then your fans return to sanity.
Of course, for that to work you need to have debugfs mounted, as this pokes OPAL debugfs to do direct LPC operations.
Next up: think of a smarter way to trigger that than “stewart runs it on the command line”. Also next up: work out the better way to determine that fan control should be on and patch the BMC.