HPE ProLiant DL380 Gen11 + Connecting PCIe Axelera Accelerator

Hi there ​@rozero !

Hmm, I’ve not personally seen anyone working with a DL380, in all honesty, but I’ve asked around internally at Axelera to see if anyone’s aware of it 👍I’m always a bit nervous when it comes to power adapers, as they’ll sometimes fit but have an unexpected pin out! 😅

Hey @rozero, no DL380 here either (my Metis is the M.2, running in a desktop), so take this as generic server tinkering rather than hands-on with that exact box.

On the pinout worry @Spanner raised: the trap with 8-pin aux cables is that PCIe and EPS12V use the same Molex Mini-Fit Jr. housing and will physically mate even when wired mirrored. HPE's GPU cable kit should be proper PCIe pinout by design, but since the riser side is proprietary you can't really tell by eye. When the cable arrives, spend five minutes with a multimeter on the card-side connector: a PCIe 8-pin should show the three +12V pins on one row and grounds (plus sense) on the other. If it reads like EPS, stop there.

Also worth checking before you order: on recent ProLiants the GPU aux output on the riser is not always live by default, it can depend on riser population and setup options. Might be worth a quick question to HPE whether that output is actually enabled in your config, otherwise you risk debugging "card not detected" when the real problem is no 12V on the aux.

One thing for later: if the card powers up fine but you get DMA timeouts or flaky PCIe link errors at boot, try disabling ASPM on that root port. Same family of problem as the S3/resume DMA timeout thread here, that one bit me for weeks on AM5:

https://community.axelera.ai/axelera-embedded-3/s3-suspend-resume-permanent-dma-timeout-on-metis-m-2-amd64-desktop-driver-has-no-resume-path-please-fix-or-document-1388

Good luck with the cluster, 4 AIPUs on one node sounds like a fun rig.