Welcome to the community ​@alsutton! Great to have you here.

Just to tick off a couple of obvious thought before digging deeper, but has Above 4G Decoding been enabled in the BIOS?

Any improvement after running axdevice --refresh in the SDK environment?

Thanks for the greeting .

Above 4G decoding has been enabled. The other potentially relevant settings (all under “Advanced” in the BIOS) are;

CPU Configuration > X2APIC = Enabled
CPU Configuration > Intel Virtualisation Technology = Disabled
Chipset > Northbridge > IIO > IOU2 = 4x4x4x4
Chipset > Northbridge > IIO > IIO2 > IIO Port Link Speed = Gen 3 (8 GT/s)
Chipset > Northbridge > IIO > IIO2 > IIO Port Non-Posted Prefetch = Disabled
Chipset > Northbridge > IIO > Intel VT for Directed I/O (VT-d] = Disabled
Chipset > PCIe/PCI/PnP Configuration > PCI PERR/SERR Support = Disabled
Chipset > PCIe/PCI/PnP Configuration > SR-IOV Support = Disabled
Chipset > PCIe/PCI/PnP Configuration > ASPM Support = Disabled

Hello!
I have tried both Intel and AMD environments, and I think the following conditions are necessary for recognition by lspci.

1. OS installation in UEFI mode (it could not be recognized under the conditions below with CSM Enable or Legacy　Boot).

2. Above 4G settings (must be enabled in the BIOS or UEFI).

3. Resizable BAR might also be a possibility. If this setting cannot be configured in UEFI, it was not recognized by lspci.

Please check if it can be set in UEFI. You might also want to consider changing the motherboard.

axdevice -refresh output;

(venv) alsutton@svr220:~/Utils/voyager-sdk$ axdevice --refresh
ERROR: No target device found in lspci output
ERROR: AXR_ERROR_CONNECTION_ERROR: No target device found in lspci output
(venv) alsutton@svr220:~/Utils/voyager-sdk$ axdevice --refresh -v
INFO: Removing 0000:00:01.0
INFO: PCIE rescan
ERROR: No target device found in lspci output
Traceback (most recent call last):
  File "/home/alsutton/.cache/axelera/venvs/83d579fa/bin/axdevice", line 8, in <module>
    sys.exit(entrypoint_main())
  File "/home/alsutton/.cache/axelera/venvs/83d579fa/lib/python3.10/site-packages/axelera/runtime/axdevice.py", line 739, in entrypoint_main
    main(args)
  File "/home/alsutton/.cache/axelera/venvs/83d579fa/lib/python3.10/site-packages/axelera/runtime/axdevice.py", line 698, in main
    found_devices = c.list_devices()
  File "/home/alsutton/.cache/axelera/venvs/83d579fa/lib/python3.10/site-packages/axelera/runtime/objects.py", line 277, in list_devices
    _maybe_raise_error(self)
  File "/home/alsutton/.cache/axelera/venvs/83d579fa/lib/python3.10/site-packages/axelera/runtime/objects.py", line 91, in _maybe_raise_error
    _raise_error(ctx, err_no)
  File "/home/alsutton/.cache/axelera/venvs/83d579fa/lib/python3.10/site-packages/axelera/runtime/objects.py", line 83, in _raise_error
    raise exc(f"{err}: {msg}")
ConnectionError: AXR_ERROR_CONNECTION_ERROR: No target device found in lspci output
(venv) alsutton@svr220:~/Utils/voyager-sdk$ 

Done.

Done.

There’s no option for SAM/Resizable BAR in the BIOS/UEFI.

Changing motherboards isn’t an option, unfortunately. It’s more likely we’d go with a different accelerator card.

Thanks for your suggestions.

Here’s the manual for the system, which includes all the possible settings, for those who are interested; https://www.supermicro.com/manuals/motherboard/C612/MNL-1597.pdf

Above 4G decoding is supported. That's good to know.

I also tried to swap the expansion slots, but since your mother is dual socket, maybe you could try with 1 CPU?

It looks like you have the UEFI set up.

If possible, I would like to update the Metis AIPU firmware on other motherboards so that it behaves differently.

I popped the card out of a server and put it in a Dell T5810 desktop. It looks, to me, like the card is faulty.

I see another non-entry in `lspci -tv`;

 |           +-01.0-[01]--
 |           +-02.0-[02]--+-00.0  NVIDIA Corporation AD104GL [RTX 4000 Ada Generation]
 |           |            \-00.1  NVIDIA Corporation AD104 High Definition Audio Controller

(The RTX4000 Ada is in the next slot, so I’d expect to see the card at `01.0`)

I’ve gone through the rescan, etc., etc., etc. process, but still no joy.

Here’s what we see in dmesg on the 5810;

<    1.509977] pci 0000:00:01.0: 08086:6f02] type 01 class 0x060400 PCIe Root Port
<    1.510026] pci 0000:00:01.0: PCI bridge to gbus 01]
<    1.510031] pci 0000:00:01.0:   bridge window imem 0xf2000000-0xf40fffff]
<    1.510061] pci 0000:00:01.0: broken device, retraining non-functional downstream link at 2.5GT/s
<    2.506890] pci 0000:00:01.0: retraining failed
<    2.506960] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
<    2.513307] pci 0000:00:01.0: PCI bridge to gbus 01]
<    2.557694] pci 0000:00:01.0: PCI bridge to gbus 01]
<    2.557704] pci 0000:00:01.0:   bridge window imem 0xf2000000-0xf40fffff]

Given we’re in the UK, and import tax has already been paid, I’m going to draw a line under this and write the card off as a learning experience.

Thanks to everyone for their help.

Hi ​@alsutton! Hmm, yeah, that’s a good test of the card, and it is pointing towards a hardware issue.

We can look at getting a replacement out to you ASAP - I’ll DM you so we can get any required details together