Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] PCI mapping problem
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
jpsollie
Tux's lil' helper
Tux's lil' helper


Joined: 17 Aug 2013
Posts: 79

PostPosted: Sun Apr 05, 2020 7:00 am    Post subject: [SOLVED] PCI mapping problem Reply with quote

Hi everyone,

I have an issue with my PCI remapping on the 5.5.15 kernel.
I have 4 PCI devices: a NVME (x4) device, a SAS device(8x), and 2 GPUs (Amd Fiji)(x16), attached to a X570 motherboard.
The SAS + 2nd GPU controller are attached to the chipset, which in turn is connected via an X4 link to the CPU. the 1st GPU and the NVME device are connected directly.The last GPU is not detected: when I do "lspci", it's simply not there.

Initial dmesg points to a situation which "might" see the PCI mapping:

Code:

[    0.447929] PCI host bridge to bus 0000:00
[    0.449010] pci 0000:00:01.1: PCI bridge to [bus 01]
[    0.449719] pci 0000:02:00.0: 31.504 Gb/s available PCIe bandwidth, limited by 8 GT/s x4 link at 0000:00:01.2 (capable of 63.008 Gb/s with 8 GT/s x8 link) <-- the SAS controller, I guess
[    0.449780] pci 0000:00:01.2: PCI bridge to [bus 02-08]
[    0.456314] pci 0000:02:00.0: PCI bridge to [bus 03-08]
[    0.456657] pci 0000:04:00.0: 31.504 Gb/s available PCIe bandwidth, limited by 8 GT/s x4 link at 0000:00:01.2 (capable of 63.008 Gb/s with 8 GT/s x8 link) <--- why is it mapped twice?
[    0.456708] pci 0000:03:01.0: PCI bridge to [bus 04]
[    0.457258] pci 0000:03:04.0: PCI bridge to [bus 05]
[    0.457613] pci 0000:06:00.0: 31.504 Gb/s available PCIe bandwidth, limited by 8 GT/s x4 link at 0000:00:01.2 (capable of 252.048 Gb/s with 16 GT/s x16 link) <--The 2nd VGA card?
[    0.459992] pci 0000:03:08.0: PCI bridge to [bus 06]
[    0.460348] pci 0000:07:00.0: 31.504 Gb/s available PCIe bandwidth, limited by 8 GT/s x4 link at 0000:00:01.2 (capable of 252.048 Gb/s with 16 GT/s x16 link) <-- why is it mapped twice?
[    0.460416] pci 0000:03:09.0: PCI bridge to [bus 07]
[    0.460773] pci 0000:08:00.0: 31.504 Gb/s available PCIe bandwidth, limited by 8 GT/s x4 link at 0000:00:01.2 (capable of 252.048 Gb/s with 16 GT/s x16 link) <--why is it mapped 3 times?
[    0.460841] pci 0000:03:0a.0: PCI bridge to [bus 08]
[    0.461207] pci 0000:00:03.1: PCI bridge to [bus 09]
[    0.461709] pci 0000:00:07.1: PCI bridge to [bus 0a]
[    0.461926] pci 0000:00:08.1: PCI bridge to [bus 0b]


lspci -k, however, does not notice any 2nd GPU:
Code:

00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Root Complex
        Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Root Complex
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) I/O Memory Management Unit
        Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) I/O Memory Management Unit
00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge
        Kernel driver in use: pcieport
00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge
        Kernel driver in use: pcieport
00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge
        Kernel driver in use: pcieport
00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
        Kernel driver in use: pcieport
00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
        Kernel driver in use: pcieport
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59)
        Subsystem: Gigabyte Technology Co., Ltd FCH SMBus Controller
        Kernel modules: i2c_piix4
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
        Subsystem: Gigabyte Technology Co., Ltd FCH LPC Bridge
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3
        Kernel driver in use: k10temp
        Kernel modules: k10temp
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5
00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6
00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7
01:00.0 Non-Volatile memory controller: Phison Electronics Corporation Device 5013 (rev 01)
        Subsystem: Phison Electronics Corporation Device 5013
        Kernel driver in use: nvme
02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream
        Kernel driver in use: pcieport
03:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
        Kernel driver in use: pcieport
03:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
        Kernel driver in use: pcieport
03:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
        Kernel driver in use: pcieport
03:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
        Kernel driver in use: pcieport
03:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
        Kernel driver in use: pcieport
04:00.0 RAID bus controller: Adaptec Series 7 6G SAS/PCIe 3 (rev 01)
        Subsystem: Adaptec Series 7 - ASR-72405 - 24 internal 0 external 6G SAS Port/PCIe 3.0
        Kernel driver in use: aacraid
        Kernel modules: aacraid
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 16)
        Subsystem: Gigabyte Technology Co., Ltd Onboard Ethernet
        Kernel driver in use: r8169
06:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
06:00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
        Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
        Kernel driver in use: xhci_hcd
06:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
        Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
        Kernel driver in use: xhci_hcd
07:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode]
        Kernel driver in use: ahci
08:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode]
        Kernel driver in use: ahci
09:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Fiji [Radeon R9 FURY / NANO Series] (rev ca)
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Radeon R9 FURY X / NANO
        Kernel driver in use: amdgpu
        Kernel modules: amdgpu
09:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Fiji HDMI/DP Audio [Radeon R9 Nano / FURY/FURY X]
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Fiji HDMI/DP Audio [Radeon R9 Nano / FURY/FURY X]
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel
0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function
        Subsystem: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function
0a:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor
        Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor
        Kernel driver in use: ccp
0a:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Zeppelin USB 3.0 Host controller
        Subsystem: Gigabyte Technology Co., Ltd Zeppelin USB 3.0 Host controller
        Kernel driver in use: xhci_hcd
0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function
        Subsystem: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function
0b:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
        Subsystem: Gigabyte Technology Co., Ltd FCH SATA Controller [AHCI mode]
        Kernel driver in use: ahci


Where does it stop detecting? I thought it was the driver, and switched from amdgpu to amdgpu-pro (and back), but it did not help.
If it's the chipset, why does it detect the SAS device which is also connected via the x570 chipset?

Thank you for your help!
_________________
I am using gentoo for over 10 years now. yet seen its possibilities, I still feel like a n00b...


Last edited by jpsollie on Fri Apr 10, 2020 8:16 am; edited 1 time in total
Back to top
View user's profile Send private message
jpsollie
Tux's lil' helper
Tux's lil' helper


Joined: 17 Aug 2013
Posts: 79

PostPosted: Sun Apr 05, 2020 8:50 am    Post subject: Reply with quote

Maybe this is the problem (just an idea):
[mem 0xf0000000-0xf7ffffff] available for PCI devices
... this is 8G. When you count 2 GPUs with 4GB of VRAM + SAS controller, it will not work (and other devices are not included yet ^^.
How do I increase PCIe memory space?
_________________
I am using gentoo for over 10 years now. yet seen its possibilities, I still feel like a n00b...
Back to top
View user's profile Send private message
jpsollie
Tux's lil' helper
Tux's lil' helper


Joined: 17 Aug 2013
Posts: 79

PostPosted: Fri Apr 10, 2020 8:16 am    Post subject: Reply with quote

Nevermind, the issue was at UEFI side:

The manufacturer(Gigabyte) made a new BIOS revision which (ironically) "improves PCIe compatibility"
All bioses beyond that point (F11) have this error.
reverting to F10 solves the problem.
I reported the error to Gigabyte, let's hope they fix it.
It may be present on other devices as well: for example, my MSI B450 mainboard also has a "Improved PCIe compatibility" bios update which was released at around the same time.
Let's see where this goes...
_________________
I am using gentoo for over 10 years now. yet seen its possibilities, I still feel like a n00b...
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum