Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] vfio-pci corrupts memory after recent world upgrade
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
Kresp
n00b
n00b


Joined: 17 Oct 2016
Posts: 73

PostPosted: Fri Oct 20, 2017 5:58 am    Post subject: [SOLVED] vfio-pci corrupts memory after recent world upgrade Reply with quote

I recently upgraded my system.
As part of it, qemu updated from 2.9.0-something to 2.10.0.
After upgrade, my Windows VM with GPU passthrough stopped working.

I dowgraded to 2.9.0-r56 (which is different revision then the old version I had. I think it was r54, but I'm not positive).

Now, with 2.9.0-r56, when I start VM with vfio-pci device, there are errors in dmesg I never had before (VM would start and usually work; these errors are only present if GPU is assigned to VM):
Code:

[   73.857120] vfio-pci 0000:25:00.0: enabling device (0000 -> 0003)
[   73.857319] vfio_ecap_init: 0000:25:00.0 hiding ecap 0x19@0x270
[   73.857328] vfio_ecap_init: 0000:25:00.0 hiding ecap 0x1b@0x2d0
[   73.857335] vfio_ecap_init: 0000:25:00.0 hiding ecap 0x1e@0x370
[   74.075965] AMD-Vi: Completion-Wait loop timed out
[   74.220074] AMD-Vi: Completion-Wait loop timed out
[   74.362429] AMD-Vi: Completion-Wait loop timed out
[   74.506765] AMD-Vi: Completion-Wait loop timed out
[   74.651175] AMD-Vi: Completion-Wait loop timed out
[   74.795324] AMD-Vi: Completion-Wait loop timed out
[   74.931650] AMD-Vi: Completion-Wait loop timed out
[   74.948106] AMD-Vi: Event logged [
[   74.948108] IOTLB_INV_TIMEOUT device=25:00.0 address=0x0000000ff87a80d0]
[   74.948109] AMD-Vi: Event logged [
[   74.948110] INVALID_DEVICE_REQUEST device=00:00.0 address=0xfffffffdf8000000 flags=0x0a00]
[   75.086088] AMD-Vi: Completion-Wait loop timed out
[   75.213104] AMD-Vi: Completion-Wait loop timed out
[   75.340122] AMD-Vi: Completion-Wait loop timed out
[   75.468146] AMD-Vi: Completion-Wait loop timed out
[   75.595162] AMD-Vi: Completion-Wait loop timed out
[   75.722181] AMD-Vi: Completion-Wait loop timed out
[   75.850210] AMD-Vi: Completion-Wait loop timed out
[   75.948169] AMD-Vi: Event logged [
[   75.948172] IOTLB_INV_TIMEOUT device=25:00.0 address=0x0000000ff87a81f0]

Sometimes it writes to memory used by embeded network controller:
Code:

[  843.620546] vfio-pci 0000:25:00.0: enabling device (0000 -> 0003)
[  843.620745] vfio_ecap_init: 0000:25:00.0 hiding ecap 0x19@0x270
[  843.620755] vfio_ecap_init: 0000:25:00.0 hiding ecap 0x1b@0x2d0
[  843.620762] vfio_ecap_init: 0000:25:00.0 hiding ecap 0x1e@0x370
[  843.829247] AMD-Vi: Completion-Wait loop timed out
[  843.963165] AMD-Vi: Completion-Wait loop timed out
[  844.090885] AMD-Vi: Completion-Wait loop timed out
[  844.221467] AMD-Vi: Completion-Wait loop timed out
[  844.350070] AMD-Vi: Completion-Wait loop timed out
[  844.480382] AMD-Vi: Completion-Wait loop timed out
[  844.608569] AMD-Vi: Completion-Wait loop timed out
[  844.701607] AMD-Vi: Event logged [
[  844.701609] IO_PAGE_FAULT device=1e:00.0 domain=0x000e address=0x00000000ffa7a438 flags=0x0050]
[  844.701610] AMD-Vi: Event logged [
[  844.701611] IO_PAGE_FAULT device=1e:00.0 domain=0x000e address=0x00000000fec89080 flags=0x0070]
[  844.701611] AMD-Vi: Event logged [
[  844.701612] IOTLB_INV_TIMEOUT device=25:00.0 address=0x0000000ff87a87f0]
[  844.701612] AMD-Vi: Event logged [
[  844.701613] INVALID_DEVICE_REQUEST device=00:00.0 address=0xfffffffdf8000000 flags=0x0a00]
[  844.830183] AMD-Vi: Completion-Wait loop timed out
[  844.956990] AMD-Vi: Completion-Wait loop timed out
[  845.083784] AMD-Vi: Completion-Wait loop timed out
[  845.212198] AMD-Vi: Completion-Wait loop timed out
[  845.339429] AMD-Vi: Completion-Wait loop timed out
[  845.466675] AMD-Vi: Completion-Wait loop timed out
[  845.594970] AMD-Vi: Completion-Wait loop timed out
[  845.701635] AMD-Vi: Event logged [
[  845.701637] IOTLB_INV_TIMEOUT device=25:00.0 address=0x0000000ff87a8910]
[  846.780840] igb 0000:1e:00.0: Detected Tx Unit Hang
                 Tx Queue             <1>
                 TDH                  <b>
                 TDT                  <4b>
                 next_to_use          <4b>
                 next_to_clean        <b>
               buffer_info[next_to_clean]
                 time_stamp           <100084fa0>
                 next_to_watch        <ffff880ff1e550d0>
                 jiffies              <1000857c0>
                 desc.status          <0>
[  846.780849] igb 0000:1e:00.0: Detected Tx Unit Hang
                 Tx Queue             <0>
                 TDH                  <ea>
                 TDT                  <d4>
                 next_to_use          <d4>
                 next_to_clean        <e8>
               buffer_info[next_to_clean]
                 time_stamp           <100084f44>
                 next_to_watch        <ffff880ff2aadeb0>
                 jiffies              <1000857c0>
                 desc.status          <0>
[  848.764920] igb 0000:1e:00.0: Detected Tx Unit Hang
                 Tx Queue             <1>
                 TDH                  <b>
                 TDT                  <4b>
                 next_to_use          <4b>
                 next_to_clean        <b>
               buffer_info[next_to_clean]
                 time_stamp           <100084fa0>
                 next_to_watch        <ffff880ff1e550d0>
                 jiffies              <100085f80>
                 desc.status          <0>
[  848.764930] igb 0000:1e:00.0: Detected Tx Unit Hang
                 Tx Queue             <0>
                 TDH                  <ea>
                 TDT                  <d4>
                 next_to_use          <d4>
                 next_to_clean        <e8>
               buffer_info[next_to_clean]
                 time_stamp           <100084f44>
                 next_to_watch        <ffff880ff2aadeb0>
                 jiffies              <100085f80>
                 desc.status          <0>
[  850.749095] igb 0000:1e:00.0: Detected Tx Unit Hang
                 Tx Queue             <1>
                 TDH                  <b>
                 TDT                  <4b>
                 next_to_use          <4b>
                 next_to_clean        <b>
               buffer_info[next_to_clean]
                 time_stamp           <100084fa0>
                 next_to_watch        <ffff880ff1e550d0>
                 jiffies              <100086740>
                 desc.status          <0>
[  850.749104] igb 0000:1e:00.0: Detected Tx Unit Hang
                 Tx Queue             <0>
                 TDH                  <ea>
                 TDT                  <d4>
                 next_to_use          <d4>
                 next_to_clean        <e8>
               buffer_info[next_to_clean]
                 time_stamp           <100084f44>
                 next_to_watch        <ffff880ff2aadeb0>
                 jiffies              <100086740>
                 desc.status          <0>
[  850.940775] NETDEV WATCHDOG: internal (igb): transmit queue 0 timed out
[  850.940784] ------------[ cut here ]------------
[  850.940789] WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:316 dev_watchdog+0x210/0x220
[  850.940789] Modules linked in: snd_hda_intel vfio_pci vfio_virqfd nvidia_drm(PO) nvidia_modeset(PO) nvidia(PO) nct6775 hwmon_vid snd_hda_codec_realtek snd_hda_codec_generic drm_kms_helper snd_hda_codec drm snd_hwdep snd_hda_core agpgart ccp efivarfs [last unloaded: snd_hda_intel]
[  850.940799] CPU: 0 PID: 0 Comm: swapper/0 Tainted: P           O    4.12.12-gentoo #1
[  850.940799] Hardware name: Micro-Star International Co., Ltd. MS-7A31/X370 XPOWER GAMING TITANIUM (MS-7A31), BIOS 1.90 09/19/2017
[  850.940800] task: ffffffff81e0f4c0 task.stack: ffffffff81e00000
[  850.940802] RIP: 0010:dev_watchdog+0x210/0x220
[  850.940802] RSP: 0018:ffff88103ec03e80 EFLAGS: 00010282
[  850.940803] RAX: 000000000000003b RBX: 0000000000000000 RCX: 0000000000000000
[  850.940804] RDX: ffff88103ec14690 RSI: ffff88103ec0caa8 RDI: ffff88103ec0caa8
[  850.940804] RBP: ffff88103ec03ea0 R08: 0000000000000001 R09: 0000000000000423
[  850.940805] R10: 0000000000000000 R11: 0000000000000423 R12: ffff880ff70cc000
[  850.940805] R13: 0000000000000000 R14: 0000000000000008 R15: ffff880ff70cc000
[  850.940806] FS:  0000000000000000(0000) GS:ffff88103ec00000(0000) knlGS:0000000000000000
[  850.940806] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  850.940807] CR2: 0000000000000000 CR3: 0000000fddf01000 CR4: 00000000003406f0
[  850.940808] Call Trace:
[  850.940808]  <IRQ>
[  850.940810]  ? qdisc_rcu_free+0x40/0x40
[  850.940812]  call_timer_fn+0x30/0x130
[  850.940813]  run_timer_softirq+0x1cd/0x400
[  850.940814]  ? ktime_get+0x36/0x90
[  850.940816]  ? lapic_next_event+0x18/0x20
[  850.940818]  ? clockevents_program_event+0x78/0xf0
[  850.940820]  __do_softirq+0xe2/0x270
[  850.940821]  irq_exit+0xab/0xb0
[  850.940822]  smp_apic_timer_interrupt+0x38/0x50
[  850.940824]  apic_timer_interrupt+0x86/0x90
[  850.940826] RIP: 0010:acpi_idle_do_entry+0x2c/0x40
[  850.940826] RSP: 0018:ffffffff81e03db0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10
[  850.940827] RAX: 0000000000000000 RBX: ffff880ff71c0000 RCX: 0000000000000034
[  850.940827] RDX: 4ec4ec4ec4ec4ec5 RSI: 0000000000000001 RDI: ffff880ff71c0064
[  850.940828] RBP: ffffffff81e03df0 R08: ffff88103ec170a4 R09: 0000000000000018
[  850.940828] R10: ffffffff81e03de0 R11: 0000000000000f64 R12: ffff880ff71c0064
[  850.940829] R13: 0000000000000001 R14: 0000000000000001 R15: 0000000000000001
[  850.940829]  </IRQ>
[  850.940832]  ? acpi_idle_enter+0xd6/0x280
[  850.940834]  cpuidle_enter_state+0xf5/0x2c0
[  850.940834]  cpuidle_enter+0x12/0x20
[  850.940836]  call_cpuidle+0x1e/0x30
[  850.940837]  do_idle+0x16f/0x1b0
[  850.940838]  cpu_startup_entry+0x18/0x20
[  850.940840]  rest_init+0x72/0x80
[  850.940841]  start_kernel+0x3bf/0x3cc
[  850.940843]  x86_64_start_reservations+0x2a/0x2c
[  850.940844]  x86_64_start_kernel+0x168/0x176
[  850.940845]  secondary_startup_64+0x9f/0x9f
[  850.940845] Code: 8c 24 24 04 00 00 eb 8f 4c 89 e7 c6 05 d3 e2 79 00 01 e8 54 c2 fd ff 89 d9 48 89 c2 4c 89 e6 48 c7 c7 60 07 c7 81 e8 31 e8 9e ff <0f> ff eb c1 66 90 66 2e 0f 1f 84 00 00 00 00 00 48 c7 47 08 00
[  850.940863] ---[ end trace 817628f0c0de7b66 ]---
[  850.940891] igb 0000:1e:00.0 internal: Reset adapter
[  854.551442] igb 0000:1e:00.0 internal: igb: internal NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX


Sometimes system can recover and continue working, but often it would start throwing segfaults randomly and eventually freeze.

Device 1e is network card, 25 is RX 480 guest GPU, 00 - host adapter.
I use GTX 960 with proprietary driver for host and vfio-pci autoloads for RX 480 on boot.
CPU is AMD R7 1800X and motherboard is MSI X370 Titanium. 4x16GB RAM, system is not overclocked, kernel 4.12.12.

Here's iomem map:
Code:

00000000-00000fff : Reserved
00001000-0009ffff : System RAM
000a0000-000fffff : Reserved
  000a0000-000bffff : PCI Bus 0000:00
  000c0000-000dffff : PCI Bus 0000:00
    000c0000-000ce7ff : Video ROM
  000f0000-000fffff : System ROM
00100000-09d7ffff : System RAM
  02600000-02f54e74 : Kernel code
  02f54e75-0354c83f : Kernel data
  03699000-03948fff : Kernel bss
09d80000-09ffffff : Reserved
0a000000-bccb7fff : System RAM
bccb8000-bcdf8fff : Reserved
bcdf9000-bceedfff : System RAM
bceee000-bd2c4fff : ACPI Non-volatile Storage
bd2c5000-be32afff : Reserved
be32b000-be3e8fff : Unknown E820 type
be3e9000-beffffff : System RAM
bf000000-bfffffff : Reserved
c0000000-fec2ffff : PCI Bus 0000:00
  c0000000-d1ffffff : PCI Bus 0000:24
    c0000000-cfffffff : 0000:24:00.0
    d0000000-d1ffffff : 0000:24:00.0
      d1000000-d12fffff : efifb
  e0000000-f01fffff : PCI Bus 0000:25
    e0000000-efffffff : 0000:25:00.0
      e0000000-efffffff : vfio-pci
    f0000000-f01fffff : 0000:25:00.0
      f0000000-f01fffff : vfio-pci
  f8000000-fbffffff : PCI MMCONFIG 0000 [bus 00-3f]
    f8000000-fbffffff : Reserved
      f8000000-fbffffff : pnp 00:00
  fc000000-fd0fffff : PCI Bus 0000:24
    fc000000-fcffffff : 0000:24:00.0
      fc000000-fcffffff : nvidia
    fd080000-fd083fff : 0000:24:00.1
      fd080000-fd083fff : ICH HD audio
  fd200000-fd5fffff : PCI Bus 0000:03
    fd200000-fd4fffff : PCI Bus 0000:04
      fd200000-fd2fffff : PCI Bus 0000:23
        fd200000-fd207fff : 0000:23:00.0
          fd200000-fd207fff : xhci-hcd
      fd300000-fd3fffff : PCI Bus 0000:22
        fd300000-fd303fff : 0000:22:00.0
          fd300000-fd303fff : nvme
      fd400000-fd4fffff : PCI Bus 0000:1e
        fd400000-fd41ffff : 0000:1e:00.0
          fd400000-fd41ffff : igb
        fd420000-fd423fff : 0000:1e:00.0
          fd420000-fd423fff : igb
    fd500000-fd57ffff : 0000:03:00.1
    fd580000-fd59ffff : 0000:03:00.1
      fd580000-fd59ffff : ahci
    fd5a0000-fd5a7fff : 0000:03:00.0
      fd5a0000-fd5a7fff : xhci-hcd
  fd600000-fd8fffff : PCI Bus 0000:26
    fd600000-fd6fffff : 0000:26:00.3
      fd600000-fd6fffff : xhci-hcd
    fd700000-fd7fffff : 0000:26:00.2
      fd700000-fd7fffff : ccp
    fd800000-fd801fff : 0000:26:00.2
      fd800000-fd801fff : ccp
  fd900000-fd9fffff : PCI Bus 0000:27
    fd900000-fd907fff : 0000:27:00.3
      fd900000-fd907fff : ICH HD audio
    fd908000-fd908fff : 0000:27:00.2
      fd908000-fd908fff : ahci
  fda00000-fdafffff : PCI Bus 0000:25
    fda00000-fda3ffff : 0000:25:00.0
      fda00000-fda3ffff : vfio-pci
    fda40000-fda5ffff : 0000:25:00.0
    fda60000-fda63fff : 0000:25:00.1
      fda60000-fda63fff : vfio-pci
  fdb00000-fdbfffff : PCI Bus 0000:01
    fdb00000-fdb03fff : 0000:01:00.0
      fdb00000-fdb03fff : nvme
  fdf00000-fdffffff : Reserved
  fea00000-fea0ffff : Reserved
  feb80000-fec01fff : Reserved
    feb80000-febfffff : amd_iommu
    fec00000-fec003ff : IOAPIC 0
    fec01000-fec013ff : IOAPIC 1
  fec10000-fec10fff : Reserved
    fec10000-fec10fff : pnp 00:05
fec30000-fec30fff : Reserved
  fec30000-fec30fff : AMDIF030:00
fed00000-fed00fff : Reserved
  fed00000-fed003ff : HPET 0
    fed00000-fed003ff : PNP0103:00
fed40000-fed44fff : Reserved
fed80000-fed8ffff : Reserved
  fed81500-fed818ff : AMDI0030:00
fedc0000-fedc0fff : pnp 00:05
fedc2000-fedcffff : Reserved
fedd4000-fedd5fff : Reserved
fee00000-ffffffff : PCI Bus 0000:00
  fee00000-feefffff : Reserved
    fee00000-fee00fff : Local APIC
      fee00000-fee00fff : pnp 00:05
  ff000000-ffffffff : Reserved
    ff000000-ffffffff : pnp 00:05
100000000-103f37ffff : System RAM
103f380000-103fffffff : RAM buffer


And lspci:
Code:

00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1450
   Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1450
   Flags: fast devsel

00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Device 1451
   Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1451
   Flags: fast devsel, IRQ 27
   Capabilities: [40] Secure device <?>
   Capabilities: [64] MSI: Enable+ Count=1/4 Maskable- 64bit+
   Capabilities: [74] HyperTransport: MSI Mapping Enable+ Fixed+

00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452
   Flags: fast devsel

00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 28
   Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
   Memory behind bridge: fdb00000-fdbfffff
   Capabilities: [50] Power Management version 3
   Capabilities: [58] Express Root Port (Slot+), MSI 00
   Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1453
   Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Capabilities: [150] Advanced Error Reporting
   Capabilities: [270] #19
   Capabilities: [2a0] Access Control Services
   Capabilities: [370] L1 PM Substates
   Capabilities: [3c4] #23
   Kernel driver in use: pcieport

00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 29
   Bus: primary=00, secondary=03, subordinate=23, sec-latency=0
   I/O behind bridge: 0000f000-0000ffff
   Memory behind bridge: fd200000-fd5fffff
   Capabilities: [50] Power Management version 3
   Capabilities: [58] Express Root Port (Slot+), MSI 00
   Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1453
   Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Capabilities: [150] Advanced Error Reporting
   Capabilities: [270] #19
   Capabilities: [2a0] Access Control Services
   Capabilities: [370] L1 PM Substates
   Capabilities: [3c4] #23
   Kernel driver in use: pcieport

00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452
   Flags: fast devsel

00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452
   Flags: fast devsel

00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 30
   Bus: primary=00, secondary=24, subordinate=24, sec-latency=0
   I/O behind bridge: 0000e000-0000efff
   Memory behind bridge: fc000000-fd0fffff
   Prefetchable memory behind bridge: 00000000c0000000-00000000d1ffffff
   Capabilities: [50] Power Management version 3
   Capabilities: [58] Express Root Port (Slot+), MSI 00
   Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1453
   Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Capabilities: [150] Advanced Error Reporting
   Capabilities: [270] #19
   Capabilities: [2a0] Access Control Services
   Capabilities: [370] L1 PM Substates
   Capabilities: [3c4] #23
   Kernel driver in use: pcieport

00:03.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1453 (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 31
   Bus: primary=00, secondary=25, subordinate=25, sec-latency=0
   I/O behind bridge: 0000d000-0000dfff
   Memory behind bridge: fda00000-fdafffff
   Prefetchable memory behind bridge: 00000000e0000000-00000000f01fffff
   Capabilities: [50] Power Management version 3
   Capabilities: [58] Express Root Port (Slot+), MSI 00
   Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1453
   Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Capabilities: [150] Advanced Error Reporting
   Capabilities: [270] #19
   Capabilities: [2a0] Access Control Services
   Capabilities: [370] L1 PM Substates
   Capabilities: [3c4] #23
   Kernel driver in use: pcieport

00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452
   Flags: fast devsel

00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452
   Flags: fast devsel

00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1454 (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 32
   Bus: primary=00, secondary=26, subordinate=26, sec-latency=0
   Memory behind bridge: fd600000-fd8fffff
   Capabilities: [50] Power Management version 3
   Capabilities: [58] Express Root Port (Slot-), MSI 00
   Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1454
   Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Capabilities: [270] #19
   Kernel driver in use: pcieport

00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1452
   Flags: fast devsel

00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1454 (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 34
   Bus: primary=00, secondary=27, subordinate=27, sec-latency=0
   Memory behind bridge: fd900000-fd9fffff
   Capabilities: [50] Power Management version 3
   Capabilities: [58] Express Root Port (Slot-), MSI 00
   Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1454
   Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Capabilities: [270] #19
   Kernel driver in use: pcieport

00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59)
   Subsystem: Micro-Star International Co., Ltd. [MSI] FCH SMBus Controller
   Flags: 66MHz, medium devsel
   Kernel driver in use: piix4_smbus

00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
   Subsystem: Micro-Star International Co., Ltd. [MSI] FCH LPC Bridge
   Flags: bus master, 66MHz, medium devsel, latency 0

00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1460
   Flags: fast devsel

00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1461
   Flags: fast devsel

00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1462
   Flags: fast devsel

00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1463
   Flags: fast devsel

00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1464
   Flags: fast devsel

00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1465
   Flags: fast devsel

00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1466
   Flags: fast devsel

00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1467
   Flags: fast devsel

01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961 (prog-if 02 [NVM Express])
   Subsystem: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961
   Flags: bus master, fast devsel, latency 0, IRQ 44, NUMA node 0
   Memory at fdb00000 (64-bit, non-prefetchable) [size=16K]
   Capabilities: [40] Power Management version 3
   Capabilities: [50] MSI: Enable- Count=1/32 Maskable- 64bit+
   Capabilities: [70] Express Endpoint, MSI 00
   Capabilities: [b0] MSI-X: Enable+ Count=8 Masked-
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [148] Device Serial Number 00-00-00-00-00-00-00-00
   Capabilities: [158] Power Budgeting <?>
   Capabilities: [168] #19
   Capabilities: [188] Latency Tolerance Reporting
   Capabilities: [190] L1 PM Substates
   Kernel driver in use: nvme

03:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 43b9 (rev 02) (prog-if 30 [XHCI])
   Subsystem: ASMedia Technology Inc. Device 1142
   Flags: bus master, fast devsel, latency 0, IRQ 69
   Memory at fd5a0000 (64-bit, non-prefetchable) [size=32K]
   Capabilities: [50] MSI: Enable+ Count=1/8 Maskable- 64bit+
   Capabilities: [78] Power Management version 3
   Capabilities: [80] Express Legacy Endpoint, MSI 00
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [200] #19
   Capabilities: [300] Latency Tolerance Reporting
   Capabilities: [400] L1 PM Substates
   Kernel driver in use: xhci_hcd

03:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] Device 43b5 (rev 02) (prog-if 01 [AHCI 1.0])
   Subsystem: ASMedia Technology Inc. Device 1062
   Flags: bus master, fast devsel, latency 0, IRQ 47
   Memory at fd580000 (32-bit, non-prefetchable) [size=128K]
   Expansion ROM at fd500000 [disabled] [size=512K]
   Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [78] Power Management version 3
   Capabilities: [80] Express Legacy Endpoint, MSI 00
   Capabilities: [100] Advanced Error Reporting
   Kernel driver in use: ahci

03:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b0 (rev 02) (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 35
   Bus: primary=03, secondary=04, subordinate=23, sec-latency=0
   I/O behind bridge: 0000f000-0000ffff
   Memory behind bridge: fd200000-fd4fffff
   Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [78] Power Management version 3
   Capabilities: [80] Express Upstream Port, MSI 00
   Capabilities: [c0] Subsystem: ASMedia Technology Inc. Device 0201
   Capabilities: [100] Advanced Error Reporting
   Kernel driver in use: pcieport

04:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 37
   Bus: primary=04, secondary=1e, subordinate=1e, sec-latency=0
   I/O behind bridge: 0000f000-0000ffff
   Memory behind bridge: fd400000-fd4fffff
   Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [78] Power Management version 3
   Capabilities: [80] Express Downstream Port (Slot+), MSI 00
   Capabilities: [c0] Subsystem: ASMedia Technology Inc. Device 3306
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [200] #19
   Capabilities: [400] L1 PM Substates
   Kernel driver in use: pcieport

04:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 38
   Bus: primary=04, secondary=1f, subordinate=1f, sec-latency=0
   Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [78] Power Management version 3
   Capabilities: [80] Express Downstream Port (Slot+), MSI 00
   Capabilities: [c0] Subsystem: ASMedia Technology Inc. Device 3306
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [200] #19
   Capabilities: [400] L1 PM Substates
   Kernel driver in use: pcieport

04:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 39
   Bus: primary=04, secondary=20, subordinate=20, sec-latency=0
   Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [78] Power Management version 3
   Capabilities: [80] Express Downstream Port (Slot+), MSI 00
   Capabilities: [c0] Subsystem: ASMedia Technology Inc. Device 3306
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [200] #19
   Capabilities: [400] L1 PM Substates
   Kernel driver in use: pcieport

04:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 41
   Bus: primary=04, secondary=21, subordinate=21, sec-latency=0
   Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [78] Power Management version 3
   Capabilities: [80] Express Downstream Port (Slot+), MSI 00
   Capabilities: [c0] Subsystem: ASMedia Technology Inc. Device 3306
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [200] #19
   Capabilities: [400] L1 PM Substates
   Kernel driver in use: pcieport

04:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 42
   Bus: primary=04, secondary=22, subordinate=22, sec-latency=0
   Memory behind bridge: fd300000-fd3fffff
   Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [78] Power Management version 3
   Capabilities: [80] Express Downstream Port (Slot+), MSI 00
   Capabilities: [c0] Subsystem: ASMedia Technology Inc. Device 3306
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [200] #19
   Capabilities: [400] L1 PM Substates
   Kernel driver in use: pcieport

04:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b4 (rev 02) (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 43
   Bus: primary=04, secondary=23, subordinate=23, sec-latency=0
   Memory behind bridge: fd200000-fd2fffff
   Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [78] Power Management version 3
   Capabilities: [80] Express Downstream Port (Slot+), MSI 00
   Capabilities: [c0] Subsystem: ASMedia Technology Inc. Device 3306
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [200] #19
   Capabilities: [400] L1 PM Substates
   Kernel driver in use: pcieport

1e:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)
   Subsystem: Micro-Star International Co., Ltd. [MSI] I211 Gigabit Network Connection
   Flags: bus master, fast devsel, latency 0, IRQ 36
   Memory at fd400000 (32-bit, non-prefetchable) [size=128K]
   I/O ports at f000 [size=32]
   Memory at fd420000 (32-bit, non-prefetchable) [size=16K]
   Capabilities: [40] Power Management version 3
   Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
   Capabilities: [70] MSI-X: Enable+ Count=5 Masked-
   Capabilities: [a0] Express Endpoint, MSI 00
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [140] Device Serial Number 4c-cc-6a-ff-ff-f5-30-06
   Capabilities: [1a0] Transaction Processing Hints
   Kernel driver in use: igb

22:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961 (prog-if 02 [NVM Express])
   Subsystem: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961
   Flags: bus master, fast devsel, latency 0, IRQ 36, NUMA node 0
   Memory at fd300000 (64-bit, non-prefetchable) [size=16K]
   Capabilities: [40] Power Management version 3
   Capabilities: [50] MSI: Enable- Count=1/32 Maskable- 64bit+
   Capabilities: [70] Express Endpoint, MSI 00
   Capabilities: [b0] MSI-X: Enable+ Count=8 Masked-
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [148] Device Serial Number 00-00-00-00-00-00-00-00
   Capabilities: [158] Power Budgeting <?>
   Capabilities: [168] #19
   Capabilities: [188] Latency Tolerance Reporting
   Capabilities: [190] L1 PM Substates
   Kernel driver in use: nvme

23:00.0 USB controller: ASMedia Technology Inc. Device 2142 (prog-if 30 [XHCI])
   Subsystem: Micro-Star International Co., Ltd. [MSI] Device 7a31
   Flags: bus master, fast devsel, latency 0, IRQ 70
   Memory at fd200000 (64-bit, non-prefetchable) [size=32K]
   Capabilities: [50] MSI: Enable+ Count=1/8 Maskable- 64bit+
   Capabilities: [78] Power Management version 3
   Capabilities: [80] Express Legacy Endpoint, MSI 00
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [200] #19
   Capabilities: [300] Latency Tolerance Reporting
   Kernel driver in use: xhci_hcd

24:00.0 VGA compatible controller: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1) (prog-if 00 [VGA controller])
   Subsystem: Gigabyte Technology Co., Ltd GM206 [GeForce GTX 960]
   Flags: bus master, fast devsel, latency 0, IRQ 79
   Memory at fc000000 (32-bit, non-prefetchable) [size=16M]
   Memory at c0000000 (64-bit, prefetchable) [size=256M]
   Memory at d0000000 (64-bit, prefetchable) [size=32M]
   I/O ports at e000 [size=128]
   [virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
   Capabilities: [60] Power Management version 3
   Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [78] Express Legacy Endpoint, MSI 00
   Capabilities: [100] Virtual Channel
   Capabilities: [250] Latency Tolerance Reporting
   Capabilities: [258] L1 PM Substates
   Capabilities: [128] Power Budgeting <?>
   Capabilities: [420] Advanced Error Reporting
   Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
   Capabilities: [900] #19
   Kernel driver in use: nvidia
   Kernel modules: nvidia_drm, nvidia

24:00.1 Audio device: NVIDIA Corporation Device 0fba (rev a1)
   Subsystem: Gigabyte Technology Co., Ltd Device 3690
   Flags: bus master, fast devsel, latency 0, IRQ 76
   Memory at fd080000 (32-bit, non-prefetchable) [size=16K]
   Capabilities: [60] Power Management version 3
   Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
   Capabilities: [78] Express Endpoint, MSI 00
   Kernel driver in use: snd_hda_intel
   Kernel modules: snd_hda_intel

25:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480] (rev c7) (prog-if 00 [VGA controller])
   Subsystem: Micro-Star International Co., Ltd. [MSI] Radeon RX 480
   Flags: bus master, fast devsel, latency 0, IRQ 83
   Memory at e0000000 (64-bit, prefetchable) [size=256M]
   Memory at f0000000 (64-bit, prefetchable) [size=2M]
   I/O ports at d000 [size=256]
   Memory at fda00000 (32-bit, non-prefetchable) [size=256K]
   Expansion ROM at fda40000 [disabled] [size=128K]
   Capabilities: [48] Vendor Specific Information: Len=08 <?>
   Capabilities: [50] Power Management version 3
   Capabilities: [58] Express Legacy Endpoint, MSI 00
   Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Capabilities: [150] Advanced Error Reporting
   Capabilities: [200] #15
   Capabilities: [270] #19
   Capabilities: [2b0] Address Translation Service (ATS)
   Capabilities: [2c0] Page Request Interface (PRI)
   Capabilities: [2d0] Process Address Space ID (PASID)
   Capabilities: [320] Latency Tolerance Reporting
   Capabilities: [328] Alternative Routing-ID Interpretation (ARI)
   Capabilities: [370] L1 PM Substates
   Kernel driver in use: vfio-pci
   Kernel modules: amdgpu

25:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Device aaf0
   Subsystem: Micro-Star International Co., Ltd. [MSI] Device aaf0
   Flags: bus master, fast devsel, latency 0, IRQ 82
   Memory at fda60000 (64-bit, non-prefetchable) [size=16K]
   Capabilities: [48] Vendor Specific Information: Len=08 <?>
   Capabilities: [50] Power Management version 3
   Capabilities: [58] Express Legacy Endpoint, MSI 00
   Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Capabilities: [150] Advanced Error Reporting
   Capabilities: [328] Alternative Routing-ID Interpretation (ARI)
   Kernel driver in use: vfio-pci
   Kernel modules: snd_hda_intel

26:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a
   Subsystem: Advanced Micro Devices, Inc. [AMD] Device 145a
   Flags: fast devsel
   Capabilities: [48] Vendor Specific Information: Len=08 <?>
   Capabilities: [50] Power Management version 3
   Capabilities: [64] Express Endpoint, MSI 00
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Capabilities: [270] #19

26:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Device 1456
   Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1456
   Flags: bus master, fast devsel, latency 0, IRQ 73
   Memory at fd700000 (32-bit, non-prefetchable) [size=1M]
   Memory at fd800000 (32-bit, non-prefetchable) [size=8K]
   Capabilities: [48] Vendor Specific Information: Len=08 <?>
   Capabilities: [50] Power Management version 3
   Capabilities: [64] Express Endpoint, MSI 00
   Capabilities: [a0] MSI: Enable- Count=1/2 Maskable- 64bit+
   Capabilities: [c0] MSI-X: Enable+ Count=2 Masked-
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Kernel driver in use: ccp
   Kernel modules: ccp

26:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 145c (prog-if 30 [XHCI])
   Subsystem: Micro-Star International Co., Ltd. [MSI] Device 7a31
   Flags: bus master, fast devsel, latency 0, IRQ 72
   Memory at fd600000 (64-bit, non-prefetchable) [size=1M]
   Capabilities: [48] Vendor Specific Information: Len=08 <?>
   Capabilities: [50] Power Management version 3
   Capabilities: [64] Express Endpoint, MSI 00
   Capabilities: [a0] MSI: Enable+ Count=1/8 Maskable- 64bit+
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Kernel driver in use: xhci_hcd

27:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455
   Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1455
   Flags: fast devsel
   Capabilities: [48] Vendor Specific Information: Len=08 <?>
   Capabilities: [50] Power Management version 3
   Capabilities: [64] Express Endpoint, MSI 00
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Capabilities: [270] #19

27:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0])
   Subsystem: Micro-Star International Co., Ltd. [MSI] FCH SATA Controller [AHCI mode]
   Flags: bus master, fast devsel, latency 0, IRQ 49
   Memory at fd908000 (32-bit, non-prefetchable) [size=4K]
   Capabilities: [48] Vendor Specific Information: Len=08 <?>
   Capabilities: [50] Power Management version 3
   Capabilities: [64] Express Endpoint, MSI 00
   Capabilities: [a0] MSI: Enable+ Count=1/16 Maskable- 64bit+
   Capabilities: [d0] SATA HBA v1.0
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Kernel driver in use: ahci

27:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Device 1457
   Subsystem: Micro-Star International Co., Ltd. [MSI] Device da31
   Flags: bus master, fast devsel, latency 0, IRQ 78
   Memory at fd900000 (32-bit, non-prefetchable) [size=32K]
   Capabilities: [48] Vendor Specific Information: Len=08 <?>
   Capabilities: [50] Power Management version 3
   Capabilities: [64] Express Endpoint, MSI 00
   Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
   Kernel driver in use: snd_hda_intel
   Kernel modules: snd_hda_intel



Last edited by Kresp on Fri Oct 20, 2017 6:53 am; edited 1 time in total
Back to top
View user's profile Send private message
Kresp
n00b
n00b


Joined: 17 Oct 2016
Posts: 73

PostPosted: Fri Oct 20, 2017 6:53 am    Post subject: Reply with quote

I checked newer, unstable, kernel version and on 4.13.5 this problem does not happen anymore:
Code:

[   63.028038] vfio-pci 0000:25:00.0: enabling device (0000 -> 0003)
[   63.028231] vfio_ecap_init: 0000:25:00.0 hiding ecap 0x19@0x270
[   63.028240] vfio_ecap_init: 0000:25:00.0 hiding ecap 0x1b@0x2d0
[   63.028247] vfio_ecap_init: 0000:25:00.0 hiding ecap 0x1e@0x370
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum