Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
How do I compile my kernel for AMD IOMMU PCI passthrough?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
RAKK
n00b
n00b


Joined: 12 Feb 2016
Posts: 17

PostPosted: Thu Jun 30, 2016 6:27 am    Post subject: How do I compile my kernel for AMD IOMMU PCI passthrough? Reply with quote

I have an AMD A4-5300 CPU with a motherboard that supports IOMMU and I am trying to enable IOMMU on my Gentoo Hardened 4.3.3-r4 kernel with the intention of using a graphics card on a KVM VM. I've already asked this question in this other forum thread and got an answer, but I just tried to do that and it didn't work.

According to the reply I got, I have to enable the following options on my Linux kernel and recompile it:

  1. Device Drivers >> IOMMU Hardware Support
  2. AMD IOMMU support
  3. Support for Intel IOMMU using DMA Remapping Devices
  4. Support for Interrupt Remapping
  5. Bus options (PCI etc.) >> PCI Stub

So I did, and yet my kernel still won't pick up my IOMMU. When it works I'm supposed to get a message saying AMD-Vi was enabled if I issue the following command, but I just get nothing:

Code:
dmesg | grep AMD-Vi


I've already managed to get my IOMMU working on Debian Stretch on this same computer, so the point of failure is most definitely in my Gentoo system. Documentation on using IOMMU on Gentoo is very scarce though, all I can find online is for Linux distributions that Just Work Out of the Box™ like Ubuntu, Fedora or Debian. What can I do to enable my IOMMU on Gentoo?
Back to top
View user's profile Send private message
i4dnf
Apprentice
Apprentice


Joined: 18 Sep 2005
Posts: 266
Location: Bucharest, Romania

PostPosted: Thu Jun 30, 2016 8:34 am    Post subject: Reply with quote

Try just:
Code:
iommu=pt

in your kernel parameters
_________________
"The only difference between me and a madman is that I am not MAD" (SALVATOR DALI)
Back to top
View user's profile Send private message
theotherjoe
Guru
Guru


Joined: 22 Nov 2003
Posts: 371

PostPosted: Thu Jun 30, 2016 2:53 pm    Post subject: Reply with quote

Code:
localhost # dmesg | grep AMD-Vi
[    0.676298] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40
[    0.676379] AMD-Vi: Interrupt remapping enabled
[    0.676518] AMD-Vi: Lazy IO/TLB flushing enabled
localhost #


Code:
localhost # grep IOMMU /usr/src/linux/.config
# CONFIG_GART_IOMMU is not set
# CONFIG_CALGARY_IOMMU is not set
CONFIG_IOMMU_HELPER=y
CONFIG_IOMMU_API=y
CONFIG_IOMMU_SUPPORT=y
# Generic IOMMU Pagetable Support
CONFIG_AMD_IOMMU=y
# CONFIG_AMD_IOMMU_STATS is not set
CONFIG_AMD_IOMMU_V2=y
# CONFIG_INTEL_IOMMU is not set
# CONFIG_IOMMU_STRESS is not set
Back to top
View user's profile Send private message
RAKK
n00b
n00b


Joined: 12 Feb 2016
Posts: 17

PostPosted: Thu Jun 30, 2016 3:15 pm    Post subject: Reply with quote

i4dnf wrote:
Try just:
Code:
iommu=pt

in your kernel parameters

Just tried this one, only amd_iommu=on and both at the same time. It didn't work.
Back to top
View user's profile Send private message
RAKK
n00b
n00b


Joined: 12 Feb 2016
Posts: 17

PostPosted: Thu Jun 30, 2016 3:22 pm    Post subject: Reply with quote

theotherjoe wrote:
Code:
localhost # grep IOMMU /usr/src/linux/.config
# CONFIG_GART_IOMMU is not set
# CONFIG_CALGARY_IOMMU is not set
CONFIG_IOMMU_HELPER=y
CONFIG_IOMMU_API=y
CONFIG_IOMMU_SUPPORT=y
# Generic IOMMU Pagetable Support
CONFIG_AMD_IOMMU=y
# CONFIG_AMD_IOMMU_STATS is not set
CONFIG_AMD_IOMMU_V2=y
# CONFIG_INTEL_IOMMU is not set
# CONFIG_IOMMU_STRESS is not set

Just tried disabling GART_IOMMU and CALGARY_IOMMU. Kernel is rebuilding at the moment.
Back to top
View user's profile Send private message
RAKK
n00b
n00b


Joined: 12 Feb 2016
Posts: 17

PostPosted: Fri Jul 01, 2016 4:21 am    Post subject: Reply with quote

Aaaaaaand no luck with GART_IOMMU and CALGARY_IOMMU disabled :(

What else do I have to enable?

Edit: from this official kernel guide I'm trying booting with iommu=force:

Code:
    force              Force the use of the hardware IOMMU even when it is
                       not actually needed (e.g. because < 3 GB memory).


Edit 2: still no luck, my IOMMU is still not enabling.
Back to top
View user's profile Send private message
theotherjoe
Guru
Guru


Joined: 22 Nov 2003
Posts: 371

PostPosted: Fri Jul 01, 2016 6:29 am    Post subject: Reply with quote

RAKK,
I don't have a series A apu available to compare. the config snippet
was from a system with a FX cpu.
Maybe the complete config and dmesg output can help you analyse your problem.

complete config: https://bpaste.net/show/e15d49194150
dmesg output: https://bpaste.net/show/6df62f665000

Since you mentioned that you have IOMMU available under Debian on
the same system I assume that IOMMU controller option or something
similar in your bios is turned on.
Back to top
View user's profile Send private message
RAKK
n00b
n00b


Joined: 12 Feb 2016
Posts: 17

PostPosted: Fri Jul 01, 2016 4:21 pm    Post subject: Reply with quote

theotherjoe wrote:
RAKK,
I don't have a series A apu available to compare. the config snippet
was from a system with a FX cpu.
Maybe the complete config and dmesg output can help you analyse your problem.

complete config: https://bpaste.net/show/e15d49194150
dmesg output: https://bpaste.net/show/6df62f665000

I just saw you have NUMA disabled. Turns out I enabled NUMA some months ago to deal with a memory problem I had on my kernel. I'm currently rebuilding mine with CONFIG_NUMA disabled. The issue I had can be found here: http://unix.stackexchange.com/questions/265629/gentoo-hardened-eats-3-gb-of-memory-after-running-for-12-hours-with-very-few-pr

If it works... that's going to be a problem, because that memory problem might appear again.
Back to top
View user's profile Send private message
RAKK
n00b
n00b


Joined: 12 Feb 2016
Posts: 17

PostPosted: Fri Jul 01, 2016 5:24 pm    Post subject: Reply with quote

And once again, IOMMU still won't work with NUMA disabled. This one is good news though.

Well... I'm running out of ideas...

Edit: Rehosting the previous paste files since they seem to have an expiration date.

Kernel config: http://pastebin.com/SncJT24v
dmesg: http://pastebin.com/c80b3Nev
Back to top
View user's profile Send private message
RAKK
n00b
n00b


Joined: 12 Feb 2016
Posts: 17

PostPosted: Sat Jul 09, 2016 11:02 am    Post subject: Reply with quote

And after much wailing, complaining and head bashing, I finally figured out what was going on.

Turns out... that IOMMU was working all along, it's just that the kernel logs weren't showing it.

But the IOMMU driver was there, and I serendipiously found out when I mistakenly typed this:

Code:
rockshooter ~ # ps -ef | grep -i iommu
root        66     2  0 04:19 ?        00:00:00 [amd_iommu_v2]


When I saw that kernel thread running I thought, maybe my IOMMU is actually working? And lo and behold, I performed the same configuration steps I did previously for Debian, and one hour later I was on my virtual machine hooked up to my graphics card playing some games.

Please save this thread for future reference, because it took me some good 3 weeks to figure this out.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum