Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
RAM access from PCIe device
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
ranran
Tux's lil' helper
Tux's lil' helper


Joined: 11 Jul 2018
Posts: 75

PostPosted: Thu Nov 28, 2019 8:21 am    Post subject: RAM access from PCIe device Reply with quote

Hello,

I have a simple kernel module which allocates continuous physical memory (using dma_alloc_coherent) and provide it to PCIe connected FPGA device for DMA access, 64-bit addressing PCIe.
But on when FPGA tries to access the cpu RAM (using FPGA's dma), it can't access and fails. This is in our lenovo computer.

In first step, I suspected kernel 4.18, but using same OS, kernel, same kernel module, and FPGA works perfectly if I just move them into older computer !

So in the second step, I started suspecting the lenovo machine, but installing windows on same machine, we get that FPGA can do DMA without any issues.

Is there any idea what can cause such strange behavior ?

Thanks


Last edited by ranran on Thu Nov 28, 2019 10:22 am; edited 2 times in total
Back to top
View user's profile Send private message
roarinelk
Guru
Guru


Joined: 04 Mar 2004
Posts: 506

PostPosted: Thu Nov 28, 2019 9:02 am    Post subject: Reply with quote

Wild guess, does the newer computer have an IOMMU, and did you enable kernel support for it?
It is the IOMMU's job to block devices it does not know about from reading/writing address space it has not been allowed to.
EDIT: or append this to the new computers kernel commandline: "iommu=pt"
Back to top
View user's profile Send private message
ranran
Tux's lil' helper
Tux's lil' helper


Joined: 11 Jul 2018
Posts: 75

PostPosted: Thu Nov 28, 2019 10:21 am    Post subject: Reply with quote

Thanks for the idea, but it did not help adding iommu=pt.

Is it that IOMMU is per port (sata/pcix8 pxix16 etc ) or one global for the processor ?
I see list in https://en.wikipedia.org/wiki/List_of_IOMMU-supporting_hardware that table shows that some boards have iommu for pcix32 but not for pcix8 for example, it makes me confused.
Back to top
View user's profile Send private message
roarinelk
Guru
Guru


Joined: 04 Mar 2004
Posts: 506

PostPosted: Thu Nov 28, 2019 11:11 am    Post subject: Reply with quote

can you disable the iommu in the bios altogether?

the way I understand it, if you allocate dma-able memory on behalf of a device (dma_alloc_* and friends), it should
more or less automatically be added to the iommu to allow transfers. However, I haven't yet had to deal with
a situation like yours yet.
Back to top
View user's profile Send private message
ranran
Tux's lil' helper
Tux's lil' helper


Joined: 11 Jul 2018
Posts: 75

PostPosted: Thu Nov 28, 2019 1:44 pm    Post subject: Reply with quote

Hi,

I tried to add intel_iommu=off and I see in dmesg:
...
[ 0.000000] DMAR: IOMMU disabled
...
Doesn't it mean that iommu is disabled ?


Thanks
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum