Kernel driver: drm_prime_sg_to_page_addr_arrays was removed in Linux 5.12 #5943

j-stephan · 2021-10-19T17:45:37Z

In Linux 5.12 the function drm_prime_sg_to_page_addr_arrays was split into two parts (AFAIK with no previous deprecation warning):

drm_prime_sg_to_page_array - marked as deprecated
drm_prime_sg_to_dma_addr_array - not deprecated, seems to be the suggested replacement for the above.

This causes DKMS to fail on up-to-date Linux distributions, such as Ubuntu 21.10:

  CC [M]  /var/lib/dkms/xrt/2.13.0/build/driver/xocl/userpf/xocl_bo.o
/var/lib/dkms/xrt/2.13.0/build/driver/xocl/userpf/xocl_bo.c: In function ‘xocl_gem_prime_import_sg_table’:
/var/lib/dkms/xrt/2.13.0/build/driver/xocl/userpf/xocl_bo.c:1175:15: error: implicit declaration of function ‘drm_prime_sg_to_page_addr_arrays’; did you mean ‘drm_prime_sg_to_dma_addr_array’? [-Werror=implicit-function-declaration]
 1175 |         ret = drm_prime_sg_to_page_addr_arrays(sgt, importing_xobj->pages,
      |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |               drm_prime_sg_to_dma_addr_array

The text was updated successfully, but these errors were encountered:

keryell · 2021-10-20T00:12:08Z

@rbramand-xilinx we need you again!

rogumag · 2021-11-04T13:54:30Z

Hello, I have the same error using Kernel 4.18.0-348.el8.x86_64 (Centos 8).

`CC [M]  /var/lib/dkms/xrt/2.12.427/build/driver/xocl/userpf/xocl_bo.o
/var/lib/dkms/xrt/2.12.427/build/driver/xocl/userpf/xocl_bo.c: In function ‘xocl_gem_prime_import_sg_table’:
/var/lib/dkms/xrt/2.12.427/build/driver/xocl/userpf/xocl_bo.c:1175:8: error: implicit declaration of function ‘drm_prime_sg_to_page_addr_arrays’; did you mean ‘drm_prime_sg_to_dma_addr_array’? [-Werror=implicit-function-declaration]
  ret = drm_prime_sg_to_page_addr_arrays(sgt, importing_xobj->pages,
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        drm_prime_sg_to_dma_addr_array
/var/lib/dkms/xrt/2.12.427/build/driver/xocl/userpf/xocl_bo.c: In function ‘xocl_gem_prime_mmap’:
/var/lib/dkms/xrt/2.12.427/build/driver/xocl/userpf/xocl_bo.c:1260:39: error: ‘const struct drm_driver’ has no member named ‘gem_vm_ops’
   vma->vm_ops = xobj->base.dev->driver->gem_vm_ops;
                                       ^~
cc1: some warnings being treated as errors`

Best regards

j-stephan · 2021-12-01T08:55:59Z

Until a proper fix / modernization is available, the following workaround works for kernel 5.13:

XRT/src/runtime_src/core/pcie/driver/linux/xocl/userpf/xocl_bo.c

Lines 1177 to 1178 in 34849b8

    
           ret = drm_prime_sg_to_page_addr_arrays(sgt, importing_xobj->pages, 
        
                  NULL, attach->dmabuf->size >> PAGE_SHIFT);

needs to be changed to

ret = drm_prime_sg_to_page_array(sgt, importing_xobj->pages,
       attach->dmabuf->size >> PAGE_SHIFT);

@rogumag Your second error should have been fixed as part of #5867. I guess the check for Red Hat's backported kernel features was missed (ping @rbramand-xilinx). Try the following change:

XRT/src/runtime_src/core/pcie/driver/linux/xocl/userpf/xocl_bo.c

Lines 1259 to 1263 in 34849b8

    
           #if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 11, 0) 
        
           		vma->vm_ops = xobj->base.funcs->vm_ops; 
        
           #else 
        
           		vma->vm_ops = xobj->base.dev->driver->gem_vm_ops; 
        
           #endif

to

#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 11, 0)
		vma->vm_ops = xobj->base.funcs->vm_ops;
#elif defined(RHEL_RELEASE_CODE)
#    if RHEL_RELEASE_CODE >= RHEL_RELEASE_VERSION(8,4)
		vma->vm_ops = xobj->base.funcs->vm_ops;
#    else
        vma->vm_ops = xobj->base.dev->driver->gem_vm_ops;
#    endif
#else
		vma->vm_ops = xobj->base.dev->driver->gem_vm_ops;
#endif

If you're not on CentOS 8.4 change the RHEL_RELEASE_VERSION(8,4) to match your version. I hope that helps.

keryell · 2021-12-02T01:06:56Z

@j-stephan thanks for your continuous support. By the way, Xilinx is hiring... ;-) @sonals @stsoe

rspotoaz · 2021-12-03T17:08:44Z

After making the prescribed changes, I was still unable to compile the drivers on 4.18.0-348. I had to make an additional change to xocl_drm.c.

XRT/src/runtime_src/core/pcie/driver/linux/xocl/userpf/xocl_drm.c

Lines 511 to 525 in d807afc

    
           #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 11, 0) 
        
           	#if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 7, 0) 
        
           		.gem_free_object_unlocked       = xocl_free_object, 
        
           	#else 
        
           		.gem_free_object		= xocl_free_object, 
        
           	#endif 
        
           #endif 
        
           #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 11, 0) 
        
                   .gem_vm_ops                     = &xocl_vm_ops, 
        
                   .gem_prime_get_sg_table         = xocl_gem_prime_get_sg_table, 
        
                   .gem_prime_vmap                 = xocl_gem_prime_vmap, 
        
                   .gem_prime_vunmap               = xocl_gem_prime_vunmap, 
        
                   .gem_prime_export               = drm_gem_prime_export, 
        
           #endif

I don't know exactly which kernel versions apply or when the source changed, but I essentially modified lines 511 and 519 to align w/ 5.11 by dropping the minimum version to 4,18,0

#if LINUX_VERSION_CODE < KERNEL_VERSION(4, 18, 0)

#if LINUX_VERSION_CODE < KERNEL_VERSION(4, 18, 0)

After that change, the drivers compiled successfully with only a few warnings which appear to be benign.

 CC [M]  /var/lib/dkms/xrt/2.12.427/build/driver/xocl/userpf/xocl_drm.o
/var/lib/dkms/xrt/2.12.427/build/driver/xocl/userpf/xocl_drm.c:58:13: warning: ‘xocl_free_object’ defined but not used [-Wunused-function]
 static void xocl_free_object(struct drm_gem_object *obj)

and

 CC [M]  /var/lib/dkms/xrt/2.12.427/build/driver/xocl/mgmtpf/mgmt-utils.o
/var/lib/dkms/xrt/2.12.427/build/driver/xocl/mgmtpf/mgmt-utils.c: In function ‘xclmgmt_reset_pci’:
/var/lib/dkms/xrt/2.12.427/build/driver/xocl/mgmtpf/mgmt-utils.c:581:2: warning: ignoring return value of  pci_enable_device’, declared with attribute warn_unused_result [-Wunused-result]
  pci_enable_device(pdev);
  ^~~~~~~~~~~~~~~~~~~~~~~
  CC [M]  /var/lib/dkms/xrt/2.12.427/build/driver/xocl/mgmtpf/mgmt-bifurcation-reset.o
/var/lib/dkms/xrt/2.12.427/build/driver/xocl/mgmtpf/mgmt-bifurcation-reset.c: In function ‘xclmgmt_reset_pci_post’:
/var/lib/dkms/xrt/2.12.427/build/driver/xocl/mgmtpf/mgmt-bifurcation-reset.c:87:2: warning: ignoring return value of ‘pci_enable_device’, declared with attribute warn_unused_result [-Wunused-result]
  pci_enable_device(pdev);

I'm pretty sure that's not the right way to change the source, but for my purposes it got the job done so if anyone else comes across this and just needs to compile hopefully this helps.

keryell · 2021-12-04T01:04:56Z

@j-stephan can you make a PR so you are credited for it, at least for normal kernels like the ones found in Debian/Ubuntu?
For RedHat kernel, it is more difficult with their Frankenstein's kernel. :-( But if people are using RedHat, this is to pay to get support. So one can just ask RedHat how much is it to have XRT fixed??? ;-)

j-stephan · 2021-12-07T13:03:34Z

can you make a PR so you are credited for it, at least for normal kernels like the ones found in Debian/Ubuntu?

I'll try to submit one this (European) evening. Unless @rbramand-xilinx wants to do this first?

rbramand-xilinx · 2021-12-08T05:37:37Z

@j-stephan thanks for making the changes, please go ahead and submit your PR.

rogumag · 2021-12-09T10:15:09Z

Until a proper fix / modernization is available, the following workaround works for kernel 5.13:

XRT/src/runtime_src/core/pcie/driver/linux/xocl/userpf/xocl_bo.c

Lines 1177 to 1178 in 34849b8

ret = drm_prime_sg_to_page_addr_arrays(sgt, importing_xobj->pages,

NULL, attach->dmabuf->size >> PAGE_SHIFT);

needs to be changed to
ret = drm_prime_sg_to_page_array(sgt, importing_xobj->pages,
       attach->dmabuf->size >> PAGE_SHIFT);
@rogumag Your second error should have been fixed as part of #5867. I guess the check for Red Hat's backported kernel features was missed (ping @rbramand-xilinx). Try the following change:

XRT/src/runtime_src/core/pcie/driver/linux/xocl/userpf/xocl_bo.c

Lines 1259 to 1263 in 34849b8

#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 11, 0)

vma->vm_ops = xobj->base.funcs->vm_ops;

#else

vma->vm_ops = xobj->base.dev->driver->gem_vm_ops;

#endif

to
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 11, 0)
		vma->vm_ops = xobj->base.funcs->vm_ops;
#elif defined(RHEL_RELEASE_CODE)
#    if RHEL_RELEASE_CODE >= RHEL_RELEASE_VERSION(8,4)
		vma->vm_ops = xobj->base.funcs->vm_ops;
#    else
        vma->vm_ops = xobj->base.dev->driver->gem_vm_ops;
#    endif
#else
		vma->vm_ops = xobj->base.dev->driver->gem_vm_ops;
#endif
If you're not on CentOS 8.4 change the RHEL_RELEASE_VERSION(8,4) to match your version. I hope that helps.

Thanks for the modification. I will try it. Best regards

keryell · 2021-12-21T22:53:36Z

While debugging my installation recipe https://github.com/keryell/sycl/blob/vitis-2021.2/sycl/doc/GettingStartedXilinxFPGA.md, it looks like in current master 6b2803f the kernel driver compiled on my machine also with 5.13, besides the expected 5.11 I am using in the meantime.
I had to merging locally #6088 and #6089.
I have not tried to run the new kernel, because I am 10000 km away from my machine for a few weeks, so just rebooting several times on the same kernel version is scary enough, but this looks like we are making some progress here. :-)
Curious, @maxzhen any idea of what fixed this? There has been a lot of commits recently, including some removing some old unused code....

keryell · 2021-12-22T18:28:19Z

There is also #6092 which should improve the situation.

keryell · 2022-02-22T17:46:47Z

What is the status of this?
I am just trying the latest master branch and it looks like it compiles on Ubuntu 21.10 Linux 5.13.0-30.
I have not tried whether it works with our U200 because the machine is booked by some debugging work.

stsoe · 2022-02-22T17:50:57Z

I believe this is fixed. Sorry don't remember PR number, but I think there were several.

keryell · 2022-02-22T18:09:36Z

Great!
Thanks.

uday610 assigned chvamshi-xilinx Dec 2, 2021

chvamshi-xilinx assigned rbramand-xilinx and unassigned chvamshi-xilinx Dec 6, 2021

stsoe closed this as completed Feb 22, 2022

This was referenced Oct 19, 2022

compile issues klogg/fl2000_drm#55

Open

error in make kernel 5.15.0-48 klogg/fl2000_drm#54

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Kernel driver: drm_prime_sg_to_page_addr_arrays was removed in Linux 5.12 #5943

Kernel driver: drm_prime_sg_to_page_addr_arrays was removed in Linux 5.12 #5943

j-stephan commented Oct 19, 2021

keryell commented Oct 20, 2021

rogumag commented Nov 4, 2021 •

edited

Loading

j-stephan commented Dec 1, 2021 •

edited

Loading

keryell commented Dec 2, 2021

rspotoaz commented Dec 3, 2021

keryell commented Dec 4, 2021

j-stephan commented Dec 7, 2021

rbramand-xilinx commented Dec 8, 2021

rogumag commented Dec 9, 2021

keryell commented Dec 21, 2021

keryell commented Dec 22, 2021

keryell commented Feb 22, 2022

stsoe commented Feb 22, 2022

keryell commented Feb 22, 2022

Kernel driver: drm_prime_sg_to_page_addr_arrays was removed in Linux 5.12 #5943

Kernel driver: drm_prime_sg_to_page_addr_arrays was removed in Linux 5.12 #5943

Comments

j-stephan commented Oct 19, 2021

keryell commented Oct 20, 2021

rogumag commented Nov 4, 2021 • edited Loading

j-stephan commented Dec 1, 2021 • edited Loading

keryell commented Dec 2, 2021

rspotoaz commented Dec 3, 2021

keryell commented Dec 4, 2021

j-stephan commented Dec 7, 2021

rbramand-xilinx commented Dec 8, 2021

rogumag commented Dec 9, 2021

keryell commented Dec 21, 2021

keryell commented Dec 22, 2021

keryell commented Feb 22, 2022

stsoe commented Feb 22, 2022

keryell commented Feb 22, 2022

rogumag commented Nov 4, 2021 •

edited

Loading

j-stephan commented Dec 1, 2021 •

edited

Loading