[Linux] Build issue in 5.9.2/5.9.3. No speed_hi in struct ethtool_cmd #2164

lmunch · 2022-10-25T08:18:42Z

Summary

OS: Linux CentOS 5
Architecture: x86_64
Psutil version: 5.9.2/5.9.3
Python version: Custom build 3.11.0
Type: build issue

Description

On CentOS 5 (yes, I know it's old) ethtool.h does not have speed_hi in struct ethtool_cmd and it also does not have the inline function ethtool_cmd_speed.

On 5.9.2 I get:

  gcc-4.8 -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=592 -DPSUTIL_LINUX=1 -DPSUTIL_ETHTOOL_MISSING_TYPES=1 -I/opt/python3/include/python3.11 -c psutil/_psutil_linux.c -o build/temp.linux-x86_64-cpython-311/psutil/_psutil_linux.o
  psutil/_psutil_linux.c: In function ‘psutil_net_if_duplex_speed’:
  psutil/_psutil_linux.c:444:9: warning: implicit declaration of function ‘ethtool_cmd_speed’ [-Wimplicit-function-declaration]
           uint_speed = ethtool_cmd_speed(&ethcmd);
           ^

On 5.9.3 I get:

3.11 -c psutil/_psutil_linux.c -o build/temp.linux-x86_64-cpython-311/psutil/_psutil_linux.o
  psutil/_psutil_linux.c: In function ‘psutil_ethtool_cmd_speed’:
  psutil/_psutil_linux.c:79:17: error: ‘const struct ethtool_cmd’ has no member named ‘speed_hi’
       return (ecmd->speed_hi << 16) | ecmd->speed;

Version 5.9.1 is fine.

Thanks

The text was updated successfully, but these errors were encountered:

lmunch · 2022-10-25T18:26:03Z

I guess get_ethtool_macro function needs to be extended in setup.py to check for speed_hi member?

giampaolo · 2022-11-09T00:43:46Z

struct ethtool_cmd’ has no member named ‘speed_hi’

This was likely introduced in d4eea1f. Adjusting this to work on CentOS 5 and keeping the bugfix added by d4eea1f is difficult. That would require a pre-processor check as in (pseudo code):

#ifdef CENTOS_5
...  // ignore speed_hi
#else
...
#endif

...which would theoretically fix this problem if psutil is compiled from sources, but since we distribute binaries pip would probably still download binaries rather than the sources (although I'm not 100% sure) and fail. I took a look at https://en.wikipedia.org/wiki/CentOS. CentOS 5 was released in 2007 and support ended in 2017. Considering it's that ancient I'm keen on dropping support and closing this the ticket as "won't fix". As a middle ground we may document this in the doc (https://psutil.readthedocs.io/en/latest/#platforms-support-history) so that people on CentOS 5 know that they can still use version 5.9.1.

But just out of curiosity, can you paste the full output of pip install psutil? I'd like to check if pip is clever enough to download the tar.gz instead of the wheel in this case.

lmunch · 2022-11-09T14:24:59Z

Thanks for looking into this and I agree that it is probably not worth the effort. Maybe I should finally kill CentOS 5 from my pool of test servers :-)

I did some digging and speed_hi was introduced with Linux kernel 2.6.27 : https://elixir.bootlin.com/linux/v2.6.27/source/include/linux/ethtool.h
so perhaps it should instead state in the "platforms-support-history" that Linux kernel prior to kernel 2.6.27 is not supported.

The output from psutil install:

pip3 install psutil==5.9.4
Collecting psutil==5.9.4
  Using cached psutil-5.9.4.tar.gz (485 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: psutil
  Building wheel for psutil (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for psutil (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [64 lines of output]
      In file included from /tmp/tmp3hkt9_w7.c:1:
      /usr/include/linux/ethtool.h:18: error: expected specifier-qualifier-list before ‘__u32’
      /usr/include/linux/ethtool.h:35: error: expected specifier-qualifier-list before ‘__u32’
      /usr/include/linux/ethtool.h:52: error: expected specifier-qualifier-list before ‘__u32’
      /usr/include/linux/ethtool.h:60: error: expected specifier-qualifier-list before ‘__u32’
      /usr/include/linux/ethtool.h:66: error: expected specifier-qualifier-list before ‘__u32’
      /usr/include/linux/ethtool.h:74: error: expected specifier-qualifier-list before ‘__u32’
      /usr/include/linux/ethtool.h:83: error: expected specifier-qualifier-list before ‘__u32’
      /usr/include/linux/ethtool.h:179: error: expected specifier-qualifier-list before ‘__u32’
      /usr/include/linux/ethtool.h:201: error: expected specifier-qualifier-list before ‘__u32’
      /usr/include/linux/ethtool.h:226: error: expected specifier-qualifier-list before ‘__u32’
      /usr/include/linux/ethtool.h:251: error: expected specifier-qualifier-list before ‘__u32’
      /usr/include/linux/ethtool.h:260: error: expected specifier-qualifier-list before ‘__u32’
      /usr/include/linux/ethtool.h:266: error: expected specifier-qualifier-list before ‘__u32’
      /tmp/tmp3hkt9_w7.c:1:27: warning: no newline at end of file
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/psutil
      copying psutil/_psosx.py -> build/lib.linux-x86_64-cpython-311/psutil
      copying psutil/_psposix.py -> build/lib.linux-x86_64-cpython-311/psutil
      copying psutil/__init__.py -> build/lib.linux-x86_64-cpython-311/psutil
      copying psutil/_common.py -> build/lib.linux-x86_64-cpython-311/psutil
      copying psutil/_psaix.py -> build/lib.linux-x86_64-cpython-311/psutil
      copying psutil/_psbsd.py -> build/lib.linux-x86_64-cpython-311/psutil
      copying psutil/_pslinux.py -> build/lib.linux-x86_64-cpython-311/psutil
      copying psutil/_pssunos.py -> build/lib.linux-x86_64-cpython-311/psutil
      copying psutil/_compat.py -> build/lib.linux-x86_64-cpython-311/psutil
      copying psutil/_pswindows.py -> build/lib.linux-x86_64-cpython-311/psutil
      creating build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_contracts.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_memleaks.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/__init__.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_aix.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/__main__.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_unicode.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_process.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/runner.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_system.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_testutils.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_connections.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
      running build_ext
      building 'psutil._psutil_linux' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/psutil
      gcc-4.8 -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=594 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -DPSUTIL_ETHTOOL_MISSING_TYPES=1 -I/opt/python3/include/python3.11 -c psutil/_psutil_common.c -o build/temp.linux-x86_64-cpython-311/psutil/_psutil_common.o
      gcc-4.8 -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=594 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -DPSUTIL_ETHTOOL_MISSING_TYPES=1 -I/opt/python3/include/python3.11 -c psutil/_psutil_linux.c -o build/temp.linux-x86_64-cpython-311/psutil/_psutil_linux.o
      psutil/_psutil_linux.c: In function ‘psutil_ethtool_cmd_speed’:
      psutil/_psutil_linux.c:84:17: error: ‘const struct ethtool_cmd’ has no member named ‘speed_hi’
           return (ecmd->speed_hi << 16) | ecmd->speed;
                       ^
      psutil/_psutil_linux.c:85:1: warning: control reaches end of non-void function [-Wreturn-type]
       }
       ^
      error: command '/opt/gcc-4.8/bin/gcc-4.8' failed with exit code 1
      [end of output]

giampaolo · 2022-11-09T15:33:25Z

Mmm ok. pip is clever enough to pick up the tarball instead of the wheel. Also interesting that you discovered the exact kernel version where this was introduced. This simplifies things. I guess we can try using the pre-processor trick then, because the maintenance cost is not too high after all. Please try #2171.

lmunch · 2022-11-10T09:32:32Z

Works like a charm. Thanks a lot.

giampaolo · 2022-11-10T09:42:40Z

You're welcome. Out of curiosity, can you run python3 -m psutil.tests? Are there failures?

lmunch · 2022-11-10T10:22:57Z

Not quite sure what is going on here:

[root@fe25b4199d3f psutil]# fbpython3 -m psutil.tests
/usr/bin/fbpython3: Error while finding module specification for 'psutil.tests' (ImportError: cannot import name '_psutil_linux' from partially initialized module 'psutil' (most likely due to a circular import) (/psutil/psutil/__init__.py))

giampaolo · 2022-11-10T10:36:11Z

make test? (you need to GIT clone the source code though)

lmunch · 2022-11-10T11:09:38Z

Ok, here we go. This is running in a CentOS 5 container:

======================================================================
ERROR: psutil.tests.test_linux.TestProcess.test_status_file_parsing
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/psutil/psutil/tests/test_linux.py", line 2178, in test_status_file_parsing
    self.assertEqual(p._proc._get_eligible_cpus(), list(range(0, 8)))
AttributeError: 'Process' object has no attribute '_get_eligible_cpus'

======================================================================
ERROR: psutil.tests.test_linux.TestProcessAgainstStatus.test_cpu_affinity
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/psutil/psutil/tests/test_linux.py", line 2258, in test_cpu_affinity
    self.proc.cpu_affinity(), list(range(min_, max_ + 1)))
AttributeError: 'Process' object has no attribute 'cpu_affinity'

======================================================================
ERROR: psutil.tests.test_linux.TestProcessAgainstStatus.test_cpu_affinity_eligible_cpus
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/psutil/psutil/tests/test_linux.py", line 2263, in test_cpu_affinity_eligible_cpus
    self.proc._proc._get_eligible_cpus()
AttributeError: 'Process' object has no attribute '_get_eligible_cpus'

======================================================================
FAIL: psutil.tests.test_contracts.TestAvailConstantsAPIs.test_rlimit
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/psutil/psutil/tests/test_contracts.py", line 99, in test_rlimit
    ae(hasattr(psutil, "RLIM_INFINITY"), LINUX or FREEBSD)
AssertionError: False != True

======================================================================
FAIL: psutil.tests.test_contracts.TestAvailProcessAPIs.test_cpu_affinity
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/psutil/psutil/tests/test_contracts.py", line 188, in test_cpu_affinity
    self.assertEqual(hasattr(psutil.Process, "cpu_affinity"),
AssertionError: False != True

======================================================================
FAIL: psutil.tests.test_contracts.TestAvailProcessAPIs.test_rlimit
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/psutil/psutil/tests/test_contracts.py", line 175, in test_rlimit
    self.assertEqual(hasattr(psutil.Process, "rlimit"), LINUX or FREEBSD)
AssertionError: False != True

======================================================================
FAIL: psutil.tests.test_system.TestMiscAPIs.test_users
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/psutil/psutil/tests/test_system.py", line 206, in test_users
    self.assertNotEqual(users, [])
AssertionError: [] == []

======================================================================
FAIL: psutil.tests.test_misc.TestScripts.test_who
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/psutil/psutil/tests/test_misc.py", line 781, in test_who
    self.assert_stdout('who.py')
  File "/psutil/psutil/tests/test_misc.py", line 734, in assert_stdout
    assert out, out
AssertionError

======================================================================
FAIL: psutil.tests.test_testutils.TestMemLeakClass.test_leak_mem
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/psutil/psutil/tests/__init__.py", line 723, in wrapper
    raise exc
  File "/psutil/psutil/tests/__init__.py", line 715, in wrapper
    return fun(*args, **kwargs)
  File "/psutil/psutil/tests/test_testutils.py", line 379, in test_leak_mem
    self.assertRaisesRegex(AssertionError, "extra-mem",
AssertionError: AssertionError not raised by execute

======================================================================
FAIL: psutil.tests.test_linux.TestRootFsDeviceFinder.test_comparisons
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/psutil/psutil/tests/test_linux.py", line 1331, in test_comparisons
    self.assertIsNotNone(finder.find())
AssertionError: unexpectedly None

======================================================================
FAIL: psutil.tests.test_linux.TestRootFsDeviceFinder.test_disk_partitions_mocked
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/psutil/psutil/tests/test_linux.py", line 1363, in test_disk_partitions_mocked
    self.assertNotEqual(part.device, "/dev/root")
AssertionError: '/dev/root' == '/dev/root'

----------------------------------------------------------------------
Ran 637 tests in 3.613s

FAILED (failures=8, errors=3, skipped=163)
FAILED
make: *** [test] Error 1

giampaolo · 2022-11-10T11:45:58Z

Not bad. Most (all) of these are test adjustments which would need fixing on centos 5, but they don't mean the underlying functionality is broken. CentOS 5 appears fully supported.

* 'master' of https://github.com/giampaolo/psutil: fix: long-description on Windows (giampaolo#2168) GH actions: update actions' deps to latest version use argparse module in 3 scripts/internal scripts setup.py: provide better err msg if can't compile giampaolo#2164: fix compilation failures on linux < 2.6.27 / CentOS 5 (giampaolo#2171)

lmunch added the bug label Oct 25, 2022

github-actions bot added the linux label Oct 25, 2022

giampaolo added a commit that referenced this issue Nov 9, 2022

#2164: fix compilation failures on linux < 2.6.27 / CentOS 5

007d2dd

giampaolo mentioned this issue Nov 9, 2022

Fix compilation failure on Linux < 2.6.27 / CentOS 5 #2171

Merged

giampaolo closed this as completed in #2171 Nov 10, 2022

giampaolo added a commit that referenced this issue Nov 10, 2022

#2164: fix compilation failures on linux < 2.6.27 / CentOS 5 (#2171)

1f1074f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Linux] Build issue in 5.9.2/5.9.3. No speed_hi in struct ethtool_cmd #2164

[Linux] Build issue in 5.9.2/5.9.3. No speed_hi in struct ethtool_cmd #2164

lmunch commented Oct 25, 2022 •

edited by giampaolo

Loading

lmunch commented Oct 25, 2022

giampaolo commented Nov 9, 2022 •

edited

Loading

lmunch commented Nov 9, 2022

giampaolo commented Nov 9, 2022

lmunch commented Nov 10, 2022

giampaolo commented Nov 10, 2022 •

edited

Loading

lmunch commented Nov 10, 2022

giampaolo commented Nov 10, 2022

lmunch commented Nov 10, 2022

giampaolo commented Nov 10, 2022

[Linux] Build issue in 5.9.2/5.9.3. No speed_hi in struct ethtool_cmd #2164

[Linux] Build issue in 5.9.2/5.9.3. No speed_hi in struct ethtool_cmd #2164

Comments

lmunch commented Oct 25, 2022 • edited by giampaolo Loading

Summary

Description

lmunch commented Oct 25, 2022

giampaolo commented Nov 9, 2022 • edited Loading

lmunch commented Nov 9, 2022

giampaolo commented Nov 9, 2022

lmunch commented Nov 10, 2022

giampaolo commented Nov 10, 2022 • edited Loading

lmunch commented Nov 10, 2022

giampaolo commented Nov 10, 2022

lmunch commented Nov 10, 2022

giampaolo commented Nov 10, 2022

lmunch commented Oct 25, 2022 •

edited by giampaolo

Loading

giampaolo commented Nov 9, 2022 •

edited

Loading

giampaolo commented Nov 10, 2022 •

edited

Loading