[Android] PR #2096 in release 5.9.2 cause to exception

[STUkh]

Summary

• OS: Android 12 Termux
• Architecture: aarch64
• Psutil version: 5.9.2
• Python version: 3.10.7
• Type: core

Description

Using home-assistant native on Android - cause module "Hardware" to fail.
Fallback to 5.9.1 make it work flawlessly.
Debug logs trace back to psutil and this function call:
d4eea1f#diff-300fc786f9e87417b39f07b9b5d0940fac8ef6098047cf8f46e9d0678edb2b6dR444

Looks like there is environments and kernels where there is no ethtool_cmd_speed function available and it's call cause exception to be thrown.

I'm not an expert in this codebase, but assume we may try/catch that call and fallback to previous variant. Thanks in advance for devs who pick up this issue. Ping me if you need any other information

[giampaolo]

Using home-assistant native on Android - cause module "Hardware" to fail.

What's the error? Do you have anything (error, traceback...) to paste?

[giampaolo]

CC @garrisoncarter

[STUkh]

I lost a log when fallback to 5.9.1 but will re-update and share full log with you.

The main point that this particular function was the last in stacktrace and fallback to previous version fix error - where this call doesn't exists. But anyway will reproduce today again for you

[STUkh]

@giampaolo as I promised - here is the log:

Traceback (most recent call last):
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/homeassistant/setup.py", line 235, in _async_setup_component
    result = await task
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/homeassistant/components/hardware/__init__.py", line 15, in async_setup
    await websocket_api.async_setup(hass)
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/homeassistant/components/hardware/websocket_api.py", line 36, in async_setup
    ha_psutil=await hass.async_add_executor_job(ha_psutil.PsutilWrapper),
  File "/data/data/com.termux/files/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/psutil_home_assistant/__init__.py", line 15, in __init__
    psutil_spec.loader.exec_module(psutil_module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/psutil/__init__.py", line 42, in <module>
    from . import _common
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/psutil/__init__.py", line 102, in <module>
    from . import _pslinux as _psplatform
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/psutil/_pslinux.py", line 26, in <module>
    from . import _psutil_linux as cext
ImportError: dlopen failed: cannot locate symbol "ethtool_cmd_speed" referenced by "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/psutil/_psutil_linux.cpython-310.so"...

[indrastorms]

@giampaolo as I promised - here is the log:

Traceback (most recent call last):
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/homeassistant/setup.py", line 235, in _async_setup_component
    result = await task
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/homeassistant/components/hardware/__init__.py", line 15, in async_setup
    await websocket_api.async_setup(hass)
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/homeassistant/components/hardware/websocket_api.py", line 36, in async_setup
    ha_psutil=await hass.async_add_executor_job(ha_psutil.PsutilWrapper),
  File "/data/data/com.termux/files/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/psutil_home_assistant/__init__.py", line 15, in __init__
    psutil_spec.loader.exec_module(psutil_module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/psutil/__init__.py", line 42, in <module>
    from . import _common
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/psutil/__init__.py", line 102, in <module>
    from . import _pslinux as _psplatform
  File "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/psutil/_pslinux.py", line 26, in <module>
    from . import _psutil_linux as cext
ImportError: dlopen failed: cannot locate symbol "ethtool_cmd_speed" referenced by "/data/data/com.termux/files/home/hass/lib/python3.10/site-packages/psutil/_psutil_linux.cpython-310.so"...

Same error while using jupyterlab

[aaronjamt]

Same issue trying to use https://github.com/nvbn/thefuck (looks like this isn't just one program, it's something to do with anything that uses 'psutil')

  File "/data/data/com.termux/files/usr/bin/fuck", line 5, in <module>
    from thefuck.entrypoints.not_configured import main
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/thefuck/entrypoints/not_configured.py", line 12, in <module>
    from psutil import Process  # noqa: E402
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/__init__.py", line 102, in <module>
    from . import _pslinux as _psplatform
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_pslinux.py", line 26, in <module>
    from . import _psutil_linux as cext
ImportError: dlopen failed: cannot locate symbol "ethtool_cmd_speed" referenced by "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_psutil_linux.cpython-310.so"...

Edit: looks like it has something to do with this line (only place I found "ethtool_cmd_speed" in this project):

psutil/psutil/_psutil_linux.c

Line 444 in d4eea1f

uint_speed = ethtool_cmd_speed(&ethcmd);

Edit 2: Searching for that function on all of Github leads to this patch file in the termux/termux-packages repo: https://github.com/termux/termux-packages/blob/8f09a5769a17fbbbf532ac29c29f4952b7a08edf/packages/samba/0005-samba-documents-provider-sambapatch.patch#L399

[giampaolo]

Fixed it in b88f4d8. Please confirm it patch works.

[timblaktu]

@giampaolo that commit fixes it for me on Termux (Android), using this line in requirements file:

psutil @ git+https://github.com/giampaolo/psutil.git@b88f4d8

...however the package doesn't really work as it does on other platforms (I know this is expected) and I'm now hitting issues similar to #1334:

~ $ python3 -c "import psutil; psutil.cpupercent(percpu=True)"
Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_common.py", line 399, in wrapper
    return cache[key]
KeyError: (('/proc',), frozenset())

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_pslinux.py", line 285, in <module>
    set_scputimes_ntuple("/proc")
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_common.py", line 401, in wrapper
    ret = cache[key] = fun(*args, **kwargs)
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_pslinux.py", line 268, in set_scputimes_ntuple
    with open_binary('%s/stat' % procfs_path) as f:
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_common.py", line 728, in open_binary
    return open(fname, "rb", buffering=FILE_READ_BUFFER_SIZE)
PermissionError: [Errno 13] Permission denied: '/proc/stat'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: module 'psutil' has no attribute 'cpupercent'. Did you mean: 'cpu_percent'?
~ $ python3 -c "import psutil; psutil.cpu_percent(percpu=True)"
Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_common.py", line 399, in wrapper
    return cache[key]
KeyError: (('/proc',), frozenset())

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_pslinux.py", line 285, in <module>
    set_scputimes_ntuple("/proc")
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_common.py", line 401, in wrapper
    ret = cache[key] = fun(*args, **kwargs)
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_pslinux.py", line 268, in set_scputimes_ntuple
    with open_binary('%s/stat' % procfs_path) as f:
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_common.py", line 728, in open_binary
    return open(fname, "rb", buffering=FILE_READ_BUFFER_SIZE)
PermissionError: [Errno 13] Permission denied: '/proc/stat'
Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_common.py", line 399, in wrapper
    return cache[key]
KeyError: (('/proc',), frozenset())

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/__init__.py", line 1766, in cpu_percent
    tot1 = cpu_times(percpu=True)
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/__init__.py", line 1615, in cpu_times
    return _psplatform.per_cpu_times()
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_pslinux.py", line 598, in per_cpu_times
    set_scputimes_ntuple(procfs_path)
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_common.py", line 401, in wrapper
    ret = cache[key] = fun(*args, **kwargs)
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_pslinux.py", line 268, in set_scputimes_ntuple
    with open_binary('%s/stat' % procfs_path) as f:
  File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/psutil/_common.py", line 728, in open_binary
    return open(fname, "rb", buffering=FILE_READ_BUFFER_SIZE)
PermissionError: [Errno 13] Permission denied: '/proc/stat'
~