len(platform.uname()) has changed in Python 3.9

[tucked]

BPO	40570
Nosy	@malemburg, @jaraco, @tucked
PRs	bpo-40570: Count the processor property in len(platform.uname()) #20009 bpo-40570: Improve compatibility of uname_result with late-bound .platform #20015

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-05-09.14:14:14.194>
created_at = <Date 2020-05-08.20:02:20.834>
labels = ['type-bug', 'library', '3.9']
title = 'len(platform.uname()) has changed in Python 3.9'
updated_at = <Date 2020-05-13.07:47:25.587>
user = 'https://github.com/tucked'

bugs.python.org fields:

activity = <Date 2020-05-13.07:47:25.587>
actor = 'lemburg'
assignee = 'none'
closed = True
closed_date = <Date 2020-05-09.14:14:14.194>
closer = 'jaraco'
components = ['Library (Lib)']
creation = <Date 2020-05-08.20:02:20.834>
creator = 'tucked'
dependencies = []
files = []
hgrepos = []
issue_num = 40570
keywords = ['3.9regression']
message_count = 11.0
messages = ['368459', '368486', '368487', '368491', '368502', '368518', '368519', '368520', '368528', '368534', '368764']
nosy_count = 3.0
nosy_names = ['lemburg', 'jaraco', 'tucked']
pr_nums = ['20009', '20015']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40570'
versions = ['Python 3.9']

[tucked]

518835f#diff-47c8e5750258a08a6dd9de3e9c3774acL741-R804

That diff changed len(platform.uname()) to 5 (from 6).

I noticed because we have some code that checks for 6 strs (arguably unnecessary),
but I can also think of contrived examples that would break (e.g. platform.uname()[-3]).
Interestingly, platform.uname()[5] still works.

Was this effect intentional?

[jaraco]

It was intentional to address bpo-35967, although it was meant to remain compatible.

Is len(uname()) an important behavior to retain? It feels like an implementation detail to me.

[jaraco]

If it is important to retain the len, it's probably also important to retain the [-N] accesses and possibly other behaviors of a length 6 tuple.

[jaraco]

Thanks David for the report and the draft PR (which helped me validate my thinking on the matter). In PR 20015, I've included additional tests. I've also re-written the compatibility functions to rely on the main __iter__ override.

Another situation that's likely to be incompatible is pickleability. Is that important?

I'm tempted to make deprecated the use of uname_result for anything other than attribute access (maybe with the ability to cast to a tuple, i.e. tuple(res)). The problem with namedtuples is that although they provide backward-compatibility for legacy behavior which returned tuples, they also bring that legacy as debt. By deprecating "access by index", that would constrain the interface to two basic usages: attribute access and iter() of all values.

[malemburg]

Hi Jason,

to achieve better backwards compatibility, it's probably better to use
the approach taken for CodeInfo in the codecs.py module:

class CodecInfo(tuple):
    """Codec details when looking up the codec registry"""

    def __new__(cls, encode, decode, streamreader=None, streamwriter=None,
        incrementalencoder=None, incrementaldecoder=None, name=None,
        *, _is_text_encoding=None):
        self = tuple.__new__(cls, (encode, decode, streamreader,
streamwriter))
        self.name = name
        self.encode = encode
        self.decode = decode
        self.incrementalencoder = incrementalencoder
        self.incrementaldecoder = incrementaldecoder
        self.streamwriter = streamwriter
        self.streamreader = streamreader
        if _is_text_encoding is not None:
            self._is_text_encoding = _is_text_encoding
        return self

    def __repr__(self):
        return "<%s.%s object for encoding %s at %#x>" % \
                (self.__class__.__module__, self.__class__.__qualname__,
                 self.name, id(self))

This used to be a 4 entry tuple and was extended to hold additional
fields. To the outside, it still looks like a 4-tuple in all aspects,
but attribute access permits accessing the additional fields.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, May 09 2020)
>>> Python Projects, Coaching and Support ...    https://www.egenix.com/
>>> Python Product Development ...        https://consulting.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
https://www.egenix.com/company/contact/
https://www.malemburg.com/

[jaraco]

Thanks Marc-Andre for the suggestion, but I don't think that approach is viable here. The whole point of bpo-35967 was to defer the execution of the .processor behavior until it was requested. The intention is not to extend the tuple, but to shorten it (at least not to require the last element to be provided at construction time). Perhaps I'm missing something here, so feel free to provide a more concrete example of what you have in mind. Just be cautious not to violate the intention (

cpython/Lib/platform.py

Lines 784 to 787 in 77c6146

	A uname_result that's largely compatible with a
	simple namedtuple except that 'platform' is
	resolved late and cached to avoid calling "uname"
	except when needed.

).

I created bpo-40578 to track deprecation of uname for positional access.

[jaraco]

New changeset 2c3d508 by Jason R. Coombs in branch 'master':
bpo-40570: Improve compatibility of uname_result with late-bound .platform (bpo-20015)
2c3d508

[jaraco]

I've gone ahead and merged PR 20015 to fix the issue, but I'm happy to revisit if a better approach is proposed.

[malemburg]

Ok, let me add some more context: When I wrote the uname interface
I was aware that calling the API will take some resources. That's
why I added the cache. IMO, that was enough as optimization.

Now, you added a late binding optimization for the whole uname return
tuple to save the effort of going out to the system and figure our
the value using separate APIs or even shell access.

I think this would have been better implemented in the various
uname() consumers
(

cpython/Lib/platform.py

Line 898 in 77c6146

def system():

and below), using a variant of the uname() API, say _uname(),
which leaves out the processor information for those APIs which don't
need it and only provide the late binding in processor() (which could
then also fill in the cache value for uname().

The uname() API would then still do the full lookup, but applications
could then use the specialized API to query only the information
they need.

I don't think that deprecating standard tuple access is an option
for the uname() return value, since it's documented to be a tuple.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, May 09 2020)
>>> Python Projects, Coaching and Support ...    https://www.egenix.com/
>>> Python Product Development ...        https://consulting.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
https://www.egenix.com/company/contact/
https://www.malemburg.com/

[jaraco]

you added a late binding optimization for the whole uname return
tuple to save the effort of ... shell access.

It wasn't to save the effort and it wasn't an optimization. There was a fundamental race condition where it became impossible to implement a uname command using Python because platform.system() (or anything else that delegated to platform.uname()) would unconditionally shell out to uname, and this would happen early, before a library could intercept the behavior. See bpo-35967 for more details on the motivation and challenges.

I think this would have been better implemented...

This idea is interesting, but I believe it also falls prey to the same race condition if any code calls platform.uname() before a Python-based uname implementation has a chance to monkeypatch the behavior.

May I suggest that you either revive the conversation in bpo-35967 or open a new ticket to discuss improving the implementation so that the details aren't lost in this ticket, which was specifically about compatibility issues?

I don't think that deprecating standard tuple access is an option
for the uname() return value, since it's documented to be a tuple.

I'll address this concern in the ticket I opened about the deprecation.

[malemburg]

Hi Jason,

I think I have to review the whole set of changes again to understand what your motivation is/was.

For https://bugs.python.org/issue35967 I had already stated that your use case is not special enough to make the platform.py logic more complex.

BTW: Please don't open several different tickets for the same problem. It doesn't really help to see what is going on. I'll reopen the bpo-35967 to continue the discussion there.

Thanks.

[dennisvang]

@malemburg Not sure if this should be a new issue, but platform.uname docs for 3.12 still say:

... Returns a namedtuple() containing six attributes: system, node, release, version, machine, and processor.

(my emphasis)

I've got python 3.10.6 installed, which returns 5 attributes, not including processor, as expected from the discussion above:

uname_result(
    system='Linux', 
    node='MyName', 
    release='5.15.0-48-generic', 
    version='#54-Ubuntu SMP Fri Aug 26 13:26:29 UTC 2022', 
    machine='x86_64'
)

[jaraco]

Moved to a new issue.