Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-107704: Argument Clinic: add support for deprecating keyword use of parameters #107984

Merged

Conversation

serhiy-storchaka
Copy link
Member

@serhiy-storchaka serhiy-storchaka commented Aug 15, 2023

It is now possible to deprecate passing keyword arguments for keyword-or-positional parameters with Argument Clinic, using the new '/ [from X.Y]' syntax.
(To be read as "positional-only from Python version X.Y")

… use of parameters

It is now possible to deprecate passing keyword arguments for
keyword-or-positional parameters with Argument Clinic, using the new
'/ [from X.Y]' syntax.
(To be read as "positional-only from Python version X.Y")
@erlend-aasland
Copy link
Contributor

Do you want an initial review of the draft, or would you like to wait until it is marked ready for review?

@serhiy-storchaka serhiy-storchaka marked this pull request as ready for review August 15, 2023 19:38
@serhiy-storchaka
Copy link
Member Author

I kept it in a draft state for the first run of tests and the final stage of self-review.

It is now ready for review.

Copy link
Contributor

@erlend-aasland erlend-aasland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be beneficial to try and reduce some of the code duplication, either in this PR, or in a following-up refactor. Perhaps the latter is preferable.

Doc/howto/clinic.rst Outdated Show resolved Hide resolved
Lib/test/test_clinic.py Show resolved Hide resolved
Lib/test/test_clinic.py Outdated Show resolved Hide resolved
Lib/test/test_clinic.py Outdated Show resolved Hide resolved
Lib/test/test_clinic.py Outdated Show resolved Hide resolved
Comment on lines 1001 to 1006
if func.kind.new_or_init:
conditions.append(f"nargs < {i+1} && kwargs && PyDict_Contains(kwargs, &_Py_ID({p.name}))")
containscheck = "PyDict_Contains"
else:
conditions.append(f"nargs < {i+1} && kwnames && PySequence_Contains(kwnames, &_Py_ID({p.name}))")
containscheck = "PySequence_Contains"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately it is the simplest general way to check that the argument was passed by keyword. It is not the fastest. For inlined parsing code we can use more efficient but more complex code, we can even build it in _PyArg_UnpackKeywords(). But if PyArg_ParseTupleAndKeywords() or similar function is used, the only other way is to use deprecated and inefficient PyMapping_HasKeyString() or introduce a new private inefficient API.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the concern with using _Py_ID() here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, there is no concern; I just wanted to point out that even though Argument Clinic does not currently care about the Limited API, we may want to add such functionality in the future; IMO, it should be possible for Argument Clinic to generate code that does not depend on internal APIs, for example for stdlib extension modules such as sqlite3.

Many core devs seem to be in favour of not using internal APIs in stdlib extension modules, but I'm not sure if there is a consensus.

Tools/clinic/clinic.py Outdated Show resolved Hide resolved
Tools/clinic/clinic.py Outdated Show resolved Hide resolved
Tools/clinic/clinic.py Outdated Show resolved Hide resolved
Tools/clinic/clinic.py Outdated Show resolved Hide resolved
Copy link
Member Author

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your review @erlend-aasland. I tried to address them. Seems the only unresolved issue is with _Py_ID().

Doc/howto/clinic.rst Outdated Show resolved Hide resolved
Lib/test/test_clinic.py Show resolved Hide resolved
Lib/test/test_clinic.py Outdated Show resolved Hide resolved
Lib/test/test_clinic.py Outdated Show resolved Hide resolved
Lib/test/test_clinic.py Outdated Show resolved Hide resolved
Tools/clinic/clinic.py Outdated Show resolved Hide resolved
Tools/clinic/clinic.py Outdated Show resolved Hide resolved
Tools/clinic/clinic.py Outdated Show resolved Hide resolved
Lib/test/test_clinic.py Show resolved Hide resolved
Tools/clinic/clinic.py Outdated Show resolved Hide resolved
Copy link
Contributor

@erlend-aasland erlend-aasland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this, @serhiy-storchaka. @AlexWaygood, do you want to have a look?

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some small suggestions for the docs:

Doc/howto/clinic.rst Outdated Show resolved Hide resolved
Doc/howto/clinic.rst Outdated Show resolved Hide resolved
Doc/howto/clinic.rst Outdated Show resolved Hide resolved
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good, just a few very minor nits:

Tools/clinic/clinic.py Show resolved Hide resolved
Tools/clinic/clinic.py Outdated Show resolved Hide resolved
Tools/clinic/clinic.py Outdated Show resolved Hide resolved
Copy link
Member Author

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your review @AlexWaygood. Do these docstrings work?

Tools/clinic/clinic.py Show resolved Hide resolved
Tools/clinic/clinic.py Outdated Show resolved Hide resolved
Tools/clinic/clinic.py Outdated Show resolved Hide resolved
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @serhiy-storchaka!

@serhiy-storchaka
Copy link
Member Author

Post-review changes:

  • I have found how to make keyword argument check more efficient in case of inlined parsing code. It now has almost zero overhead.
  • Since the tests with the new generated code no longer covered all cases in the generator, I added new tests, and they exposed bugs in a corner case in constructors (including * [from ...]). Fixed.

@serhiy-storchaka serhiy-storchaka merged commit 2f31143 into python:main Aug 19, 2023
@serhiy-storchaka serhiy-storchaka deleted the clinic-deprecated-keyword branch August 19, 2023 07:13
AA-Turner pushed a commit to AA-Turner/devguide that referenced this pull request Sep 26, 2023
…eyword use of parameters (python/cpython#107984)

It is now possible to deprecate passing keyword arguments for
keyword-or-positional parameters with Argument Clinic, using the new
'/ [from X.Y]' syntax.
(To be read as "positional-only from Python version X.Y")

Co-authored-by: Erlend E. Aasland <[email protected]>
Co-authored-by: Alex Waygood <[email protected]>
erlend-aasland added a commit to python/devguide that referenced this pull request Oct 1, 2023
…eyword use of parameters (python/cpython#107984)

It is now possible to deprecate passing keyword arguments for
keyword-or-positional parameters with Argument Clinic, using the new
'/ [from X.Y]' syntax.
(To be read as "positional-only from Python version X.Y")

Co-authored-by: Erlend E. Aasland <[email protected]>
Co-authored-by: Alex Waygood <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants