Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow CallsiteParameterAdder to be pickled #603

Merged
merged 5 commits into from
Mar 14, 2024

Conversation

Lunderberg
Copy link
Contributor

@Lunderberg Lunderberg commented Mar 11, 2024

Summary

Prior to this commit, the CallsiteParameterAdder object could not be pickled. As a result, structlog configurations could not be propagated to subprocesses using pickle.dumps(structlog.get_config()). This commit updates the handlers in CallsiteParameterAdder._handlers to be free functions, rather than lambda functions, so that all class members can be pickled.

Closes #600

Pull Request Check List

  • Do not open pull requests from your main branch – use a separate branch!
    • There's a ton of footguns waiting if you don't heed this warning. You can still go back to your project, create a branch from your main branch, push it, and open the pull request from the new branch.
    • This is not a pre-requisite for your your pull request to be accepted, but you have been warned.
  • Added tests for changed code.
    • The CI fails with less than 100% coverage.
  • [n/a] New APIs are added to our typing tests in api.py.
  • [n/a] Updated documentation for changed code.
    • [n/a] New functions/classes have to be added to docs/api.rst by hand.
    • [n/a] Changed/added classes/methods/functions have appropriate versionadded, versionchanged, or deprecated directives.
      • The next version is the second number in the current release + 1. The first number represents the current year. So if the current version on PyPI is 23.1.0, the next version is gonna be 23.2.0. If the next version is the first in the new year, it'll be 24.1.0.
  • Documentation in .rst and .md files is written using semantic newlines.
  • Changes (and possible deprecations) are documented in the changelog.
  • Consider granting push permissions to the PR branch, so maintainers can fix minor issues themselves without pestering you.

Lunderberg and others added 2 commits March 11, 2024 18:15
Prior to this commit, the `CallsiteParameterAdder` object could not be
pickled.  As a result, structlog configurations could not be propagated
to subprocesses using `pickle.dumps(structlog.get_config())`.  This
commit updates the handlers in `CallsiteParameterAdder._handlers` to be
free functions, rather than lambda functions, so that all class members
can be pickled.

Closes hynek#600
Lunderberg added a commit to octoml/mlc-llm that referenced this pull request Mar 11, 2024
Required for compatibility with TVM PR
apache/tvm#16618.  Can be removed after
upstream `structlog` hynek/structlog#603
lands.

This is a backport of the OLLM PR
octoml/ollm#409, and is not needed after the
MLC-serve migration.
masahi pushed a commit to octoml/mlc-llm that referenced this pull request Mar 11, 2024
Required for compatibility with TVM PR
apache/tvm#16618.  Can be removed after
upstream `structlog` hynek/structlog#603
lands.

This is a backport of the OLLM PR
octoml/ollm#409, and is not needed after the
MLC-serve migration.
Copy link
Owner

@hynek hynek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make the function names verbs please? (iow: _pathname_get_pathname).

Changelog entry would be great too ("structlog.processors.CallsiteParameterAdder is now pickleable." + link here).

(oh and get the CI green too, pls 😇 – I think it's just missing type hints for the functions?)

@Lunderberg
Copy link
Contributor Author

Can you make the function names verbs please?

Can do. I've updated each function name from FOO to _get_callsite_FOO.

Changelog entry would be great too

And updated. (Made the commit for the changelog yesterday, but forgot to push it.)

(oh and get the CI green too, pls 😇 – I think it's just missing type hints for the functions?)

Thank you on pointing me in the right direction. Updated the signatures from (module, frame_info) to (module: str, frame_info: inspect.Traceback) to match the existing type annotation for CallsiteParameterAdder._handlers.

Copy link
Owner

@hynek hynek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks!

@hynek hynek enabled auto-merge (squash) March 14, 2024 05:35
@hynek hynek merged commit a62bbf6 into hynek:main Mar 14, 2024
16 checks passed
@Lunderberg Lunderberg deleted the picklable_callsite_parameter_adder branch March 14, 2024 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CallsiteParameterAdder not pickleable
2 participants