Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use fix multiprocessing bug on macOS where qsize() is not implemented #1374

Merged
merged 9 commits into from
Jul 12, 2023

Conversation

Donnype
Copy link
Contributor

@Donnype Donnype commented Jul 11, 2023

Changes

This should fix the MacOSX issues due to qsize() not working (as documented in the library.. For the last 15 years..)

Proof

Tested on my Mac, we could consider adding a matrix in github but this is probably overkill.


Code Checklist

  • All the commits in this PR are properly PGP-signed and verified;
  • This PR only contains functionality relevant to the issue; tickets have been created for newly discovered issues.
  • I have written unit tests for the changes or fixes I made.
  • For any non-trivial functionality, I have added integration and/or end-to-end tests.
  • I have performed a self-review of my code and refactored it to the best of my abilities.

Communication

  • I have informed others of any required .env changes files if required and changed the .env-dist accordingly.
  • I have made corresponding changes to the documentation, if necessary.

Checklist for code reviewers:

Copy-paste the checklist from the docs/source/templates folder into your comment.


Checklist for QA:

Copy-paste the checklist from the docs/source/templates folder into your comment.

@Donnype Donnype requested a review from a team as a code owner July 11, 2023 15:30
praseodym
praseodym previously approved these changes Jul 11, 2023
Copy link
Contributor

@praseodym praseodym left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

self.handling_tasks = mp.Manager().dict()
manager = mp.Manager()

self.task_queue = manager.Queue() # multiprocessing.Queue() will not work on OSX, see mp.Queue.qsize()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to macOS since then :)

ammar92
ammar92 previously approved these changes Jul 11, 2023
@Donnype Donnype dismissed stale reviews from ammar92 and praseodym via 6a38433 July 11, 2023 18:04
@Donnype Donnype changed the title Fix/scheduler client factor pickle issues Use fix multiprocessing bug on macOS where qsize() is not implemented Jul 11, 2023
@underdarknl
Copy link
Contributor

@noamblitz could you see if the problem persists?

Copy link
Contributor

@noamblitz noamblitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works, nice fix.

@underdarknl underdarknl merged commit 0cc8910 into main Jul 12, 2023
@underdarknl underdarknl deleted the fix/scheduler-client-factor-pickle-issues branch July 12, 2023 12:58
jpbruinsslot added a commit that referenced this pull request Jul 24, 2023
* main:
  Fix robot test (#1420)
  Use the correct clearance level variable in organization member list template (#1427)
  Fix translation in Debian package (#1432)
  Reschedule tasks when no results in bytes are found after grace period (#1410)
  Don't scan hostname nmap in nmap boefje (#1415)
  Add and use our own CVE API (#1383)
  Add `task_id` as a query parameter to the `GET /origins` endpoint (#1414)
  Remove member group checks and check for permission instead (#1275)
  Bump cryptography from 41.0.0 to 41.0.2 in /boefjes/boefjes/plugins/kat_ssl_certificates (#1396)
  Bump cryptography from 41.0.1 to 41.0.2 in /bytes (#1397)
  Build the Debian build image on the main branch (#1387)
  Add explicit `black` config to all modules (#1395)
  Fix <no title> in the user guide docs (#1391)
  Add configurable octpoes request timeout (#1382)
  Remove hardcoded clearance level in member list for superusers (#1390)
  Add Debian build depends for CVE API package (#1384)
  Add buttons to manual rerun tasks, both boefjes or normalizers (#1339)
  Use fix multiprocessing bug on macOS where `qsize()` is not implemented (#1374)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants