Moto server #35655
Moto server #35655
Conversation
|
@martindurant the old pyarrow seems to be coming from defaults. Perhaps only conda-forge has new enough packages for us. So maybe make sure that all the pyarrows have |
|
Interestingly, travis-37-locale doesn't have pyarrow listed at all (but the newest version on defaults is 0.16.0) |
|
Thanks! |
| # OK is bucket already exists | ||
| pass | ||
| timeout = 2 | ||
| while not cli.list_buckets()["Buckets"] and timeout > 0: |
There was a problem hiding this comment.
Do you think bumping this timeout will solve the issue in below (not sure why it would have > 2 secs though..):
| s3 = s3fs.S3FileSystem(client_kwargs={"endpoint_url": "http://127.0.0.1:5555/"}) | ||
|
|
||
| try: | ||
| s3.rm(bucket, recursive=True) |
There was a problem hiding this comment.
Is there a possibility this runs into race conditions with tests running in parallel (Running pytest xdist).
e.g one test function attempting to create the bucket whilst another deletes it?
There was a problem hiding this comment.
I do not recommend running tests in parallel for this, indeed - they access the same port and same bucket.
If you wanted to make parallel tests, but keep the one server process (which seems to be required by windows), you would need to randomise the bucket being accessed, and propagate this to all of the tests in the fixture output.
Changes moto for s3 tests from monkeypatched/mocking to server mode. This allows aiobotocore exceptions to raise correctly, needed for tests to pass; the change is required by the upcoming async release of s3fs, but also works for old sync (botocore) version. A release of s3fs without this change would break pandas tests.
Note: since I now need to pass storage_options to the various s3 calls in the tests to make them pass (giving the endpoint of the moto s3 server), I needed to also plumb storage_options through the excel IO, which was previously missing, and update the excel tests.
black pandasgit diff upstream/master -u -- "*.py" | flake8 --diff