Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix issue #66 aws-s3 paging/next-marker is not correct #67

Merged
merged 2 commits into from
Aug 25, 2018

Conversation

peczenyj
Copy link
Contributor

I am not sure if I can provide a unit test for this case, my problem is iterate over a huge s3 bucket without manually handle the next marker

I am not sure if I can provide a unit test for this case, my problem is iterate over a huge s3 bucket without manually handle the next marker
@araddon
Copy link
Contributor

araddon commented Aug 23, 2018

I am Re-running travis tests now. And, let me poke around and see if there is a way to get the unit tests to fail first.

Thank you very much for the patch.

fix typo
Copy link
Contributor Author

@peczenyj peczenyj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I type fast and made a mistake, fixed now :)

@codecov-io
Copy link

Codecov Report

Merging #67 into master will decrease coverage by 44.43%.
The diff coverage is 0%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master      #67       +/-   ##
===========================================
- Coverage   64.89%   20.46%   -44.44%     
===========================================
  Files          13       13               
  Lines        1997     1999        +2     
===========================================
- Hits         1296      409      -887     
- Misses        491     1524     +1033     
+ Partials      210       66      -144
Impacted Files Coverage Δ
awss3/store.go 0.29% <0%> (-72.92%) ⬇️
azure/store.go 0.3% <0%> (-69.21%) ⬇️
google/store.go 0.66% <0%> (-61.88%) ⬇️
sftp/store.go 13.62% <0%> (-48.04%) ⬇️
google/client.go 0% <0%> (-31.43%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3646b62...72506c7. Read the comment docs.

@peczenyj
Copy link
Contributor Author

Hello

I did not understand this result from codecov, since it shows a decrease in some files that I don't change at all.

running tests on master branch

tpeczenyj@CPL-tpeczenyj:~/work/go/src/github.com/lytics/cloudstorage (master)$ bash go.test.sh 
ok  	github.com/lytics/cloudstorage	1.822s	coverage: 72.4% of statements
ok  	github.com/lytics/cloudstorage/awss3	1.014s	coverage: 0.4% of statements
ok  	github.com/lytics/cloudstorage/azure	1.011s	coverage: 0.4% of statements
?   	github.com/lytics/cloudstorage/azure/example	[no test files]
?   	github.com/lytics/cloudstorage/csbufio	[no test files]
ok  	github.com/lytics/cloudstorage/google	1.015s	coverage: 0.3% of statements
ok  	github.com/lytics/cloudstorage/google/storeutils	1.013s	coverage: 0.0% of statements
ok  	github.com/lytics/cloudstorage/localfs	1.773s	coverage: 81.0% of statements
ok  	github.com/lytics/cloudstorage/sftp	1.285s	coverage: 8.8% of statements
?   	github.com/lytics/cloudstorage/testutils	[no test files]

running on my change

tpeczenyj@CPL-tpeczenyj:~/work/go/src/github.com/lytics/cloudstorage ((6ccd349...))$ bash go.test.sh 
ok  	github.com/lytics/cloudstorage	1.803s	coverage: 73.4% of statements
ok  	github.com/lytics/cloudstorage/awss3	1.014s	coverage: 0.4% of statements
ok  	github.com/lytics/cloudstorage/azure	1.012s	coverage: 0.4% of statements
?   	github.com/lytics/cloudstorage/azure/example	[no test files]
?   	github.com/lytics/cloudstorage/csbufio	[no test files]
ok  	github.com/lytics/cloudstorage/google	1.015s	coverage: 0.3% of statements
ok  	github.com/lytics/cloudstorage/google/storeutils	1.014s	coverage: 0.0% of statements
ok  	github.com/lytics/cloudstorage/localfs	1.777s	coverage: 81.0% of statements
ok  	github.com/lytics/cloudstorage/sftp	1.270s	coverage: 8.8% of statements
?   	github.com/lytics/cloudstorage/testutils	[no test files]

¯_(ツ)_/¯

@araddon araddon changed the title fix issue #66 fix issue #66 aws-s3 paging/next-marker is not correct Aug 25, 2018
@araddon araddon changed the base branch from master to 66_s3_nextmarker August 25, 2018 20:54
@araddon araddon merged commit e0f80de into lytics:66_s3_nextmarker Aug 25, 2018
@araddon
Copy link
Contributor

araddon commented Aug 25, 2018

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants