Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hide creds when saving and use netrc file #73

Merged
merged 5 commits into from
Apr 4, 2017

Conversation

drebs
Copy link
Contributor

@drebs drebs commented Mar 31, 2017

This is related to #72.

When using --benchmark-storage with elasticsearch feature pytest-benchmark prints the whole URL when saving the data after tests finished. If the URL contains credentials and if the results of the run are public, then the credentials are leaked. The first commit in this pull request addresses this issue by masking eventual credentials before printing the URL.

Note that this does not protect other leaks such as when using tox together with pytest, as tox will print the whole pytest command line before running it and leak credentials anyway. Our current workaround for this is to use CI environment variables to set the content of a pytest.ini file with the relevant options for pytest. Those options are not printed by tox because they are loaded by pytest itself.

The second commit of this pull request implements the use of the ~/.netrc file by adding the --benchmark-netrc option. Further work can be done to add a --benchmark-netrc-file (using the same semantics as curl) to point to a specific file other than the default one in user's home directory.

@ionelmc
Copy link
Owner

ionelmc commented Mar 31, 2017

Shouldn't netrc be used by default? Are there any other apps in which you have to explicitly enable netrc support?

@ionelmc
Copy link
Owner

ionelmc commented Mar 31, 2017

Hmmm, seems curl does it explicitly: https://ec.haxx.se/usingcurl-netrc.html

Perhaps we could allow optional path? Eg: nargs="?", default=[], const=True, (just like --benchmark-compare)

Otherwise very well, please update authors/changelog.rst.

@codecov-io
Copy link

codecov-io commented Apr 3, 2017

Codecov Report

Merging #73 into master will decrease coverage by 1.89%.
The diff coverage is 53.33%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master      #73     +/-   ##
=========================================
- Coverage   77.44%   75.54%   -1.9%     
=========================================
  Files          16       16             
  Lines        1667     1693     +26     
  Branches      300      289     -11     
=========================================
- Hits         1291     1279     -12     
- Misses        315      348     +33     
- Partials       61       66      +5
Impacted Files Coverage Δ
src/pytest_benchmark/session.py 91.71% <ø> (ø) ⬆️
src/pytest_benchmark/cli.py 34.95% <0%> (ø) ⬆️
src/pytest_benchmark/plugin.py 86.95% <100%> (+0.07%) ⬆️
src/pytest_benchmark/storage/elasticsearch.py 26.73% <40%> (-8.69%) ⬇️
src/pytest_benchmark/utils.py 77.54% <56.52%> (-3.36%) ⬇️
src/pytest_benchmark/compat.py 40.74% <0%> (-14.82%) ⬇️
src/pytest_benchmark/logger.py 78.43% <0%> (-3.93%) ⬇️
src/pytest_benchmark/stats.py 90.54% <0%> (-1.36%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b996cf4...f9d1c5b. Read the comment docs.

@drebs
Copy link
Contributor Author

drebs commented Apr 3, 2017

I implemented the optional path it a bit differently than you proposed, because it seemed to make more sense for the implementation. The default value if the --benchmark-netrc option is not given will be ''. If the option is given with no argument, the const value will be '~/.netrc' and if an argument is passed then the value is the argument. Does it make sense?

Should I do something about the test coverage?

@ionelmc
Copy link
Owner

ionelmc commented Apr 3, 2017

Perhaps a 3 tests, one asserting the output has no visible creds and one with --benchmark-netrc and one with --benchmark-netrc=something ?

@drebs drebs force-pushed the use-netrc-file branch 2 times, most recently from a2eb9a0 to f9d1c5b Compare April 4, 2017 06:50
@drebs
Copy link
Contributor Author

drebs commented Apr 4, 2017

I added tests for creds handling in different scenarios (no creds, creds in url, creds in netrc, creds in url superseding creds in netrc), and one test for host masking. Do you think simple tests like these are enough or have you thought about more complex setups?

@drebs drebs mentioned this pull request Apr 4, 2017
@ionelmc
Copy link
Owner

ionelmc commented Apr 4, 2017

Alright, thanks!

@ionelmc ionelmc merged commit bbb9406 into ionelmc:master Apr 4, 2017
leap-code-o-matic pushed a commit to leapcode/soledad that referenced this pull request Apr 20, 2017
There were some changes needed in pytest-benchmark so we could
successfully use it for soledad benchmarks graphing:

ionelmc/pytest-benchmark#73
ionelmc/pytest-benchmark#74
ionelmc/pytest-benchmark#75

The contributions were accepted but not released yet, so this commit
uses the code from upstream git repository's master branch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants