added safety check to python dependencies

[heartsucker]

Status

Ready for review

Description of Changes

Fixes #2451

Adds safety check to a few scripts and Makefiles as well as CI.

Testing

make safety

This breaks right now as we have a CVE in the code. Once it's removed, we can merge this in.

[ghost]

I confirm circleci fails as expected https://circleci.com/gh/freedomofpress/securedrop/2960?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

-> ansible, installed 2.2.1, affected <2.3.1, id 34941
ansible before 2.3.1 is vulnerable to CVE-2017-7481 - data for lookup plugins used as variables was not being correctly marked as "unsafe".
--

[ghost]

I marked as PR: pending additional work because it needs to wait on the CVE to be dealt with. I did not know about this way to find CVE, interesting !

[ghost]

One minor cleanup on in the requirement.txt update. Otherwise looks good to me. And very useful too :-)

[ghost]

In the spirit of keeping this PR to the minimum, would you mind stripping out the module upgrades that are unrelated to the safety module ?

[heartsucker]

I could, but I was using pip-compile and I don't actually know if there's any weird breakages, so I'm letting that tool do it's magic. This is something one of the ops homies would need to comment on.

[ghost]

FWIW when I did the same in the past I just manually removed all dependencies that were not strictly related to the module I was adding. I don't think there is a way to ask pip-compile to focus on a single module and leave the others alone.

[msheiny]

Yeah @dachary is correct here ;) Unfortunately the pip-compile story sucks for just updating one dependency. You could temporarily peg all the dependencies in the requirements.in file and then run it ... that's a hacky way around it. Or yeah, do what @dachary said and try to manually edit the commit changes using something like git add -p

[redshiftzero]

Once #2483 is merged in you can rebase on this and tests should pass ✨

[heartsucker]

Pinging for merge now that CI his happy.

[ghost]

CircleCI is happy

#!/bin/bash -eo pipefail
make safety

Checking file ./install_files/ansible-base/requirements.txt
safety report
checked 15 packages, using default DB
---
No known security vulnerabilities found.


Checking file ./testinfra/requirements.txt
safety report
checked 72 packages, using default DB
---
No known security vulnerabilities found.


Checking file ./securedrop/requirements/develop-requirements.txt
safety report
checked 61 packages, using default DB
---
No known security vulnerabilities found.


Checking file ./securedrop/requirements/test-requirements.txt
safety report
checked 28 packages, using default DB
---
No known security vulnerabilities found.


Checking file ./securedrop/requirements/securedrop-requirements.txt
safety report
checked 25 packages, using default DB
---
No known security vulnerabilities found.

[ghost]

Very good !