Documentation for tool `tox` #11165

Honny1 · 2023-10-02T12:59:43Z

Description:

This PR creates documentation for the use of tox. And disables the automatic execution of flake8 when using tox.

github-actions · 2023-10-02T13:01:23Z

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment

Oracle Linux 8 Environment

jan-cerny

This is a good idea and I love the various examples.

jan-cerny · 2023-10-02T18:30:56Z

docs/manual/developer/08_content_tests.md

@@ -15,9 +15,44 @@ To add a script to be checked with mypy use the `mypy_test` macro in `tests/CMak

 ### Unit Tests

-`ctest` is used to run the unit tests for the `ssg` python package that is in the repo.
+`ctest` is used to run the unit tests for the `ssg` python package that is in the repo. But you need to build content and `ctest` isn't able to run unit tests with differed versions of python. To run only unit tests with differed version of python for the `ssg` python package is recommended to use `tox`.


This is against the Style Guide https://github.com/ComplianceAsCode/content/blob/master/docs/manual/developer/04_style_guide.md#markup-languages which says that we shall have one sentence per line.

jan-cerny · 2023-10-02T18:31:10Z

docs/manual/developer/08_content_tests.md

@@ -15,9 +15,44 @@ To add a script to be checked with mypy use the `mypy_test` macro in `tests/CMak

 ### Unit Tests

-`ctest` is used to run the unit tests for the `ssg` python package that is in the repo.
+`ctest` is used to run the unit tests for the `ssg` python package that is in the repo. But you need to build content and `ctest` isn't able to run unit tests with differed versions of python. To run only unit tests with differed version of python for the `ssg` python package is recommended to use `tox`.


Python should be with capital P.

jan-cerny · 2023-10-02T18:31:24Z

docs/manual/developer/08_content_tests.md

@@ -15,9 +15,44 @@ To add a script to be checked with mypy use the `mypy_test` macro in `tests/CMak

 ### Unit Tests

-`ctest` is used to run the unit tests for the `ssg` python package that is in the repo.
+`ctest` is used to run the unit tests for the `ssg` python package that is in the repo. But you need to build content and `ctest` isn't able to run unit tests with differed versions of python. To run only unit tests with differed version of python for the `ssg` python package is recommended to use `tox`.


jan-cerny · 2023-10-02T18:32:34Z

docs/manual/developer/08_content_tests.md

@@ -15,9 +15,44 @@ To add a script to be checked with mypy use the `mypy_test` macro in `tests/CMak

 ### Unit Tests

-`ctest` is used to run the unit tests for the `ssg` python package that is in the repo.
+`ctest` is used to run the unit tests for the `ssg` python package that is in the repo. But you need to build content and `ctest` isn't able to run unit tests with differed versions of python. To run only unit tests with differed version of python for the `ssg` python package is recommended to use `tox`.


I don't understand what you mean by "But you need to build content and ..." . If you mean that you need to build SCAP content in order to run unit tests then that's not true.

If I understand documentation correctly? It means that the unit tests are part of the ctest test suite, and before running ctest you need to execute the ./build_product command to build the content. For tox, just clone the repo, install tox and run it.

But we're talking about Python unit tests of the ssg Python package. For them to run, you don't have to build anything, it's enough to just configure:

cd build rm -rf * cmake .. ctest -R python-unit-ssg-module

Okay, I'll add it to the documentation. This information was missing from the documentation.

jan-cerny · 2023-10-02T18:33:03Z

docs/manual/developer/08_content_tests.md

-`ctest` is used to run the unit tests for the `ssg` python package that is in the repo.
+`ctest` is used to run the unit tests for the `ssg` python package that is in the repo. But you need to build content and `ctest` isn't able to run unit tests with differed versions of python. To run only unit tests with differed version of python for the `ssg` python package is recommended to use `tox`.
+
+Note that `tox` creates a virtual environment that handles all dependencies defined in the test request file and performs unit tests with multiple versions of python. Thus, you can test your changes with different versions of python on your machine and don't have to wait for the upstream CI to check them for you.


Python should be with capital P.

jan-cerny · 2023-10-02T18:33:48Z

docs/manual/developer/08_content_tests.md

+
+*You must be in the root directory of the project before running the `tox` command.*
+
+Runs tests with python2, python3.8, python3.9 and python3-latest on machine:


Is the python2 Python 2.7 or which version?

jan-cerny · 2023-10-02T18:34:45Z

docs/manual/developer/08_content_tests.md

-`ctest` is used to run the unit tests for the `ssg` python package that is in the repo.
+`ctest` is used to run the unit tests for the `ssg` python package that is in the repo. But you need to build content and `ctest` isn't able to run unit tests with differed versions of python. To run only unit tests with differed version of python for the `ssg` python package is recommended to use `tox`.
+
+Note that `tox` creates a virtual environment that handles all dependencies defined in the test request file and performs unit tests with multiple versions of python. Thus, you can test your changes with different versions of python on your machine and don't have to wait for the upstream CI to check them for you.


I would omit the "Note that"

jan-cerny · 2023-10-03T12:53:24Z

/packit retest-failed

codeclimate · 2023-10-04T09:15:19Z

Code Climate has analyzed commit fca6810 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 56.9%.

View more on Code Climate.

jan-cerny · 2023-10-04T12:20:40Z

/packit retest-failed

jan-cerny

I like these changes very much! They're really detailed and precise.

The TF CS9 fail is tracked in #11179 and isn't caused by the changes in this PR.

jan-cerny · 2023-10-05T11:21:41Z

/packit retest-failed

jan-cerny · 2023-10-05T11:55:10Z

/packit retest-failed

jan-cerny · 2023-10-05T13:32:30Z

/packit retest-failed

Honny1 · 2023-10-06T08:01:58Z

@jan-cerny All tests were successful.

Honny1 requested a review from jan-cerny October 2, 2023 12:59

Honny1 added the Documentation Update in project documentation. label Oct 2, 2023

jan-cerny self-assigned this Oct 2, 2023

jan-cerny added this to the 0.1.71 milestone Oct 2, 2023

jan-cerny requested changes Oct 2, 2023

View reviewed changes

Honny1 force-pushed the doc-tox branch from b68bc71 to 6a34b2f Compare October 3, 2023 08:57

Honny1 requested a review from jan-cerny October 3, 2023 09:10

Honny1 force-pushed the doc-tox branch from 6a34b2f to e72d611 Compare October 3, 2023 15:12

Honny1 added 3 commits October 4, 2023 10:58

Disable flake8

98d596a

Add Documentation about tox unit tests

6a0abd2

Add how to run unit tests via ctest

fca6810

Honny1 force-pushed the doc-tox branch from e72d611 to fca6810 Compare October 4, 2023 08:58

jan-cerny approved these changes Oct 5, 2023

View reviewed changes

Honny1 requested a review from jan-cerny October 6, 2023 08:00

jan-cerny merged commit 4530b79 into ComplianceAsCode:master Oct 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Documentation for tool `tox` #11165

Documentation for tool `tox` #11165

Honny1 commented Oct 2, 2023

github-actions bot commented Oct 2, 2023

jan-cerny left a comment

jan-cerny Oct 2, 2023

jan-cerny Oct 2, 2023

jan-cerny Oct 2, 2023

jan-cerny Oct 2, 2023

Honny1 Oct 3, 2023

jan-cerny Oct 3, 2023

Honny1 Oct 3, 2023

jan-cerny Oct 2, 2023

jan-cerny Oct 2, 2023

jan-cerny Oct 2, 2023

jan-cerny commented Oct 3, 2023

codeclimate bot commented Oct 4, 2023

jan-cerny commented Oct 4, 2023

jan-cerny left a comment

jan-cerny commented Oct 5, 2023

jan-cerny commented Oct 5, 2023

jan-cerny commented Oct 5, 2023

Honny1 commented Oct 6, 2023


		You must be in the root directory of the project before running the `tox` command.

		Runs tests with python2, python3.8, python3.9 and python3-latest on machine:

Documentation for tool tox #11165

Documentation for tool tox #11165

Conversation

Honny1 commented Oct 2, 2023

Description:

github-actions bot commented Oct 2, 2023

jan-cerny left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jan-cerny commented Oct 3, 2023

codeclimate bot commented Oct 4, 2023

jan-cerny commented Oct 4, 2023

jan-cerny left a comment

Choose a reason for hiding this comment

jan-cerny commented Oct 5, 2023

jan-cerny commented Oct 5, 2023

jan-cerny commented Oct 5, 2023

Honny1 commented Oct 6, 2023

Documentation for tool `tox` #11165

Documentation for tool `tox` #11165