Flake8 Fixes (Part1)

[laserprec]

Description

This a step in the a sequence of updates to our DevOps infrastructures:

1. Propose and draft the CI pipeline in Github Action
2. Setup self-hosted machines to run our GPU workloads
3. Create feature parity with the existing CI pipelines (pr-gates & nightly-build)
4. Run tests on the appropriate dependency subsets
5. Optimize build time for unit tests
6. Enforce flake8 (coding style checks) on the build and clean up coding styles to pass flake8 checks <---- (Current PR)
7. Deprecate CI pipelines in ADO and switch to Github Actions

Existing flake8 Issues

In this PR, we addressed the following flake8 issues

Simple style issues (deletion/addition)

• W291 trailing whitespace **
• W292 no newline at end of file
• W391 blank line at end of file **
• W293 blank line contains whitespace **
• E303 too many blank lines (2)
• E262 inline comment should start with '# '
• E266 too many leading '#' for block comment
• E265 block comment should start with '# '
• E501 line too long
• E302 expected 2 blank lines, found 1
• E128 continuation line under-indented for visual indent
• E126 continuation line over-indented for hanging
• E226 missing whitespace around arithmetic operator
• E401 multiple imports on one line
• F541 f-string is missing placeholders
• F811 redefinition of unused 'pd' from line 4
• F841 local variable 'a' is assigned to but never used**
• F401 'X' imported but unused **

For reviewers: rules marked with (**) have significant number of violations and may need more attention there.

Below are the flake8 rules we can ignore permanently

• W503 line break before binary operator
  • ex:

           print(
                   "at epoch {0:d}".format(epoch)
                   + "\ntrain info: "
                   + train_info
                   + "\neval info: "
                   + eval_info
                   + "\ntest info: "
                   + test_info
               #___^
               )

• E203 whitespace before ':'
  • ex:

     lambda x: x[:, hparams.title_size + hparams.body_size + 1 :]
                                                         #____^

If there's any other flake8 rules we wish to undo and ignore permanently, we can do that easily too 😃 .

NOTE: folders including tests, docs, contrib, examples are excluded from flake8 checks. For more details please see tox.ini

Todo in a subsequent PR:

Coding Convention Issues

• E712 comparison to True should be 'if cond is True:' or 'if cond:'
• E711 comparison to None should be 'if cond is None:'
• E722 do not use bare 'except'
• E741 ambiguous variable name 'l'
• F403 'from X import *' used; unable to detect undefined names
• F405 'Dense' may be undefined, or defined from star imports: tensorflow.keras.layers

Critical issues (can be indication of potential bugs)

• W605 invalid escape sequence
• F821 undefined name 'K'

Checklist:

• I have followed the contribution guidelines and code style for this project.
• I have added tests covering my contributions.
• I have updated the documentation accordingly.
• This PR is being made to staging branch and not to main branch.

[codecov-commenter]

Codecov Report

Merging #1550 (6f485c3) into staging (ce81bdd) will decrease coverage by 0.20%.
The diff coverage is 58.13%.

@@             Coverage Diff             @@
##           staging    #1550      +/-   ##
===========================================
- Coverage    62.07%   61.86%   -0.21%     
===========================================
  Files           84       84              
  Lines         8492     8436      -56     
===========================================
- Hits          5271     5219      -52     
+ Misses        3221     3217       -4     

Flag	Coverage Δ
pr-gate	61.86% <58.13%> (-0.21%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
recommenders/datasets/cosmos_cli.py	0.00% <ø> (ø)
recommenders/datasets/covid_utils.py	90.00% <ø> (-0.25%)	⬇️
recommenders/datasets/criteo.py	86.79% <ø> (ø)
recommenders/datasets/download_utils.py	90.00% <ø> (-0.20%)	⬇️
recommenders/datasets/movielens.py	68.44% <ø> (-0.14%)	⬇️
recommenders/datasets/sparse.py	90.47% <ø> (ø)
recommenders/datasets/split_utils.py	87.75% <ø> (-0.25%)	⬇️
recommenders/evaluation/python_evaluation.py	93.68% <ø> (ø)
recommenders/evaluation/spark_evaluation.py	86.60% <ø> (ø)
...ecommenders/models/deeprec/DataModel/ImplicitCF.py	80.68% <ø> (-0.22%)	⬇️
... and 40 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d867e87...6f485c3. Read the comment docs.

[miguelgfierro]

This is great. @laserprec, were all these changes automatic or did you do them manually?

[anargyri]

Have you checked that these docstrings appear as before after compilation in the docs/html ?

[laserprec]

I have but let me confirm again.

[laserprec]

After newlines added (<TEST_TEST_TEST> is added to differentiate):

Current version

Confirmed. Newlines shouldn't affect the sphinx documentation.

[laserprec]

This is great. @laserprec, were all these changes automatic or did you do them manually?

Thanks @miguelgfierro! Some of the fixes are done using vscode shortcut and black, but it is orchestrated manually 😝.