Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[R-package][ci] Added more R linters (fixes #2477) #2533

Merged
merged 22 commits into from
Nov 16, 2019

Conversation

jameslamb
Copy link
Collaborator

lintr released 2.0.0 to CRAN (and r-lintr to conda-forge).

In this PR, I propose adding additional linters on the R code in the project, to enforce particular style. See #2477 for background.

Start with the diff to .ci/lint_r_code.R to see the new linters being added.

One change you will see a LOT of is something like replacing i + 1 with i +1L to satisfy the implicit_integer_linter. This one caught a lot of cases where we were relying on R sometimes treating 1 as double precision and sometimes as integer. This is basically always fine in R, but it can mean you're using more memory than necessary. So think of the addition of the L as a tiny tiny memory savor and, more importantly, documentation for people reading the code.

.ci/test.sh Outdated Show resolved Hide resolved
@jameslamb
Copy link
Collaborator Author

Just rebased to most recent master. I think this is ready for review

Copy link
Collaborator

@StrikerRUS StrikerRUS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Below are some comments about a CI part.

.ci/lint_r_code.R Outdated Show resolved Hide resolved
.ci/test.sh Outdated Show resolved Hide resolved
.ci/test.sh Outdated Show resolved Hide resolved
.ci/test.sh Outdated Show resolved Hide resolved
.ci/test.sh Outdated
pydocstyle
conda install -q -y -n $CONDA_ENV -c conda-forge \
r-lintr==2.0.0
Rscript -e "install.packages('stringi', repos = 'http://cran.rstudio.com')"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It takes soo much time! Is it the only available fix? Maybe it will be faster to update lintr from CRAN?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try setting up an R environment like I'm doing in #2530 so that we can just use install.packages() directly. Let's see if that speeds it up.

But in general the install process for R packages is notoriously time-consuming :/

Copy link
Collaborator

@StrikerRUS StrikerRUS Nov 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh no! I think we should stick to conda's environment, since it's significantly faster than ordinary one and suits to our purpose (just linting) very well.
Sorry for my ambiguous comment above! I just meant to try Rscript -e "install.packages('lintr', repos = 'http://cran.rstudio.com')" to see whether it's faster than reinstalling stringi or not.

But in general the install process for R packages is notoriously time-consuming :/

But not from conda 😉

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah I see, ok. Yes I think I misinterpreted what you were saying. I'll change it back and try.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like linting task took almost 7 minutes to run still with this approach (using alternative R setup)

image

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

creating an R environment from conda and then install.packages('lintr') from CRAN, the lint task ran for 8m38s and then failed with

ERROR: dependency ‘curl’ is not available for package ‘httr’

I could add an install.packages('curl') but that would make it run even longer than 8 minutes.

I decided to summarize all the attempts here (and add a few new ideas):

  1. (-failure-) R env from conda, lintr from conda-forge
  2. (success) R env from conda, lintr from conda-forge, install.packages('stringi'): 6m9s
  3. (success) R env from CRAN like [R-package][ci] added CI stage for R package (fixes #2335, fixes #2569) #2530, install.packages('lintr'): 6m48s
  4. (-failure-) R env from conda, install.packages('lintr'): 8+ minutes
  5. (-failure-) R env from conda, lintr from conda-forge, re-installing stringi from default conda channels: failed
  6. (success) R env from conda, installing stringi from conda first, then lintr from conda-forge: 1m57s

So I think we should go with installing stringi explicitly, THEN lintr! This works and runs in under 2 minutes 🎉🎉 🎉

.ci/test.sh Outdated Show resolved Hide resolved
Copy link
Collaborator

@StrikerRUS StrikerRUS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jameslamb Thanks a lot for making it work! Please see my comments below.

.ci/test.sh Outdated Show resolved Hide resolved
.ci/test.sh Outdated Show resolved Hide resolved
.ci/test.sh Outdated Show resolved Hide resolved
.ci/test.sh Outdated Show resolved Hide resolved
.ci/test.sh Outdated Show resolved Hide resolved
.ci/test.sh Outdated Show resolved Hide resolved
@jameslamb
Copy link
Collaborator Author

ping @Laurae2 for review on changes to the R code

Copy link
Collaborator

@StrikerRUS StrikerRUS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need progress bars for downloading packages, which only litter logs. Moreover, sometimes progress bar causes errors during the installation process 😮 .

.ci/test.sh Outdated Show resolved Hide resolved
.ci/test.sh Outdated Show resolved Hide resolved
R-package/R/lgb.Dataset.R Outdated Show resolved Hide resolved
.ci/test.sh Outdated Show resolved Hide resolved
Copy link
Collaborator

@StrikerRUS StrikerRUS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for CI scripting part! Thank you very much!

@jameslamb
Copy link
Collaborator Author

just rebased this to most recent master

@StrikerRUS StrikerRUS merged commit 1f7b06b into microsoft:master Nov 16, 2019
@jameslamb jameslamb deleted the ci/more_r_linting branch January 27, 2020 00:14
@lock lock bot locked as resolved and limited conversation to collaborators Mar 27, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants