[R] CRAN packaging checklist for 14.0.0

[paleolimbot]

Describe the enhancement requested

cc @assignUser @thisisnic

Before the release candidate is cut

• Create a GitHub issue entitled [R] CRAN packaging checklist for version X.X.X and copy this checklist to the issue.
• Evaluate the status of any failing nightly tests and nightly packaging builds. These checks replicate most of the checks that CRAN runs, so we need them all to be passing or to understand that the failures may (though won't necessarily) result in a rejection from CRAN.
• Check current CRAN check results
• Ensure the contents of the README are accurate and up to date.
• Run urlchecker::url_check() on the R directory at the release candidate.
  commit. Ignore any errors with badges as they will be removed in the CRAN release branch.
• Polish NEWS but do not update version numbers (this is done automatically later). You can find commits by, for example, git log --oneline aa057d0..HEAD | grep "\[R\]"
• Run preliminary reverse dependency checks using archery docker run r-revdepcheck.
• For major releases, prepare tweet thread highlighting new features.

Wait for the release candidate to be cut:

After release candidate has been cut

• Create a CRAN-release branch from the release candidate commit

Prepare and check the .tar.gz that will be released to CRAN.

• git fetch upstream && git checkout release-X.X.X-rcXX && git clean -f -d
• run Rscript tools/update-checksums.R X.X.X to download shasums for the precompiled binaries
• Run make build. This copies Arrow C++ into tools/cpp, prunes some
  unnecessary components, and runs R CMD build to generate the source tarball.
  Because this will install the package, you will need to ensure that the version
  of Arrow C++ available to the configure script is the same as the version
  that is vendored into the R package (e.g., you may need to unset ARROW_HOME).
• devtools::check_built("arrow_X.X.X.tar.gz") locally
• Run reverse dependency checks using archery docker run r-revdepcheck.

Release vote

• Release vote passed!

Generate R package to submit to CRAN

• If the release candidate commit updated, rebase the CRAN release branch
  on that commit.
• Pick any commits that were made to main since the release commit that
  were needed to fix CRAN-related submission issues identified in the above
  steps.
• Remove badges from README.md
• Run urlchecker::url_check() on the R directory
• Create a PR entitled WIP: [R] Verify CRAN release-10.0.1-rc0. Add
  a comment @github-actions crossbow submit --group r to run all R crossbow
  jobs against the CRAN-specific release branch.
• Regenerate arrow_X.X.X.tar.gz (i.e., make build)

Ensure linux and macos binary packages are available:

• Ensure linux binaries are available in the artifactory:
  https://apache.jfrog.io/ui/repos/tree/General/arrow/r

Check binary Arrow C++ distributions specific to the R package

• Upload the .tar.gz to win-builder (r-devel only)
  and confirm (with Nic, who will automatically receive an email about the results) that the check is clean.
• Upload the .tar.gz to MacBuilder
  and confirm that the check is clean
• Check install.packages("arrow_X.X.X.tar.gz") on Ubuntu and ensure that the
  hosted binaries are used
• devtools::check_built("arrow_X.X.X.tar.gz") locally one more time (for luck)

CRAN submission

• Upload arrow_X.X.X.tar.gz to the
  CRAN submit page
• Confirm the submission email

Wait for CRAN...

• Accepted!
• Tag the tip of the CRAN-specific release branch
• Add a new line to the matrix in the backwards compatability job
• (patch releases only) Update the package version in ci/scripts/PKGBUILD, dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb, r/DESCRIPTION, and r/NEWS.md
• (CRAN-only releases) Rebuild news page with pkgdown::build_news() and submit a PR to the asf-site branch of the docs site with the contents of arrow/r/docs/news/index.html replacing the current contents of arrow-site/docs/r/news/index.html
• (CRAN-only releases) Bump the version number in r/pkgdown/assets/versions.json, and update this on the the asf-site branch of the docs site too.
• Update the packaging checklist template to reflect any new realities of the
  packaging process.
• Wait for CRAN-hosted binaries on the
  CRAN package page to reflect the
  new version
• Tweet!

Component(s)

R

[thisisnic]

Thanks for kicking this off @paleolimbot! I'm happy to prep the social media stuff a bit later on.

[assignUser]

We can remove all of the autobrew related stuff right? As well as the rtools-packages PRs as we are never looking for a local version of the win library unless the envvar is set...

[thisisnic]

Yeah, we should create a new ticket to update these instructions later, but I figured we might want to get 14.0.0 released first so we're aware of any other changes we need to make, and update it all in one PR.

I've opened #38196

[paleolimbot]

I think all the autobrew stuff is already gone (I just opened this issue before that merged)

[paleolimbot]

We've fixed our nightlies and our reverse dependency checks are clear! We've picked up 9 new reverse dependencies since 13.0.0 (for a total of 49!).

[assignUser]

I added the checksum script (though this is only necessary now (see #38205 ). The advanced refactor remving winlibs is 2a4e0db#diff-ee05cb2be40729ac4ed9b086fa4b8493347bd079b3c8944fc84877c4ed413181

[thisisnic]

Macbuilder on release branch with URLs fixed: https://mac.r-project.org/macbuilder/results/1699847336-f93888c5bca3f09c/

[thisisnic]

Macbuilder with fixes for Winbuilder: https://mac.r-project.org/macbuilder/results/1699899373-13af7a74c51dd423/
Winbuilder with fixes for Winbuilder: https://win-builder.r-project.org/mm0LgxGZI90m/

[thisisnic]

devtools::check_built() resulted in a warning for:

❯ checking compilation flags used ... WARNING
  Compilation used the following non-portable flag(s):
    ‘-Wno-noexcept-type’ ‘-msse4.2’
  including flag(s) suppressing warnings

I could have sworn we only used to get 1 of these. Is this a blocker to submitting or just some relic of my local setup?

[assignUser]

That doesn't happen on the macbuilder so it should be ok (you probably have arrow_r_dev set?)

[thisisnic]

OK, great, submitted now!

[thisisnic]

Cherry-picked #38716 across

[assignUser]

e543ee6 is the fix for the 'Additional issues' error

[assignUser]

c6682f1 adds more timeout so the windows failure we had doesn't happen again when the artifactory is a little slow.

[thisisnic]

Once #38819 is merged, we should cherry-pick that across too

[assignUser]

Merged: 490cd83

@thisisnic that should be all for now, we will have to see how it goes...

[assignUser]

eb5de18 fixes an issue when (cross-)compiling arrow with strict linker options (but is not an R only change so ??)

[thisisnic]

@assignUser I've already submitted 14.0.0.1 now. We could include it in 14.0.0.2 but I suppose it's a question of whether we should.

[assignUser]

A RC for 14.0.2 will be cut this week so I am adding all of the cherrypicked issues (including the cmake change as there is no issue adding that to a new release 🎉 )

[thisisnic]

Thanks @assignUser!