Submit new R package to CRAN

[terrytangyuan]

@hetong007 @tqchen @pommedeterresautee @khotilov

I received some requests recently. Could anyone submit the latest version to CRAN (major changes are the callbacks I believe)?

[hetong007]

I was a bit off recently. I will run the CRAN checks, and submit the current version soon.
Related issue: #1423

[terrytangyuan]

Sorry I missed that issue. Thanks @hetong007.

[tqchen]

We will need a VM to get solaris checks pass, @thirdwing can you help on that?

[thirdwing]

I will try to find some time.

[thirdwing]

I just try a 32-bit Solaris VM. However, I can't install all the dependencies on that.

[hetong007]

@thirdwing Do you have a list of those dependencies?

[thirdwing]

NMF can't be install on x86 VM. See https://cran.r-project.org/web/checks/check_results_NMF.html

We don't depend on this directly. This is a dependency for DiagrameR

[thirdwing]

BTW, I can install xgboost on x86 Solaris.

[tqchen]

@thirdwing can you confirm the R check as-cran pass?

[thirdwing]

I can just install it. It seems impossible to do the check on x86 Solaris machine.

[khotilov]

The igraph depends on NMF , the DiagrammeR depends on igraph; and we call some functions from both DiagrammeR and igraph in some non-essential plotting methods. There are a few unit tests which call those - is that why the check for solaris doesn't work? Could it make sense to disable those particular tests just for the CRAN release?

[thirdwing]

@khotilov Failure on x86 Solaris is somewhat acceptable by CRAN, since NMF hasn't been removed. I will try to get access to a real Solaris machine.

[ck37]

+1 to CRAN release, would be very helpful since the current version is from July 12.

[terrytangyuan]

Any updates on this? Have you sorted out the issue with raw test mismatch?

[hetong007]

As mentioned by @yixuan in #1826 (comment), this error can be observed when R compiles with -O1 or -O2, and vanishes when using -O0.

I am considering about commenting out the related tests when submitting to CRAN.

[terrytangyuan]

Yeah I think we can submit it for now by commenting those out.

…

On Tue, Dec 13, 2016 at 10:56 PM Tong He ***@***.***> wrote: As mentioned by @yixuan <https://github.com/yixuan> in #1826 (comment) <#1826 (comment)>, this error can be observed when R compiles with -O1 or -O2, and vanishes when using -O0. I am considering about commenting out the related tests when submitting to CRAN. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1812 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEEnSuPdkI1vGNyGEyLew3W5aieksvenks5rH3cLgaJpZM4K8HBx> .

[hetong007]

I will do that in this weekend.

[thirdwing]

xgboost failed on solaris: https://www.r-project.org/nosvn/R.check/r-patched-solaris-x86/xgboost-00install.html

This is because the openmp support on CRAN machine.

Two solution: (1) disabled openmp; (2) add a separate autoconf script.

[hetong007]

The first solution would make the package much weaker on CRAN, thus not really benefits the CRAN users.
@thirdwing can you help with the second solution?

[thirdwing]

see my repo thirdwing@e1af62c

It should work on Solaris and Linux. Not sure about Windows.

[tqchen]

i think it is fine to disable OpenMP on solaris because solaris is not used widely anyway. It is interesting that C++11 compiler in solaris is g++ instead of Solaris studio. We do need to make sure the checks pass in other OS

[thirdwing]

If you have CXX_STD = CXX11, the solaris will also use g++.

Actually I think it will be more complicated to disable openmp on only one OS.

[tqchen]

What I intended to say is that this is the problem of current CRAN's solaris build system. see also my comment in #1880 By default, it should be the build system's job to switch off openmp by not passing anything in SHLIB_OPENMP_CFLAGS .

This happens in OSX automatically. Solaris have openmp support with solaris studio, but need to be able to disable it when c++11 is turned on. This is the problem of current build script, that affect every package with c++11 and openmp turned on

[yixuan]

The problem is due to the wrong flag to enable OpenMP. For Solaris Studio the flag for OpenMP is -xopenmp, but for GCC it is -fopenmp. From the CRAN check log it can be seen that -xopenmp was passed to GCC.

[thirdwing]

That is why we need a configure script

…

On Dec 16, 2016 4:46 PM, "Yixuan Qiu" ***@***.***> wrote: The problem is due to the wrong flag to enable OpenMP. For Solaris Studio the flag for OpenMP is -xopenmp, but for GCC it is -fopenmp. From the CRAN check log it can be seen that -xopenmp was passed to GCC. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1812 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABebVQF57CG7_wdxjyoQ-vwSQCHiMZ7Jks5rIwaegaJpZM4K8HBx> .

[yixuan]

Yep, and here is what I found: #1881 .

[hetong007]

Currently xgboost 0.6-2 is on. The version number "2" is the need for the post-submission bug fixing.

However after the fixes from @yixuan and @thirdwing , installation on Solaris is still broken: https://cran.r-project.org/web/checks/check_results_xgboost.html.

Another issue is that DiagrammeR 0.9 will be on CRAN. It is also left to be tested to see if this update breaks xgboost.

[tqchen]

@thirdwing Can you grab a VM to look into this?

The Sparc compilation error is due to lack of support of thread local in Sparc, which can be solved by optionally disable thread local macro here https://github.com/dmlc/xgboost/blob/master/src/common/thread_local.h#L22 to check solaris and simply define MX_TREAD_LOCAL to be empty

[tqchen]

The segfault of x86 solaris is more mysterious, and we might need a reproducible example to see what exactly happened.

[thirdwing]

I only have the x86 VM. I will have a try if possible.

[thirdwing]

@tqchen I can't reproduce the error on the X86 solaris VM.

[thirdwing]

From the CRAN checking log, the error on X86 Solaris is from https://github.com/dmlc/xgboost/blob/master/R-package/src/xgboost_R.cc#L88

It involves the usage of openmp, so maybe we can try to disable openmp on solaris.

[tqchen]

Let us do that

[tqchen]

Let us also fix the thread local issue by patching the fix i mentioned

[tqchen]

@thirdwing can you followup on this? I also noticed there is an error in windows, reporting conflicting defintition of fopen, do you know what happened there?

[thirdwing]

I am working on my own fork. The make on Solaris doesn't support if ,which makes things much harder.

[hetong007]

@thirdwing just opened a new PR (merged) for solaris: #1912 .

For the error on old release windows, the error message is

d:\compiler\gcc-4.6.3\bin\../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/include/stdio.h:396:83: error: 'FILE* std::fopen(const char*, const char*)' should have been declared inside 'std'
d:\compiler\gcc-4.6.3\bin\../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/include/stdio.h:396:83: error: declaration of 'FILE* std::fopen(const char*, const char*)'
d:\compiler\gcc-4.6.3\bin\../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/include/stdio.h:395:17: error: conflicts with previous declaration 'FILE* fopen(const char*, const char*)'

I found several related previous issues:

• Compiling issues on Windows with Git Bash #1174 (comment)
• Error installing xgboost #1267

Hope someone can shed light on it.

[thirdwing]

@hetong007 I suggest adding Depends: R (>= 3.3.0), so we do not need to worry about old windows release.

[terrytangyuan]

There have been some updates and typo fixes to docs. Could you also regenerate the docs before resubmitting the package?

…

On Fri, Dec 30, 2016 at 11:24 PM Qiang Kou (KK) ***@***.***> wrote: @hetong007 <https://github.com/hetong007> I suggest adding Depends: R (>= 3.3.0), so we do not need to worry about old windows release. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1812 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEEnSoOWt6Ibojz7_HfUsLX5ZJ9QEFwhks5rNSI8gaJpZM4K8HBx> .

[hetong007]

There are some plot-related errors on R 3.3.2 when checked by win-builder: https://win-builder.r-project.org/Oc7UhRHG4Y2w/00check.log (link will expire after roughly 72 hours).

However the check on R-devel is fine: https://win-builder.r-project.org/qG0ASTr2Yd45/00check.log

The errors, e.g. Error: 'graph_from_data_frame' is not an exported object from 'namespace:igraph', are not reproducible on Linux. I will compile and check it on a windows machine later today to see if this is actually caused by win-builder.

[thirdwing]

Please also update this https://github.com/dmlc/xgboost/blob/master/R-package/vignettes/xgboost.Rnw#L24

[thirdwing]

You should use something like xgboost.version <- packageDescription("xgboost")$Version

[thirdwing]

We are getting closer. Now errors only come from Old Windows release and Sparc Solaris: https://cran.r-project.org/web/checks/check_results_xgboost.html

My opinions are as follow:

(1) We don't need to care much about old Windows release;

(2) Sparc Solaris failed in https://github.com/dmlc/xgboost/blob/master/R-package/tests/testthat/test_helpers.R#L86

The simplest solution is to disable this test on Solaris by detecting Sys.info()[['sysname']].