Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-1325: [R] Initial R package that builds against the arrow C++ library #2489

Closed
wants to merge 6 commits into from

Conversation

romainfrancois
Copy link
Contributor

This is very minimal in functionality, it just gives a simple R package that calls a function from the arrow C++ library.

@wesm wesm changed the title Initial R package that builds against the arrow C++ library ARROW-1325: [R] Initial R package that builds against the arrow C++ library Aug 28, 2018
@codecov-io
Copy link

codecov-io commented Aug 28, 2018

Codecov Report

Merging #2489 into master will decrease coverage by 0.11%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2489      +/-   ##
==========================================
- Coverage   86.33%   86.22%   -0.12%     
==========================================
  Files         308      308              
  Lines       47120    47120              
==========================================
- Hits        40682    40629      -53     
- Misses       6366     6417      +51     
- Partials       72       74       +2
Impacted Files Coverage Δ
go/arrow/math/int64_avx2_amd64.go 0% <0%> (-100%) ⬇️
go/arrow/math/float64_avx2_amd64.go 0% <0%> (-100%) ⬇️
go/arrow/memory/memory_avx2_amd64.go 0% <0%> (-100%) ⬇️
go/arrow/math/uint64_avx2_amd64.go 0% <0%> (-100%) ⬇️
go/arrow/math/uint64_amd64.go 33.33% <0%> (-33.34%) ⬇️
go/arrow/math/int64_amd64.go 33.33% <0%> (-33.34%) ⬇️
go/arrow/math/float64_amd64.go 33.33% <0%> (-33.34%) ⬇️
go/arrow/math/math_amd64.go 31.57% <0%> (-31.58%) ⬇️
go/arrow/memory/memory_amd64.go 28.57% <0%> (-28.58%) ⬇️
rust/src/array.rs 84.03% <0%> (ø) ⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 21ef18b...9e3ffb4. Read the comment docs.

Copy link
Member

@wesm wesm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. Let's try to merge this ASAP for the sake of development productivity

@@ -31,6 +31,7 @@ export ARROW_JS_DIR=${TRAVIS_BUILD_DIR}/js
export ARROW_INTEGRATION_DIR=$TRAVIS_BUILD_DIR/integration
export ARROW_RUBY_DIR=$TRAVIS_BUILD_DIR/ruby
export ARROW_RUST_DIR=${TRAVIS_BUILD_DIR}/rust
export ARROW_R_DIR=${TRAVIS_BUILD_DIR}/r
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any strong opinions about "R/" vs. "r/" I am OK with either

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lower case is fine, that's what other languages do

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think lower case for the outer subdirectory is fine, but the R code has to be in an uppercase R. So as long as it doesn't look too weird to have r/R/*.

r/DESCRIPTION Outdated
Title: Bindings to 'Apache' 'Arrow'
Version: 0.0.0.9000
Authors@R: c(
person("Romain", "François", email = "[email protected]", role = c("aut", "cre"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put "Apache Arrow" here and "[email protected]"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r/DESCRIPTION Outdated
@@ -0,0 +1,20 @@
Package: arrow
Title: Bindings to 'Apache' 'Arrow'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This package will be a bit more than bindings. Perhaps "R integration with Apache Arrow"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r/LICENSE.md Outdated
@@ -0,0 +1,194 @@
Apache License
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this license file need to be duplicated? Can the one from the top level be used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably not, deleted it

r/LICENSE.md Outdated
the same “printed page” as the copyright notice for easier identification within
third-party archives.

Copyright 2018 Romain François
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the project license

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleted the license file for now.

r/README.Rmd Outdated

## Installation

First install a release build of the C++ bindings to arrow.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it have to be a release build?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure, I just stole this from @jimhester. I guess it could be other types of builds as long as pkg-config finds the relevant information in the ./configure script

r/README.md Outdated

# arrow

Low level bindings to the C++ Apache Arrow library
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe "R interface to the ..." or "R integration with...", since the extent of the work involved is a bit more than bindings

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Made it R integration with Apache Arrow. for now, we can always revisit this later when we can better define what this is.

PKG_CPPFLAGS=@cflags@
PKG_CXXFLAGS=$(C_VISIBILITY)
CXX_STD=CXX11
PKG_LIBS=@libs@ -Wl,-rpath,/usr/local/lib
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard-coded rpath?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's another thing I borrowed from @jimhester who probably had a reason in this commit of rarrow: jimhester/rarrow@ac3aafc

Can maybe pkg-config give just /usr/local/lib somehow ?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't recall exactly why the issue here did you try linking without this linker option entirely?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That works locally, I just sent a travis job to see what happens 🤞

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 maybe not, getting this now locally:

** testing if installed package can be loaded
Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/3.5/Resources/library/arrow/libs/arrow.so':
  dlopen(/Library/Frameworks/R.framework/Versions/3.5/Resources/library/arrow/libs/arrow.so, 6): Library not loaded: @rpath/libarrow.10.dylib
  Referenced from: /Library/Frameworks/R.framework/Versions/3.5/Resources/library/arrow/libs/arrow.so
  Reason: image not found
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library/arrow’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library/arrow’

Exited with status 1.

@jimhester is this what you had previously in rarrow perhaps ?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does otool -L arrow.so give you?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

romain@purrplex ~/git/apache/arrow/r/src $ otool -L arrow.so
arrow.so:
	arrow.so (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libarrow.10.dylib (compatibility version 10.0.0, current version 10.0.0)
	/Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libR.dylib (compatibility version 3.5.0, current version 3.5.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1454.90.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4)

r/src/hello.cpp Outdated
// under the License.

#include <Rcpp.h>
#include "rrrow_types.h"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r_arrow_types.h?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#pragma once

#include <Rcpp.h>
#undef Free
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this cause a conflict?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, R defines Free here: https://github.com/wch/r-source/blob/trunk/src/include/R_ext/RS.h#L74

==> R CMD INSTALL --preclean --no-multiarch --with-keep.source r

* installing to library ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library’
* installing *source* package ‘arrow’ ...
Found pkg-config cflags and libs!
PKG_CFLAGS=-DNDEBUG -I/usr/local/include
PKG_LIBS=-L/usr/local/lib -larrow
** libs
clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DNDEBUG -I/usr/local/include -I"/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp/include" -I/usr/local/include   -fPIC  -Wno-unused-result -O3 -c RcppExports.cpp -o RcppExports.o
clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DNDEBUG -I/usr/local/include -I"/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp/include" -I/usr/local/include   -fPIC  -Wno-unused-result -O3 -c hello.cpp -o hello.o
In file included from hello.cpp:19:
In file included from ./r_arrow_types.h:22:
In file included from /usr/local/include/arrow/api.h:23:
In file included from /usr/local/include/arrow/array.h:28:
In file included from /usr/local/include/arrow/buffer.h:28:
/usr/local/include/arrow/memory_pool.h:55:38: error: too many arguments provided to function-like macro invocation
  virtual void Free(uint8_t* buffer, int64_t size) = 0;
                                     ^
/Library/Frameworks/R.framework/Resources/include/R_ext/RS.h:74:9: note: macro 'Free' defined here
#define Free(p)        (R_chk_free( (void *)(p) ), (p) = NULL)
        ^
In file included from hello.cpp:19:
In file included from ./r_arrow_types.h:22:
In file included from /usr/local/include/arrow/api.h:23:
In file included from /usr/local/include/arrow/array.h:28:
In file included from /usr/local/include/arrow/buffer.h:28:
/usr/local/include/arrow/memory_pool.h:55:3: error: 'virtual' can only appear on non-static member functions
  virtual void Free(uint8_t* buffer, int64_t size) = 0;
  ^
/usr/local/include/arrow/memory_pool.h:55:16: error: field has incomplete type 'void'
  virtual void Free(uint8_t* buffer, int64_t size) = 0;
               ^
/usr/local/include/arrow/memory_pool.h:79:30: error: too many arguments provided to function-like macro invocation
  void Free(uint8_t* buffer, int64_t size) override;
                             ^
/Library/Frameworks/R.framework/Resources/include/R_ext/RS.h:74:9: note: macro 'Free' defined here
#define Free(p)        (R_chk_free( (void *)(p) ), (p) = NULL)
        ^
In file included from hello.cpp:19:
In file included from ./r_arrow_types.h:22:
In file included from /usr/local/include/arrow/api.h:23:
In file included from /usr/local/include/arrow/array.h:28:
In file included from /usr/local/include/arrow/buffer.h:28:
/usr/local/include/arrow/memory_pool.h:79:8: error: field has incomplete type 'void'
  void Free(uint8_t* buffer, int64_t size) override;
       ^
/usr/local/include/arrow/memory_pool.h:101:30: error: too many arguments provided to function-like macro invocation
  void Free(uint8_t* buffer, int64_t size) override;
                             ^
/Library/Frameworks/R.framework/Resources/include/R_ext/RS.h:74:9: note: macro 'Free' defined here
#define Free(p)        (R_chk_free( (void *)(p) ), (p) = NULL)
        ^
In file included from hello.cpp:19:
In file included from ./r_arrow_types.h:22:
In file included from /usr/local/include/arrow/api.h:23:
In file included from /usr/local/include/arrow/array.h:28:
In file included from /usr/local/include/arrow/buffer.h:28:
/usr/local/include/arrow/memory_pool.h:101:8: error: field has incomplete type 'void'
  void Free(uint8_t* buffer, int64_t size) override;
       ^
7 errors generated.
make: *** [hello.o] Error 1
ERROR: compilation failed for package ‘arrow’
* removing ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library/arrow’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library/arrow’

It looks like an alternative is to define STRICT_R_HEADERS so that it does not define Free as a macro.

#define STRICT_R_HEADERS
#include <Rcpp.h>
#include <arrow/api.h>

Copy link
Contributor Author

@romainfrancois romainfrancois left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might need some help from @jimhester about rpath

@@ -31,6 +31,7 @@ export ARROW_JS_DIR=${TRAVIS_BUILD_DIR}/js
export ARROW_INTEGRATION_DIR=$TRAVIS_BUILD_DIR/integration
export ARROW_RUBY_DIR=$TRAVIS_BUILD_DIR/ruby
export ARROW_RUST_DIR=${TRAVIS_BUILD_DIR}/rust
export ARROW_R_DIR=${TRAVIS_BUILD_DIR}/r
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lower case is fine, that's what other languages do

r/DESCRIPTION Outdated
@@ -0,0 +1,20 @@
Package: arrow
Title: Bindings to 'Apache' 'Arrow'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r/DESCRIPTION Outdated
Title: Bindings to 'Apache' 'Arrow'
Version: 0.0.0.9000
Authors@R: c(
person("Romain", "François", email = "[email protected]", role = c("aut", "cre"))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r/LICENSE.md Outdated
@@ -0,0 +1,194 @@
Apache License
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably not, deleted it

r/LICENSE.md Outdated
the same “printed page” as the copyright notice for easier identification within
third-party archives.

Copyright 2018 Romain François
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleted the license file for now.

r/README.Rmd Outdated

## Installation

First install a release build of the C++ bindings to arrow.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure, I just stole this from @jimhester. I guess it could be other types of builds as long as pkg-config finds the relevant information in the ./configure script

r/README.md Outdated

# arrow

Low level bindings to the C++ Apache Arrow library
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Made it R integration with Apache Arrow. for now, we can always revisit this later when we can better define what this is.

PKG_CPPFLAGS=@cflags@
PKG_CXXFLAGS=$(C_VISIBILITY)
CXX_STD=CXX11
PKG_LIBS=@libs@ -Wl,-rpath,/usr/local/lib
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's another thing I borrowed from @jimhester who probably had a reason in this commit of rarrow: jimhester/rarrow@ac3aafc

Can maybe pkg-config give just /usr/local/lib somehow ?

r/src/hello.cpp Outdated
// under the License.

#include <Rcpp.h>
#include "rrrow_types.h"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#pragma once

#include <Rcpp.h>
#undef Free
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, R defines Free here: https://github.com/wch/r-source/blob/trunk/src/include/R_ext/RS.h#L74

==> R CMD INSTALL --preclean --no-multiarch --with-keep.source r

* installing to library ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library’
* installing *source* package ‘arrow’ ...
Found pkg-config cflags and libs!
PKG_CFLAGS=-DNDEBUG -I/usr/local/include
PKG_LIBS=-L/usr/local/lib -larrow
** libs
clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DNDEBUG -I/usr/local/include -I"/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp/include" -I/usr/local/include   -fPIC  -Wno-unused-result -O3 -c RcppExports.cpp -o RcppExports.o
clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DNDEBUG -I/usr/local/include -I"/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp/include" -I/usr/local/include   -fPIC  -Wno-unused-result -O3 -c hello.cpp -o hello.o
In file included from hello.cpp:19:
In file included from ./r_arrow_types.h:22:
In file included from /usr/local/include/arrow/api.h:23:
In file included from /usr/local/include/arrow/array.h:28:
In file included from /usr/local/include/arrow/buffer.h:28:
/usr/local/include/arrow/memory_pool.h:55:38: error: too many arguments provided to function-like macro invocation
  virtual void Free(uint8_t* buffer, int64_t size) = 0;
                                     ^
/Library/Frameworks/R.framework/Resources/include/R_ext/RS.h:74:9: note: macro 'Free' defined here
#define Free(p)        (R_chk_free( (void *)(p) ), (p) = NULL)
        ^
In file included from hello.cpp:19:
In file included from ./r_arrow_types.h:22:
In file included from /usr/local/include/arrow/api.h:23:
In file included from /usr/local/include/arrow/array.h:28:
In file included from /usr/local/include/arrow/buffer.h:28:
/usr/local/include/arrow/memory_pool.h:55:3: error: 'virtual' can only appear on non-static member functions
  virtual void Free(uint8_t* buffer, int64_t size) = 0;
  ^
/usr/local/include/arrow/memory_pool.h:55:16: error: field has incomplete type 'void'
  virtual void Free(uint8_t* buffer, int64_t size) = 0;
               ^
/usr/local/include/arrow/memory_pool.h:79:30: error: too many arguments provided to function-like macro invocation
  void Free(uint8_t* buffer, int64_t size) override;
                             ^
/Library/Frameworks/R.framework/Resources/include/R_ext/RS.h:74:9: note: macro 'Free' defined here
#define Free(p)        (R_chk_free( (void *)(p) ), (p) = NULL)
        ^
In file included from hello.cpp:19:
In file included from ./r_arrow_types.h:22:
In file included from /usr/local/include/arrow/api.h:23:
In file included from /usr/local/include/arrow/array.h:28:
In file included from /usr/local/include/arrow/buffer.h:28:
/usr/local/include/arrow/memory_pool.h:79:8: error: field has incomplete type 'void'
  void Free(uint8_t* buffer, int64_t size) override;
       ^
/usr/local/include/arrow/memory_pool.h:101:30: error: too many arguments provided to function-like macro invocation
  void Free(uint8_t* buffer, int64_t size) override;
                             ^
/Library/Frameworks/R.framework/Resources/include/R_ext/RS.h:74:9: note: macro 'Free' defined here
#define Free(p)        (R_chk_free( (void *)(p) ), (p) = NULL)
        ^
In file included from hello.cpp:19:
In file included from ./r_arrow_types.h:22:
In file included from /usr/local/include/arrow/api.h:23:
In file included from /usr/local/include/arrow/array.h:28:
In file included from /usr/local/include/arrow/buffer.h:28:
/usr/local/include/arrow/memory_pool.h:101:8: error: field has incomplete type 'void'
  void Free(uint8_t* buffer, int64_t size) override;
       ^
7 errors generated.
make: *** [hello.o] Error 1
ERROR: compilation failed for package ‘arrow’
* removing ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library/arrow’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library/arrow’

It looks like an alternative is to define STRICT_R_HEADERS so that it does not define Free as a macro.

#define STRICT_R_HEADERS
#include <Rcpp.h>
#include <arrow/api.h>

romainfrancois added a commit to romainfrancois/arrow that referenced this pull request Sep 7, 2018
Merge branch 'r-bootstrap' into r-dev

# Conflicts:
#	dev/release/rat_exclude_files.txt
#	r/.Rbuildignore
#	r/DESCRIPTION
#	r/NAMESPACE
#	r/R/RcppExports.R
#	r/README.Rmd
#	r/README.md
#	r/src/RcppExports.cpp
Copy link
Member

@wesm wesm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good except for the licensing issue w/ anticonf cc @jeroen

# specific language governing permissions and limitations
# under the License.

# Anticonf (tm) script by Jeroen Ooms, Jim Hester (2017)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed this in my first pass. What is the license of this file? For future reference, any third party IP has to be carefully documented -- e.g. if you copy paste code from any other codebase the license must be noted and possibly added to https://github.com/apache/arrow/blob/master/LICENSE.txt. I'm especially wary of things from the R ecosystem because of GPL-ness

@jeroen
Copy link
Contributor

jeroen commented Sep 7, 2018

Not sure where exactly the script was adapted from, but you can use all my code under MIT license or anything compatible (i.e. pretty much any license).

@jeroen
Copy link
Contributor

jeroen commented Sep 7, 2018

However it looks like the information in that script is not accurate. The idea is to display which deb / rpm / brew package can be used to obtain the library. However there is no package called "arrow".

@jimhester
Copy link

@jeroen you are correct, in my prototype I changed the brew one, but not the rest. Looks like there are unofficial debian packages at https://arrow.apache.org/install/#c-and-glib-c-packages-for-debian-gnulinux-ubuntu-and-centos-unofficial, so these should be changed to libarrow-dev or similar and the error message with maybe a link to that page.

Change-Id: I29dd0a8490bb545e68c15510b683a676fb3c611c
Copy link
Member

@wesm wesm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. I added a note and LICENSE addendum about the anticonf license. I will merge this shortly

@wesm wesm closed this in e7b269d Sep 7, 2018
@romainfrancois
Copy link
Contributor Author

Thanks 🎉

romainfrancois added a commit to romainfrancois/arrow that referenced this pull request Sep 10, 2018
romainfrancois added a commit to romainfrancois/arrow that referenced this pull request Sep 10, 2018
romainfrancois added a commit to romainfrancois/arrow that referenced this pull request Sep 20, 2018
romainfrancois added a commit to romainfrancois/arrow that referenced this pull request Sep 20, 2018
@romainfrancois romainfrancois deleted the r-bootstrap branch September 21, 2018 14:49
romainfrancois added a commit to romainfrancois/arrow that referenced this pull request Sep 24, 2018
romainfrancois added a commit to romainfrancois/arrow that referenced this pull request Sep 24, 2018
wesm pushed a commit that referenced this pull request Sep 25, 2018
* Wrapping C++ pointers to arrow objects as R6 classes holding an R external pointer.
* Factory functions for the metadata types, int32(), ...
* Factory to create schemas and struct
* Create Array, RecordBatch, Table from R vectors and data frames. initially only support integer (int32), numeric (float64) and raw (int8) vectors.
* Reading and Writing record batches and Table to files.

Author: Romain Francois <[email protected]>

Closes #2596 from romainfrancois/r-dev-buffer and squashes the following commits:

9ab1882 <Romain Francois> mark Roxygen and Rcpp generated files
661f370 <Romain Francois> Using FirstTimeBitmapWriter instead of BitmapWriter.
e81b72b <Romain Francois> only set null_bitmap if null_count > 0
bfe853d <Romain Francois> using 0-based indices in the tests.
b391556 <Romain Francois> Also use arrow::internak::BitmapWriter
9e60555 <Romain Francois> name fixes. Using __ consistently
bf814bb <Romain Francois> Using arrow::internal::BitmapReader
c8aa703 <Romain Francois> Also use std::shared_ptr for MemoryPool.
2aa8a5f <Romain Francois> need dev version of `vctrs`
394bd33 <Romain Francois> 🐀 + RecordBatch$Slice
de93a4f <Romain Francois> RecordBatch tests
9d208a4 <Romain Francois> +Array$RangeEquals
f860063 <Romain Francois> Move each class to their own file
a89a9a8 <Romain Francois> Move RecordBatch impl to own file
a2f9f51 <Romain Francois> correctly handling offset()
8263c0d <Romain Francois> + tests for ChunkedArray
e02e24f <Romain Francois> +chunked_array and tests
b20e4b0 <Romain Francois> More tests
d11cda0 <Romain Francois> +R6 class ChunkedArray
29af2ea <Romain Francois> license headers
2f53ebf <Romain Francois> Additional tests for read_arrow / write_arrow
4237c32 <Romain Francois> Clear the bit for non NA.
ede8e44 <Romain Francois> Handle null buffer in R <-> Array conversions
a5b8190 <Romain Francois> update README with example of reading/writing arrow::Table
d951db8 <Romain Francois> "documentation" to quiet check()
908c2ac <Romain Francois> read_arrow and write_arrow now relate to arrow::Table.
110b00d <Romain Francois> resolving conflicts
ae55f8b <Romain Francois> ..
767e9d9 <Romain Francois> more generic print method
8d8cdd1 <Romain Francois> + read_arrow / write_arrow for now
c1385a0 <Romain Francois> export Array_as_vector, +Array$ToString
23fbd01 <Romain Francois> + column names
97659ff <Romain Francois> + as_tibble.arrow::RecordBatch
fa4ee22 <Romain Francois> + read_record_batch
f27eeba <Romain Francois> - MakeArray
4977bb2 <Romain Francois> no need to make ArrayData directly
ef7cda1 <Romain Francois> class constructors only take the external pointers, logic moved to factory functions
81e059a <Romain Francois> rebasing
421e471 <Romain Francois> +macro R_ERROR_NOT_OK similar to RETURN_NOT_OK but that Rcpp::stop()s
f5e3eff <Romain Francois> attempt RecordBatch$to_file
79205fb <Romain Francois> initial stab at arrow::table(data.frame)
f6f1775 <Romain Francois> s/data/.data/
b9c215b <Romain Francois> "document" array and record_batch
edf6098 <Romain Francois> Need to install `vctrs` from github for now
6aecdce <Romain Francois> skip using rpath linker option
b8dac54 <Romain Francois> +RecordBatch$schema
1fc3cc2 <Romain Francois> no longer need this
05da931 <Romain Francois> initial stab at record_batch
f4d0a34 <Romain Francois> must include arrow_types.h first
aee2d0a <Romain Francois> initial stab at arrow::array
a6ae2f3 <Romain Francois> cleanup
e14b546 <Romain Francois> follow up from @wesm comments on #2489
36e9801 <Romain Francois> + installation instructions
108caf9 <Romain Francois> not checking for headers on these files
b829bdf <Romain Francois> initial R 📦 with travis setup and testthat suite, that links to arrow c++ library and calls arrow::int32()
26e712d <Romain Francois> Initial work for type metadata, with tests.
e251299 <Romain Francois> + installation instructions
a9a8bbb <Romain Francois> not checking for headers on these files
e0a7eff <Romain Francois> initial R 📦 with travis setup and testthat suite, that links to arrow c++ library and calls arrow::int32()
b1c1109 <Romain Francois> finished rebasing after initial R patch merged
887df48 <Romain Francois> skip using rpath linker option
a6de975 <Romain Francois> cleanup
8526e51 <Romain Francois> follow up from @wesm comments on #2489
f03a277 <Romain Francois> + installation instructions
0995ca4 <Romain Francois> not checking for headers on these files
1cb547e <Romain Francois> initial R 📦 with travis setup and testthat suite, that links to arrow c++ library and calls arrow::int32()
705c125 <Romain Francois> exclude Rd files 🐀
605e302 <Romain Francois> time32 only handles second and millisecond time64 only handles microsecond and nanosecond
afdbae6 <Romain Francois> + licence header for R6.R file
65563f5 <Romain Francois> minimal documentation for check()
b7135c7 <Romain Francois> stop exporting everything
6aaf192 <Romain Francois> ignoring the .clang-format file
d854f2f <Romain Francois> + license headers for R files 🙊
d992b26 <Romain Francois> Initial work for type metadata, with tests.
614dd07 <Romain Francois> + installation instructions
afce06a <Romain Francois> initial R 📦 with travis setup and testthat suite, that links to arrow c++ library and calls arrow::int32()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants