Verification failure on 30.0.0 RC1 -- failure to read object_store path while building parquet

[alamb]

Describe the bug
The verification of arrow rc fails related to object_store (which I don't think it should depend on)

re #3336

To Reproduce

./dev/release/verify-release-candidate.sh 30.0.0 1
....


running 3 tests
test src/client.rs - client::FlightClient (line 39) - compile ... ok
test src/client.rs - client::FlightClient::do_get (line 167) - compile ... ok
test src/client.rs - client::FlightClient::get_flight_info (line 212) - compile ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.11s

+ cd parquet
+ cargo build
    Updating crates.io index
error: failed to get `object_store` as a dependency of package `parquet v30.0.0 (/private/var/folders/s3/h5hgj43j0bv83shtmz_t_w400000gn/T/arrow-30.0.0.XXXXX.82hofd7V/apache-arrow-rs-30.0.0/parquet)`

Caused by:
  failed to load source for dependency `object_store`

Caused by:
  Unable to update /private/var/folders/s3/h5hgj43j0bv83shtmz_t_w400000gn/T/arrow-30.0.0.XXXXX.82hofd7V/apache-arrow-rs-30.0.0/object_store

Caused by:
  failed to read `/private/var/folders/s3/h5hgj43j0bv83shtmz_t_w400000gn/T/arrow-30.0.0.XXXXX.82hofd7V/apache-arrow-rs-30.0.0/object_store/Cargo.toml`

Caused by:
  No such file or directory (os error 2)
+ cleanup
+ '[' no = yes ']'
+ echo 'Failed to verify release candidate. See /var/folders/s3/h5hgj43j0bv83shtmz_t_w400000gn/T/arrow-30.0.0.XXXXX.82hofd7V for details.'
Failed to verify release candidate. See /var/folders/s3/h5hgj43j0bv83shtmz_t_w400000gn/T/arrow-30.0.0.XXXXX.82hofd7V for details.

Expected behavior
Verify should pass

Additional context
Looking a little more, it appears the issue is that cargo build for parquet uses the object store 🤔

$ cd /var/folders/s3/h5hgj43j0bv83shtmz_t_w400000gn/T/arrow-30.0.0.XXXXX.82hofd7V
$ ls -l
total 2584
-rw-r--r--   1 alamb  staff    99K Dec 29 11:32 KEYS
drwxr-xr-x  48 alamb  staff   1.5K Dec 29 11:32 apache-arrow-rs-30.0.0/
-rw-r--r--   1 alamb  staff   1.2M Dec 29 11:32 apache-arrow-rs-30.0.0.tar.gz
-rw-r--r--   1 alamb  staff   833B Dec 29 11:32 apache-arrow-rs-30.0.0.tar.gz.asc
-rw-r--r--   1 alamb  staff    96B Dec 29 11:32 apache-arrow-rs-30.0.0.tar.gz.sha256
-rw-r--r--   1 alamb  staff   160B Dec 29 11:32 apache-arrow-rs-30.0.0.tar.gz.sha512
$ cd apache-arrow-rs-30.0.0
$(cd parquet && cargo build && cargo test)
    Updating crates.io index
error: failed to get `object_store` as a dependency of package `parquet v30.0.0 (/private/var/folders/s3/h5hgj43j0bv83shtmz_t_w400000gn/T/arrow-30.0.0.XXXXX.82hofd7V/apache-arrow-rs-30.0.0/parquet)`

Caused by:
  failed to load source for dependency `object_store`

Caused by:
  Unable to update /private/var/folders/s3/h5hgj43j0bv83shtmz_t_w400000gn/T/arrow-30.0.0.XXXXX.82hofd7V/apache-arrow-rs-30.0.0/object_store

Caused by:
  failed to read `/private/var/folders/s3/h5hgj43j0bv83shtmz_t_w400000gn/T/arrow-30.0.0.XXXXX.82hofd7V/apache-arrow-rs-30.0.0/object_store/Cargo.toml`

Caused by:
  No such file or directory (os error 2)

[alamb]

@viirya or @tustvold do you have any thoughts about this error? Did we change the feature flags somehow 🤔

[tustvold]

Yes I added parquet integration in #3370

I suspect something funky is occurring with the path dependency, I suspect we will need cargo to strip that out prior to running the verification

[alamb]

I need to run to a family event this afternoon and traveling tomorrow, so am not sure I will be able to work on this for a day or two

[alamb]

Maybe we can add a patch to the verify script to remove the path note in the object store dependency

When I changed parquet/Cargo.toml 's object_store line

object_store = { version = "0.5", path = "../object_store", default-features = false, optional = true }

to

object_store = { version = "0.5", default-features = false, optional = true }

Than it seems to have worked

[viirya]

cd parquet && cargo build && cargo test? I cannot reproduce above error. I think object_store is not enabled by default?

cargo build --features=object_store or cargo test --features=object_store also works. I already synced to latest commit.

[viirya]

Is it possible due to cargo version?

cargo --version                                                          
cargo 1.66.0 (d65d197ad 2022-11-15)

[alamb]

Hi @viirya -- I checked my cargo version and it matches yours

cargo --version
cargo 1.66.0 (d65d197ad 2022-11-15)

I think the difference is that the verify script is running cargo build against the contents of the release tarball for arrow (which doesn't include the object_store directory) --

arrow-rs/dev/release/create-tarball.sh

Line 126 in 9b57621

| $tar --delete ${release}/'object_store' \

Thus the build is still looking for object_store in ../object_store rather than in crates.io as it should.

I will make a PR to update the verification script

[tustvold]

I will make a PR to update the verification script

IMO it is the tarball at fault, it isn't a coherent build system. I can have a play this afternoon if you like, ideally cargo would generate this for us, but not sure if you can make it handle unreleased crates... I can have a play

cargo publish --dry-run --no-verify looks promising

[alamb]

see #3414 for underlying issue. I will close this issue with a workaround