Skip to content

Commit

Permalink
[SPARK-43395][BUILD] Exclude macOS tar extended metadata in make-dist…
Browse files Browse the repository at this point in the history 
…ribution.sh

### What changes were proposed in this pull request?

Add args `--no-mac-metadata --no-xattrs --no-fflags` to `tar` on macOS in `dev/make-distribution.sh` to exclude macOS-specific extended metadata.

### Why are the changes needed?

The binary tarball created on macOS includes extended macOS-specific metadata and xattrs, which causes warnings when unarchiving it on Linux.

Step to reproduce

1. create tarball on macOS (13.3.1)
```
➜  apache-spark git:(master) tar --version
bsdtar 3.5.3 - libarchive 3.5.3 zlib/1.2.11 liblzma/5.0.5 bz2lib/1.0.8
```

```
➜  apache-spark git:(master) dev/make-distribution.sh --tgz
```

2. unarchive the binary tarball on Linux (CentOS-7)

```
➜  ~ tar --version
tar (GNU tar) 1.26
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by John Gilmore and Jay Fenlason.
```

```
➜  ~ tar -xzf spark-3.5.0-SNAPSHOT-bin-3.3.5.tgz
tar: Ignoring unknown extended header keyword `SCHILY.fflags'
tar: Ignoring unknown extended header keyword `LIBARCHIVE.xattr.com.apple.FinderInfo'
```

### Does this PR introduce _any_ user-facing change?

No, dev only.

### How was this patch tested?

Create binary tarball on macOS then unarchive on Linux, warnings disappear after this change.

Closes apache#41074 from pan3793/SPARK-43395.

Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Sean Owen <[email protected]>
(cherry picked from commit 2d0240d)
Signed-off-by: Sean Owen <[email protected]>
  • Loading branch information
@pan3793
pan3793 authored and Gladwin committed Oct 10, 2023
1 parent d7c034b commit eea5dac
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion dev/make-distribution.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,10 @@ if [ "$MAKE_TGZ" == "true" ]; then
TARDIR="$SPARK_HOME/$TARDIR_NAME"
rm -rf "$TARDIR"
cp -r "$DISTDIR" "$TARDIR"
tar czf "spark-$VERSION-bin-$NAME.tgz" -C "$SPARK_HOME" "$TARDIR_NAME"
TAR="tar"
if [ "$(uname -s)" = "Darwin" ]; then
TAR="tar --no-mac-metadata --no-xattrs --no-fflags"
fi
$TAR -czf "spark-$VERSION-bin-$NAME.tgz" -C "$SPARK_HOME" "$TARDIR_NAME"
rm -rf "$TARDIR"
fi

0 comments on commit eea5dac

Please sign in to comment.