Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix NOTICE and LICENSE in the flink-runtime jar #12145

Merged
merged 1 commit into from
Feb 6, 2025

Conversation

jbonofre
Copy link
Member

@jbonofre jbonofre commented Jan 31, 2025

This is a proposal including:

  • cleanup ASF projects references
  • remove animal sniffer annotations from LICENSE as it's not in the jar or the runtime classpath
  • remove yetus from LICENSE as it's not in the jar or the runtime classpath
  • fix Nessie in LICENSE (NOTICE should go in NOTICE)
  • add Codahale Metrics in LICENSE (it's in the jar)
  • add RoaringBitmap in LICENSE (it's in the jar)
  • add Eclipse Microprofile OpenAPI in LICENSE (it's in the jar)
  • add Zstd in LICENSE (it's in the jar)
  • update Nessie NOTICE (with the correct Nessie version)
  • add Eclipse Microprofile OpenAPI NOTICE

@jbonofre
Copy link
Member Author

@amogh-jahagirdar @Fokko @rdblue this is the change for flink-runtime extracted from #12095

Comment on lines 103 to 70
| * GNU General Public License, Version 2 with the GNU Classpath Exception
|
Copy link
Member Author

@jbonofre jbonofre Jan 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NB: I checked on Nessie "license report" (https://github.com/projectnessie/nessie/releases/download/nessie-0.102.2/nessie-aggregated-license-report-0.102.2.zip):

  1. It's GPL with GNU Classpath Exception. It means that we can "extend" the classpath with Nessie, but problematic to bundle in our jar.
  2. It's dual licenses (GPL + CDDL or GPL + EPL)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GPL, even with the classpath exception, is a Category X license: https://www.apache.org/legal/resolved.html

We can't include it in a release. The Nessie docs need to be updated to remove this before we can ship a release with Nessie.

Copy link
Member Author

@jbonofre jbonofre Feb 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Nessie team informed me that they checked and cleanup their NOTICE (no actual GPL code copied): https://github.com/projectnessie/nessie/blob/main/NOTICE

They also checked the binary distribution and no GPL dependency bundled.

They said they gonna do a new release. I'm already updating the NOTICE here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jbonofre for checking with them!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We updated to Nessie 0.102.4 that fixed their NOTICE in regards of GPL.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this is still too unclear to include in Iceberg. The Nessie binary NOTICE file that applies to Jars (binaries) still includes mention of GPL code. The fixed Nessie NOTICE states that it applies to Nessie source code so this is still not clear from the documentation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the Nessie NOTICE with the 0.102.5 release.

@jbonofre jbonofre force-pushed the notice-fix-flink-runtime branch 2 times, most recently from 14db2b3 to 26c76b7 Compare February 2, 2025 07:36
Copy link
Contributor

@amogh-jahagirdar amogh-jahagirdar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jbonofre , I think we're incorrectly removing stuff that's still in the bundle. Left comments

flink/v1.20/flink-runtime/LICENSE Outdated Show resolved Hide resolved
flink/v1.20/flink-runtime/LICENSE Outdated Show resolved Hide resolved
flink/v1.20/flink-runtime/LICENSE Outdated Show resolved Hide resolved
flink/v1.20/flink-runtime/LICENSE Outdated Show resolved Hide resolved
flink/v1.20/flink-runtime/LICENSE Outdated Show resolved Hide resolved
flink/v1.20/flink-runtime/LICENSE Outdated Show resolved Hide resolved
@jbonofre jbonofre force-pushed the notice-fix-flink-runtime branch from 26c76b7 to 769739a Compare February 3, 2025 17:26
Copy link
Contributor

@Fokko Fokko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure that we capture everthing? Unzipping the JAR shows more dependencies, such as datasketches:

unzip iceberg-flink-runtime-1.20-1.8.0-SNAPSHOT.jar
➜  libs git:(main) ls -lah org/apache/datasketches | head
total 8
drwxr-xr-x  22 fokko.driesprong  staff   704B Feb  1  1980 .
drwxr-xr-x   5 fokko.driesprong  staff   160B Feb  1  1980 ..
drwxr-xr-x  22 fokko.driesprong  staff   704B Feb  1  1980 common
drwxr-xr-x  29 fokko.driesprong  staff   928B Feb  1  1980 cpc
drwxr-xr-x   6 fokko.driesprong  staff   192B Feb  1  1980 fdt
drwxr-xr-x   4 fokko.driesprong  staff   128B Feb  1  1980 filters
drwxr-xr-x  16 fokko.driesprong  staff   512B Feb  1  1980 frequencies
drwxr-xr-x   7 fokko.driesprong  staff   224B Feb  1  1980 hash
drwxr-xr-x  47 fokko.driesprong  staff   1.5K Feb  1  1980 hll

flink/v1.20/flink-runtime/LICENSE Outdated Show resolved Hide resolved
@jbonofre
Copy link
Member Author

jbonofre commented Feb 5, 2025

Are we sure that we capture everthing? Unzipping the JAR shows more dependencies, such as datasketches:


unzip iceberg-flink-runtime-1.20-1.8.0-SNAPSHOT.jar

➜  libs git:(main) ls -lah org/apache/datasketches | head

total 8

drwxr-xr-x  22 fokko.driesprong  staff   704B Feb  1  1980 .

drwxr-xr-x   5 fokko.driesprong  staff   160B Feb  1  1980 ..

drwxr-xr-x  22 fokko.driesprong  staff   704B Feb  1  1980 common

drwxr-xr-x  29 fokko.driesprong  staff   928B Feb  1  1980 cpc

drwxr-xr-x   6 fokko.driesprong  staff   192B Feb  1  1980 fdt

drwxr-xr-x   4 fokko.driesprong  staff   128B Feb  1  1980 filters

drwxr-xr-x  16 fokko.driesprong  staff   512B Feb  1  1980 frequencies

drwxr-xr-x   7 fokko.driesprong  staff   224B Feb  1  1980 hash

drwxr-xr-x  47 fokko.driesprong  staff   1.5K Feb  1  1980 hll

We discussed about not including all ASF projects (not required).

@jbonofre
Copy link
Member Author

jbonofre commented Feb 5, 2025

Are we sure that we capture everthing? Unzipping the JAR shows more dependencies, such as datasketches:


unzip iceberg-flink-runtime-1.20-1.8.0-SNAPSHOT.jar

➜  libs git:(main) ls -lah org/apache/datasketches | head

total 8

drwxr-xr-x  22 fokko.driesprong  staff   704B Feb  1  1980 .

drwxr-xr-x   5 fokko.driesprong  staff   160B Feb  1  1980 ..

drwxr-xr-x  22 fokko.driesprong  staff   704B Feb  1  1980 common

drwxr-xr-x  29 fokko.driesprong  staff   928B Feb  1  1980 cpc

drwxr-xr-x   6 fokko.driesprong  staff   192B Feb  1  1980 fdt

drwxr-xr-x   4 fokko.driesprong  staff   128B Feb  1  1980 filters

drwxr-xr-x  16 fokko.driesprong  staff   512B Feb  1  1980 frequencies

drwxr-xr-x   7 fokko.driesprong  staff   224B Feb  1  1980 hash

drwxr-xr-x  47 fokko.driesprong  staff   1.5K Feb  1  1980 hll

In the first version of the PR, I documented datasketches. But discussing with @rdblue it's not necessary to include the ASF projects.

@jbonofre jbonofre force-pushed the notice-fix-flink-runtime branch 2 times, most recently from d9a5e7f to 79e27fa Compare February 5, 2025 19:54
@Fokko Fokko added this to the Iceberg 1.8.0 milestone Feb 6, 2025
Copy link
Contributor

@rdblue rdblue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, but I'd prefer to remove the Iceberg NOTICE from the copied Nessie NOTICE.

@jbonofre jbonofre force-pushed the notice-fix-flink-runtime branch from 79e27fa to c3a54e0 Compare February 6, 2025 17:25
@rdblue rdblue merged commit a803643 into apache:main Feb 6, 2025
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants