[#4528] improvement(hive-catalog): reduce hive catalog libs size from 146MB to 43MB #4531
Conversation
mchades
force-pushed
the
shrink-hive-lib
branch
from
f85d200 to
070c07d
Compare
mchades
changed the title
mchades
force-pushed
the
shrink-hive-lib
branch
from
070c07d to
52e19f1
Compare
mchades
changed the title
|
Can you please list all the jars after you excluded @mchades? |
|
@jerryshao FYI: after: |
mchades
force-pushed
the
shrink-hive-lib
branch
from
52e19f1 to
078ed87
Compare
|
I'm definitely sure that there're many libs from Hive can be excluded, like llap, so please carefully check them one by one. |
I have excluded llap before, but encountered an error, so let me try to understand the error again. |
mchades
force-pushed
the
shrink-hive-lib
branch
from
078ed87 to
eddbda2
Compare
mchades
changed the title
|
@jerryshao 
|
mchades
force-pushed
the
shrink-hive-lib
branch
from
6bbab8b to
dd66565
Compare
mchades
force-pushed
the
shrink-hive-lib
branch
from
dd66565 to
4c79e0a
Compare
|
Do we need that "hive-vector-code-gen"? |
| implementation(libs.javax.jaxb.api) { | ||
| exclude("*") | ||
| } | ||
| implementation(libs.rome) |
There was a problem hiding this comment.
Why do we need to introduce some dependencies like this?
There was a problem hiding this comment.
Because I excluded some Hive catalog dependencies, RangerHiveIT encountered a ClassNotFoundException, so I introduced these dependencies to fix it.
I think the root cause is that the AuthRanger plugin uses the same class loader as the Hive catalog. Previously, the Ranger plugin used dependencies of the Hive catalog.
| woodstox-core = "5.3.0" | ||
| mail = "1.4.1" | ||
| rome = "1.0" | ||
| jettison = "1.1" |
There was a problem hiding this comment.
If we need to add these new dependencies, we should also update the license.bin file.
What changes were proposed in this pull request?
remove some unnecessary dependencies
Why are the changes needed?
Fix: #4528
Does this PR introduce any user-facing change?
no
How was this patch tested?
CI passed