Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix][Connector-V2][connector-file-base-hadoop] Fixed HdfsFile source load the krb5_path configuration #7870

Merged
merged 1 commit into from
Oct 18, 2024

Conversation

JeremyXin
Copy link
Contributor

Purpose of this pull request

This pull request fixed HdfsFile source load the krb5_path configuration
issues: #7867

Does this PR introduce any user-facing change?

No

How was this patch tested?

test cases in commit: add seatunnel-engine-examples test cases

use this seatunnel config in seatunnel-examples/seatunnel-engine-examples/src/main/resources/examples/fake_to_console.conf:

`
env {
parallelism = 1
job.mode = "BATCH"
}

source {
HdfsFile {
path = "/user/hive/test/dt=2024-10-15"
file_format_type = "parquet"
fs.defaultFS = "hdfs://hadoop"
hdfs_site_path = "/local/hadoop/hdfs-site.xml"
krb5_path = "/local/kerberos/krb5.conf"
kerberos_principal = "xxx"
kerberos_keytab_path = "/local/kerberos/xxx.keytab"
}
}

sink {
console {
}
}
`

and debug SeaTunnelEngineExample.class,the value of hadoopConf can be seen at the breakpoint of the BaseHdfsFileSource class as shown in the following figure:
hadoopConfig

Check list

@Hisoka-X
Copy link
Member

Hi @JeremyXin , please run 'mvn spotless:apply' to fix code style before commit. Thanks

@JeremyXin
Copy link
Contributor Author

Hi, @Hisoka-X , sry It's my fault. I have ran 'mvn spotless:apply' and resubmitted. Thanks

@Hisoka-X
Copy link
Member

Please do not commit code from seatunnel-example.
image

@JeremyXin
Copy link
Contributor Author

Hi, @Hisoka-X , I have removed this part of commit, please check. Thanks

Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @JeremyXin !

@Hisoka-X
Copy link
Member

The failure of the test case is not related to the current PR.

@Hisoka-X Hisoka-X merged commit cd9836b into apache:dev Oct 18, 2024
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants