[Bug] [connector-file-base-hadoop] The krb5_path configuration of the HdfsFile source does not take effect

[JeremyXin]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

In my requirement, I need to sync data from one hdfs to another, for which I execute seatunnel using the following configuration, but an error is reported:

Caused by: javax.security.auth.login.LoginException: Cannot locate KDC
at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:804)
at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1924)
at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1836)
... 17 more

The krb5.conf specified in the seatunnel configuration file contains the right KDC configurations, but it looks like does not take effect. The program appears to have read the /etc/krb5.conf path file by default.

After I change the content of /etc/krb5.conf to that of the krb5.conf file in the seatunnel configuration, the program runs normally.

Question:

The krb5_path configuration of source HdfsFile does not take effect. The program will only read the /etc/krb5.conf path file by default

SeaTunnel Version

SeaTunnel Version=2.3.7

SeaTunnel Config

source {
  HdfsFile {
    path = "/user/hive/test/dt=2024-10-15"
    file_format_type = "parquet"
    fs.defaultFS = "hdfs://hadoop"
    hdfs_site_path = "/local/hadoop/hdfs-site.xml"
    krb5_path = "/local/kerberos/krb5.conf"
    kerberos_principal = "xxx"
    kerberos_keytab_path = "/local/kerberos/xxx.keytab"
  }
}

sink {
  HdfsFile {
    fs.defaultFS = "hdfs://hadoop"
    path = "/user/hive/test_result_table"
    tmp_path = "/tmp/seatunnel"
    file_format_type = "parquet"
    hdfs_site_path = "/local/hadoop2/hdfs-site.xml"
    have_partition = true
    partition_by = ["dt"]
  }
}

Running Command

/opt/soft/seatunnel/apache-seatunnel-2.3.7/bin/seatunnel.sh --config /config/hdfs-sync-data.conf -m local

Error Exception

Caused by: javax.security.auth.login.LoginException: Cannot locate KDC
		at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:804)
		at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at java.lang.reflect.Method.invoke(Method.java:498)
		at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
		at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
		at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
		at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
		at java.security.AccessController.doPrivileged(Native Method)
		at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
		at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
		at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1924)
		at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1836)
		... 17 more
	Caused by: KrbException: Cannot locate KDC
		at sun.security.krb5.Config.getKDCList(Config.java:1084)
		at sun.security.krb5.KdcComm.send(KdcComm.java:218)
		at sun.security.krb5.KdcComm.send(KdcComm.java:200)
		at sun.security.krb5.KrbAsReqBuilder.send(KrbAsReqBuilder.java:316)
		at sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:361)
		at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:776)
		... 31 more

Zeta or Flink or Spark Version

No response

Java or Scala Version

jdk verson: 1.8 scala version 2.12

Screenshots

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[github-actions]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.