doris是否已经正式弃用libhdfs3？

[dongxiao1198]

我看目前默认都是不走libhdfs3了，社区之前提到过libhdfs3 missing太多feature，其中包括Kerberos么？ 我尝试用doris维护的libhdfs3访问带kerberos的hdfs发现各种问题。其中一个问题faq提到：https://doris.apache.org/zh-CN/docs/3.0/faq/lakehouse-faq

BlockMissingExcetpion: Could not obtain block: BP-XXXXXXXXX No live nodes contain current block

可能的处理方式有：

通过 hdfs fsck file -files -blocks -locations 来查看具体该文件是否健康。
通过 telnet 来检查与 datanode 的连通性。
查看 datanode 日志。
如果出现以下错误：

org.apache.hadoop.hdfs.server.datanode.DataNode: Failed to read expected SASL data transfer protection handshake from client at /XXX.XXX.XXX.XXX:XXXXX. Perhaps the client is running an older version of Hadoop which does not support SASL data transfer protection 则为当前 hdfs 开启了加密传输方式，而客户端未开启导致的错误。

使用下面的任意一种解决方案即可：

拷贝 hdfs-site.xml 以及 core-site.xml 到 be/conf 和 fe/conf 目录。(推荐)
在 hdfs-site.xml 找到相应的配置 dfs.data.transfer.protection，并且在 catalog 里面设置该参数

这个问题看起来只在libhdfs模式修复了，但是libhdfs3是绕不过去的？

thx for your help~

[dongxiao1198]

#9524
[deps] libhdfs3 build enable kerberos support (#9524)

Currently, the libhdfs3 library integrated by doris BE does not support accessing the cluster with kerberos authentication
enabled, and found that kerberos-related dependencies（gsasl and krb5） were not added when build libhdfs3.

so, this pr will enable kerberos support and rebuild libhdfs3 with dependencies gsasl and krb5:

- gsasl version: 1.8.0
- krb5 version: 1.19

#15037
If not set, it can't find gssapi.h, which is generated by krb5.
This will cause the libhdfs3 failed to visit hdfs with kerberos authentication

#15753
In #15037, I modified the build script of libgsasl to enable GSSAPI,
but it is still wrong, because the PATH does not include the thirdparty/installed/bin,
so when building libgsasl, it will report error:
WARNING: MIT Kerberos krb5-config not found, disabling GSSAPI
but krb5-config is in thirdparty/installed/bin.
Without GSSAPI, the libhdfs3 can not access hdfs with kerberos authentication.

#17435
deps add official hadoop libhdfs for x86 (#17435)

This is the first step to introduce official hadoop libhdfs to Doris.
Because the current hdfs client libhdfs3 lacks some important feature and is hard to maintain.

Download the hadoop 3.3.4 binary from hadoop website: https://hadoop.apache.org/releases.html
Extract libs and headers which are used for libhdfs, and pack them into hadoop_lib_3.3.4-x86.tar.gz
Upload it to https://github.com/apache/doris-thirdparty/releases/tag/hadoop-libs-3.3.4

TODO:
The hadoop libs for arm is missing, we need to find a way to build it

看着改了好多次去fix libhdfs3访问Kerberos hdfs的问题。。不知道最后一个patch进去后是否在kerberos下真的能work。。然后看到最后还是重新引入了libhdfs。。