How can I configure my Iceberg REST Catalog with HDFS storage?

[fpierrick]

Hello folks,

I'm trying to configure Iceberg Rest catalog with a backend on Postgresql and warehouse on HDFS but I'm little confused :)
In gravitino-iceberg-rest-server-0.7.0-incubating-bin/conf/gravitino-iceberg-rest-server.conf I set :

gravitino.iceberg-rest.catalog-backend = jdbc
gravitino.iceberg-rest.jdbc-driver = org.postgresql.Driver
gravitino.iceberg-rest.uri = jdbc:postgresql://localhost:5432/iceberg-metalake
gravitino.iceberg-rest.jdbc-user = gravitino
gravitino.iceberg-rest.jdbc-password = XXX
gravitino.iceberg-rest.warehouse = hdfs://<REALM>/apps/iceberg/warehouse`

And I put my core-site.xml and hdfs-site.xml in the conf folder

But when I want to create the catalog in gravitino, It doesn't take the Rest catalog itself but again the backend and storage configuration.

curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" -d '{
  "name":"iceberg_hdfs",
  "comment":"Iceberg on hdfs", 
  "type":"RELATIONAL", 
  "provider":"lakehouse-iceberg", 
  "properties":{
      "catalog-backend":"jdbc", 
      "jdbc-user":"gravitino", 
      "jdbc-driver":"org.postgresql.Driver", 
      "jdbc-password":"XXX", 
      "warehouse":"hdfs://MYREALM/apps/iceberg/warehouse/", 
      "authType":"simple", "uri":"jdbc:postgresql://localhost:5432/iceberg_metalake"
  }
}' http://localhost:8090/api/metalakes/metalake/catalogs

Why it didn't use directly the Iceberg Rest Catalog : http://localhost:9001/iceberg?
I probably missed something :D

I Have also an issue to communicate with hdfs. Is it posible to provide a kerberos keytab?
(Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at java.security.jgss/sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:162) ~[java.security.jgss:?])

Have you any example to configure an Iceberg Rest Catalog with backend in jdbc and storage on HDFS with an HDFS authentication through kerberos?

Thanks for your help

[FANNG1]

There are two servers in Gravitino,

1. Gravitino server with a unified interface to manage Hive, Iceberg, Paimon, Hudi, etc.
2. Gravitino Iceberg REST server with Iceberg REST spec interface to manage Iceberg tables.

Iceberg REST server could use the Iceberg catalog configuration from the Gravitino server but not the inverse. Unfortunately Gravitino only support Kerbors with Hive catalog backend, not support for JDBC backend yet. You could create an Github issue for your requirement or implement it by your self, refer https://github.com/apache/gravitino/blob/main/iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/utils/IcebergCatalogUtil.java#L79-L95