create table parquet external_location no values

[ashfordm]

Set-up minio data store with parquet file Weather.parquet
-Can Access file from duckdb
SELECT count(1) FROM 's3://datalake/Weather.parquet';
=> 366

When attempting the same in trino get 0 rows ... please advise

Catalog File ...
connector.name=hive
hive.metastore=file
hive.s3-file-system-type=TRINO
hive.metastore.catalog.dir=s3://datalake/
hive.s3.aws-access-key=xxx
hive.s3.aws-secret-key=xxx
hive.s3.endpoint=127.0.0.1:9000
hive.s3.path-style-access=true
hive.s3.ssl.enabled=false

All below runs wo error and not finding errors in logs (INFO or DEBUG)

drop schema datalake.weather;

CREATE SCHEMA datalake.weather
WITH (location = 's3://datalake/');

drop table datalake.weather.weather_table;

CREATE TABLE datalake.weather.weather_table (
MinTemp DOUBLE,
MaxTemp DOUBLE,
Rainfall DOUBLE,
Evaporation DOUBLE,
Sunshine VARCHAR,
WindGustDir VARCHAR,
WindGustSpeed VARCHAR,
WindDir9am VARCHAR,
WindDir3pm VARCHAR,
WindSpeed9am VARCHAR,
WindSpeed3pm BIGINT,
Humidity9am BIGINT,
Humidity3pm BIGINT,
Pressure9am DOUBLE,
Pressure3pm DOUBLE,
Cloud9am BIGINT,
Cloud3pm BIGINT,
Temp9am DOUBLE,
Temp3pm DOUBLE,
RainToday VARCHAR,
RISK_MM DOUBLE,
RainTomorrow VARCHAR
)
WITH (
format = 'parquet',
external_location = 's3://datalake/Weather.parquet'
);

select count(1) from datalake.weather.weather_table;
=> 0

[ashfordm]

what other information is required?

[awdavidson]

I believe you need to use system.sync_partition_metadata - more information can be found here https://trino.io/docs/current/connector/hive.html#procedures

[ashfordm]

SQL Error [30]: Query failed (#20240425_151922_00018_yt64s): Table is not partitioned: weather.weather_table

[tlm365]

Your property external_location = 's3://datalake/Weather.parquet' looks quite weird. I think you should try to put your data to s3://datalake/weather_table (data file path will be: s3://datalake/weather_table/Weather.parquet). Then, create table command look like this:

CREATE TABLE datalake.weather.weather_table (
...
)
WITH (
format = 'parquet',
external_location = 's3://datalake/weather_table'
);

[ashfordm]

Thank for your help ... that change made another empty table. I'm just interested in reading a parquet file in trino?

[JunseoChoJJ]

Set-up minio data store with parquet file Weather.parquet
-Can Access file from duckdb
SELECT count(1) FROM 's3://datalake/Weather.parquet';

from here so you put your data in minio right? and want to read data from minio using trino? you might use s3a instead s3 I guess

CREATE TABLE datalake.weather.weather_table (
...
)
WITH (
format = 'parquet',
external_location = 's3a://datalake/weather_table'
);

[ashfordm]

...from here so you put your data in minio right?
True
and want to read data from minio using trino?
True
you might use s3a instead s3 I guess

SQL Error [16777216]: Query failed (#20240502_141556_00015_sfafn): Got exception: org.apache.hadoop.fs.UnsupportedFileSystemException No FileSystem for scheme "s3"

... with 3a

SQL Error [16777216]: Query failed (#20240502_141707_00016_sfafn): Got exception: org.apache.hadoop.fs.FileAlreadyExistsException Path is a file: s3a://datalake/Weather.parquet

[ashfordm]

Here's my stack ...
Minio (S3 - Parquet)
Mariadb - Hive Metastore
Trino

[JunseoChoJJ]

did you init schema?
he made some good contents https://github.com/joshuarobinson/trino-on-k8s?tab=readme-ov-file you can follow his instructions and you may got an error with hivemetastore configuration in xml part (I think)

[ashfordm]

did you init schema? Yes