Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support ingesting Timestamp(nanoseconds) from Iceberg #19163

Open
lmatz opened this issue Oct 29, 2024 · 4 comments
Open

feat: support ingesting Timestamp(nanoseconds) from Iceberg #19163

lmatz opened this issue Oct 29, 2024 · 4 comments
Assignees
Milestone

Comments

@lmatz
Copy link
Contributor

lmatz commented Oct 29, 2024

Right now

ERROR:  Failed to run the query

Caused by:
  DataInvalid => Unsupported Arrow data type: Timestamp(Nanosecond, None)

https://github.com/apache/iceberg/milestone/42?closed=1
apache/iceberg#8657
apache/iceberg-rust#542

@github-actions github-actions bot added this to the release-2.2 milestone Oct 29, 2024
@chenzl25
Copy link
Contributor

If risingwave doesn't have a nanoseconds timestamp type, we can't support it. The issue as decimal with 38 digits.

@lmatz
Copy link
Contributor Author

lmatz commented Oct 29, 2024

how about a struct of timestamp + integer (for the nanosecond part) as a workaround

@chenzl25
Copy link
Contributor

how about a struct of timestamp + integer (for the nanosecond part) as a workaround

Do you mean use a struct of timestamp + integer to implement a nanoseconds timestamp in risingwave?

@chenzl25
Copy link
Contributor

@lmatz The Iceberg table currently doesn't contain a nano timestamp column type (because no engine can create such a table at least for now). The original time stamp issued came from the parquet file source.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants