You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.
Currently we parse the Parquet on the client side, so every time even for cache hit, we have to re-parse the Parquet files again.
How about parsing it on the server side, which enables to cache Arrow result and save one disk I/O (no need to store the file into disk in the client)
But it is a trade-off: more cache space is needed on the server side, and more data needs to be transmitted via the network.
The text was updated successfully, but these errors were encountered:
lanlou1554
changed the title
Parse Parquet to Arrow
Optimization: Where to parse Parquet to Arrow
Apr 3, 2024
Currently we parse the Parquet on the client side, so every time even for cache hit, we have to re-parse the Parquet files again.
How about parsing it on the server side, which enables to cache Arrow result and save one disk I/O (no need to store the file into disk in the client)
But it is a trade-off: more cache space is needed on the server side, and more data needs to be transmitted via the network.
The text was updated successfully, but these errors were encountered: