Skip to content

Question: How to deserialize to a columnar format #193

@jorgecarleitao

Description

@jorgecarleitao

Hey, Thank you so much for this library!

I am considering deserializing avro to arrow, and I am looking for an efficient way of doing it.

In other row-based formats (CSV, JSON), we have been doing the following:

  1. separate "reading" (Read+Seek -> Vec<Row> or AsyncRead+AsyncSeek -> Future<Vec<Row>>) from "deserializing" (Vec<Row> -> Vec<Columns>)
  2. use a reading to &mut [Row], so that we can use a reuse a single Vec<Row> for multiple "batches of rows".

Could someone offer some guide for doing this?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions