Skip to content
This repository has been archived by the owner on Aug 13, 2022. It is now read-only.
/ rust-csv-py Public archive

Python CSV reader using the csv crate and PyO3

License

Notifications You must be signed in to change notification settings

joar/rust-csv-py

Repository files navigation

rust-csv + PyO3 = Not much slower than csv 🎉

travis-badge

BIG DISCLAIMER

  • This is not a production-ready library.
  • I'm not a production-ready Rust programmer.
  • Either Python 3's csv stdlib module is pretty %#!& fast or my Rust code is %#!& slow.

Installation

pip install rustcsv

Examples

CSVReader instance from path

examples/reader_from_path.py:

import tempfile
from rustcsv import CSVReader

# Create a temporary file to put our CSV content in,
# automatically delete it once we're done.
with tempfile.NamedTemporaryFile(mode="w") as writable_fd:
    writable_fd.write(
        """\
spam1,spam2,spam3
spam4,spam5,spam6
"""
    )
    writable_fd.flush()

    for row_number, row in enumerate(CSVReader(writable_fd.name), start=1):
        print(
            "row #{row_number}: {row}".format(row_number=row_number, row=row)
        )

# Prints:
# row #1: ("spam1", "spam2", "spam3")
# row #2: ("spam4", "spam5", "spam6")

CSVReader instance from a binary file object

examples/reader_from_file_object.py:

import tempfile
from rustcsv import CSVReader

# Create a temporary file to put our CSV content in,
# automatically delete it once we're done.
with tempfile.NamedTemporaryFile(mode="w") as writable_fd:
    writable_fd.write(
        """\
spam1,spam2,spam3
spam4,spam5,spam6
"""
    )
    writable_fd.flush()

    readable_fd = open(writable_fd.name, "rb")

    for row_number, row in enumerate(CSVReader(readable_fd), start=1):
        print(
            "row #{row_number}: {row}".format(row_number=row_number, row=row)
        )

# Prints:
# row #1: ("spam1", "spam2", "spam3")
# row #2: ("spam4", "spam5", "spam6")

Development

Development Installation

Install and build the extension locally from e.g. a git checkout.

Requirements

Install Python dependencies

pipenv install --dev

Build the rustcsv._rustcsv extension

Either

  1. Using the "debug" cargo profile, or

    make develop-debug
  2. Using the "release" cargo profile

    make develop-release

Run tests

make test

Run benchmarks

make benchmark

Note: make benchmark will always build the extension using the "release" cargo profile.

Benchmarks

Benchmarks are executed as the last step in the Travis CI project.

You can also run it yourself, see Development and Run benchmarks.

References