Skip to content

Commit

Permalink
update 1brc README
Browse files Browse the repository at this point in the history
  • Loading branch information
briangu committed Jan 7, 2024
1 parent 17f66bc commit 3570c48
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions examples/1brc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ Example for 1 Billion Row contest.

KlongPy isn't very suited to this problem because most of the problem is processing the data as it's read. For KlongPy to shine here, the data needs to already be loaded into memory so NumPy can do its magic.

So for this example, we should a few neat things you can do with KlongPy.
So for this example, we show a few neat things you can do with KlongPy.

1. Use Python multiprocessing to parallel read the data. This is the normal solution that's fast.
1. Use Python multiprocessing to parallel read the data. This is part of the normal fast solution.
2. Instead of processing the data as its read, we buffer the data into chunks and pass it to a KlongPy function which does vector processing on it to compute stats.
3. Results are aggregated in the primary process and reported.

Expand Down

0 comments on commit 3570c48

Please sign in to comment.