Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--null-data seems to inhibit some optimizations #2832

Open
BurntSushi opened this issue Jun 5, 2024 · 0 comments
Open

--null-data seems to inhibit some optimizations #2832

BurntSushi opened this issue Jun 5, 2024 · 0 comments
Labels
enhancement An enhancement to the functionality of the software.

Comments

@BurntSushi
Copy link
Owner

To reproduce, first create a large binary file:

dd status=progress if=/dev/zero count=10240000 bs=1024 > ./binary-test.bin ; echo 'ZQZQZQZQZQ' >> ./binary-test.bin ; dd if=/dev/zero count=10240000 bs=1024 status=progress >> ./binary-test.bin

And then try to search it with --null-data:

$ time rg --null-data --no-mmap ZQZQZQZQZQ binary-test.bin > out.rg

real    2:00.24
user    1:57.95
sys     2.246
maxmem  19 MB
faults  0

It should not take that long. A quick peak at perf top while the above process was running reveals it is doing a line-by-line search, probably because of some faulty reasoning about the line terminator in this case. Since most of the file is just line terminators, this spends a lot of time iterating over lines.

I found this as part of #2831.

@BurntSushi BurntSushi added the enhancement An enhancement to the functionality of the software. label Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement An enhancement to the functionality of the software.
Projects
None yet
Development

No branches or pull requests

1 participant