Memory issue with current implementation

[malachig]

I was trying to use this to count a large number of sites (~6 million potential germline SNP positions). Even with 10GB of memory requested my jobs were being killed due to exceeded memory usage.

I believe the issue is that it loads the whole ROI into memory before writing out the files needed by bam-readcount?

[chrisamiller]

Yeah, that sounds about right. As a temporary workaround, I imagine you could split by chromosome.

One way to fix this in the code would be to set a maximum number of lines to test at a time, read those, run bam readcount, and loop until the whole file is done.