Skip to content

Latest commit

 

History

History
160 lines (115 loc) · 8.31 KB

README.md

File metadata and controls

160 lines (115 loc) · 8.31 KB
Raw

C - read vs mmap vs fread

Read and mmap are both fundamentally important system calls, used to access bytes in files. Read uses the standard file descriptor to access files while mmap maps files to RAM. The purpose of this experiment was to compare performance differences with read when compared to mmap.

Approach

  1. Randomly generate file content for different file sizes.
  2. Calculate file histogram and measure elapsed time - 10 times
  3. Write elapsed time, average time and standard deviation to a file.

These tests were conducted on a Ubuntu 20.04.6 LTS system utilizing an Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz × 6 with 9.4.0 gcc version.

Getting started

Clone the project or download it as a zip and then unzip it.

Conduct standard test

Run

bash script.sh -h

and then use the appropriate flag.

Conduct custom test

Run

make
./main -h

and then use the appropriate flag.

Note: you can firstly use bash script.sh -c to generate test files.

Results

Buffer sizes are in bytes, time is in milliseconds.

File size: 512B

512B read(512) read(1024) read(4096) read(8192) read(16384) mmap fread(512) fread(1024) fread(4096) fread(8192) fread(16384)
Average 0.0039 0.0033 0.0034 0.0045 0.0061 0.0074 0.0429 0.045 0.0443 0.0473 0.0476
Standard deviation 0.0036 0.0038 0.0034 0.0037 0.0035 0.0045 0.0046 0.0051 0.0049 0.0061 0.0058

Better method: read(1024)

File size: 2kB

2kB read(512) read(1024) read(4096) read(8192) read(16384) mmap fread(512) fread(1024) fread(4096) fread(8192) fread(16384)
Average 0.0058 0.0053 0.0044 0.0055 0.0075 0.0088 0.0467 0.0434 0.0447 0.0495 0.0466
Standard deviation 0.004 0.0038 0.0037 0.0044 0.0039 0.0061 0.0044 0.0035 0.004 0.011 0.0041

Better method: read(4096)

File size: 8kB

8kB read(512) read(1024) read(4096) read(8192) read(16384) mmap fread(512) fread(1024) fread(4096) fread(8192) fread(16384)
Average 0.0144 0.0118 0.01 0.0107 0.0117 0.0124 0.0468 0.044 0.0444 0.0485 0.0515
Standard deviation 0.0059 0.0047 0.0046 0.0054 0.0047 0.0066 0.0082 0.0054 0.0048 0.0047 0.0046

Better method: read(4096)

File size: 32kB

32kB read(512) read(1024) read(4096) read(8192) read(16384) mmap fread(512) fread(1024) fread(4096) fread(8192) fread(16384)
Average 0.0466 0.0398 0.0297 0.0298 0.0302 0.0288 0.044 0.0439 0.0467 0.0499 0.0569
Standard deviation 0.0072 0.0117 0.0073 0.0082 0.0056 0.0071 0.0063 0.005 0.0069 0.0049 0.0063

Better method: mmap

File size: 128kB

128kB read(512) read(1024) read(4096) read(8192) read(16384) mmap fread(512) fread(1024) fread(4096) fread(8192) fread(16384)
Average 0.185 0.1407 0.1154 0.1056 0.1089 0.0927 0.0478 0.0441 0.0451 0.0505 0.0591
Standard deviation 0.0167 0.0134 0.0179 0.0145 0.0152 0.0145 0.0063 0.005 0.0049 0.0066 0.0088

Better method: fread(1024)

File size: 512kB

512kB read(512) read(1024) read(4096) read(8192) read(16384) mmap fread(512) fread(1024) fread(4096) fread(8192) fread(16384)
Average 0.7137 0.5605 0.4293 0.4132 0.4039 0.3593 0.0446 0.045 0.0457 0.0486 0.0637
Standard deviation 0.0387 0.0462 0.0352 0.0493 0.0324 0.0911 0.0053 0.0046 0.0049 0.0053 0.0088

Better method: fread(512)

File size: 1MB

1MB read(512) read(1024) read(4096) read(8192) read(16384) mmap fread(512) fread(1024) fread(4096) fread(8192) fread(16384)
Average 1.4308 1.1021 0.8901 0.8104 0.8133 0.6797 0.0464 0.0451 0.0462 0.0499 0.0642
Standard deviation 0.0771 0.0656 0.0668 0.077 0.0673 0.0797 0.0062 0.0102 0.0074 0.0059 0.0148

Better method: fread(1024)

File size: 8MB

8MB read(512) read(1024) read(4096) read(8192) read(16384) mmap fread(512) fread(1024) fread(4096) fread(8192) fread(16384)
Average 11.2172 8.8375 7.0907 6.6529 6.4918 5.1665 0.0454 0.0434 0.0439 0.0483 0.059
Standard deviation 0.6225 0.4995 0.4453 0.5361 0.5419 0.5467 0.0058 0.0049 0.0049 0.0059 0.0094

Better method: fread(1024)

File size: 32MB

32MB read(512) read(1024) read(4096) read(8192) read(16384) mmap fread(512) fread(1024) fread(4096) fread(8192) fread(16384)
Average 44.5635 35.1965 28.2577 26.4077 25.6555 20.9024 0.0442 0.069 0.0473 0.0521 0.0563
Standard deviation 2.3593 1.8763 1.9638 2.1978 1.8554 1.9359 0.0052 0.0261 0.0065 0.0115 0.0065

Better method: fread(512)

File size: 256MB

256MB read(512) read(1024) read(4096) read(8192) read(16384) mmap fread(512) fread(1024) fread(4096) fread(8192) fread(16384)
Average 350.7526 289.6301 259.0249 237.116 207.8174 170.2994 0.0461 0.0441 0.0495 0.0533 0.0568
Standard deviation 14.843 19.1331 30.4049 41.2148 20.9232 16.17 0.0141 0.0047 0.007 0.0072 0.0059

Better method: fread(1024)

File size: 1GB

1GB read(512) read(1024) read(4096) read(8192) read(16384) mmap fread(512) fread(1024) fread(4096) fread(8192) fread(16384)
Average 1390.0621 1104.0805 981.7573 896.7686 876.6501 728.8994 0.0498 0.0439 0.0475 0.0496 0.0559
Standard deviation 92.2652 50.4362 90.6253 89.6159 59.6426 57.3032 0.0124 0.0057 0.0073 0.0059 0.0064

Better method: fread(1024)

Conclusion

For smaller files, the read method is faster when compared to the mmap and fread methods. That changes with large files - mmap is faster than read, and fread is faster than mmap.

For smaller files, the most appropriate buffer size would be 4096B or 8192B, which is the same as a page size on the computer used in the experiment. The page size can be found by running getconf PAGE_SIZE.

For larger files, the most suitable buffer size would be 16kB.