-
Notifications
You must be signed in to change notification settings - Fork 0
Benchmark
Theodore edited this page Mar 19, 2024
·
1 revision
The following results are obtained via testing on an HDD, the specification can be seen here.
A json object is generated with random data, the fields can be a nested dict, a nested list, or a primitive value. The total encoded size is around 200 MB.
The object is them dumped to files with different thresholds, leading to different amounts of extra space required to store the table of contents.
All valid paths (json pointers) are collected and shuffled and written to disk.
The test opens the file and reads a given number of paths.
with open("path.txt", "r") as f:
with LazyReader(f"archive.msg") as reader:
while p := f.readline():
_ = reader.visit(p.strip())The number of paths ranges from
The thresholds range from
---------------------------------------------------------------------------------------- benchmark: 65 tests --------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
path_count:10**1-block:2**17 3.5691 (1.0) 10.7349 (1.0) 4.3209 (1.0) 0.7170 (1.0) 4.2060 (1.0) 0.5464 (1.0) 19;10 231.4308 (1.0) 271 1
path_count:10**1-block:2**18 4.4467 (1.25) 11.0880 (1.03) 5.2932 (1.23) 0.7720 (1.08) 5.1487 (1.22) 0.7531 (1.38) 27;6 188.9203 (0.82) 223 1
path_count:10**1-block:2**16 6.3947 (1.79) 15.3981 (1.43) 7.5846 (1.76) 0.8687 (1.21) 7.4955 (1.78) 0.8052 (1.47) 30;2 131.8464 (0.57) 157 1
path_count:10**1-block:2**19 7.8197 (2.19) 16.3515 (1.52) 9.7248 (2.25) 1.0042 (1.40) 9.6228 (2.29) 1.1229 (2.05) 24;3 102.8295 (0.44) 131 1
path_count:10**1-block:2**20 7.8955 (2.21) 17.3258 (1.61) 9.6820 (2.24) 1.1022 (1.54) 9.5805 (2.28) 1.1976 (2.19) 26;2 103.2849 (0.45) 129 1
path_count:10**1-block:2**21 8.0091 (2.24) 17.2686 (1.61) 9.8095 (2.27) 1.0438 (1.46) 9.7301 (2.31) 1.2436 (2.28) 22;2 101.9418 (0.44) 125 1
path_count:10**1-block:2**15 9.9265 (2.78) 14.1045 (1.31) 12.1139 (2.80) 0.8832 (1.23) 12.0837 (2.87) 1.2518 (2.29) 34;0 82.5499 (0.36) 104 1
path_count:10**1-block:2**14 12.1398 (3.40) 16.9082 (1.58) 14.6358 (3.39) 1.0511 (1.47) 14.6605 (3.49) 1.3925 (2.55) 32;0 68.3257 (0.30) 82 1
path_count:10**1-block:2**13 15.6434 (4.38) 20.1302 (1.88) 17.6125 (4.08) 0.9533 (1.33) 17.5828 (4.18) 1.2939 (2.37) 22;0 56.7779 (0.25) 65 1
path_count:10**1-block:2**12 20.0139 (5.61) 31.7688 (2.96) 22.5658 (5.22) 1.7566 (2.45) 22.2649 (5.29) 1.5239 (2.79) 6;1 44.3149 (0.19) 49 1
path_count:10**2-block:2**13 28.2981 (7.93) 46.1233 (4.30) 32.0816 (7.42) 3.3600 (4.69) 31.5012 (7.49) 2.4802 (4.54) 3;2 31.1705 (0.13) 33 1
path_count:10**2-block:2**17 29.1887 (8.18) 34.9248 (3.25) 31.3133 (7.25) 1.2989 (1.81) 31.3154 (7.45) 1.4802 (2.71) 10;1 31.9353 (0.14) 36 1
path_count:10**2-block:2**16 29.6543 (8.31) 45.1725 (4.21) 33.3722 (7.72) 2.5678 (3.58) 33.1410 (7.88) 1.4630 (2.68) 3;3 29.9651 (0.13) 31 1
path_count:10**2-block:2**12 30.6826 (8.60) 36.3438 (3.39) 33.1682 (7.68) 1.3948 (1.95) 33.1054 (7.87) 1.9042 (3.49) 9;0 30.1494 (0.13) 33 1
path_count:10**2-block:2**14 31.0434 (8.70) 40.2329 (3.75) 34.1496 (7.90) 1.8350 (2.56) 34.1650 (8.12) 2.1377 (3.91) 10;1 29.2829 (0.13) 33 1
path_count:10**2-block:2**15 32.3306 (9.06) 37.5863 (3.50) 34.9235 (8.08) 1.1798 (1.65) 34.9384 (8.31) 1.7058 (3.12) 10;0 28.6340 (0.12) 30 1
path_count:10**2-block:2**18 56.9170 (15.95) 64.9282 (6.05) 60.2461 (13.94) 2.0651 (2.88) 60.4064 (14.36) 2.7486 (5.03) 5;0 16.5986 (0.07) 18 1
path_count:10**2-block:2**19 116.6076 (32.67) 122.1983 (11.38) 119.5012 (27.66) 2.3028 (3.21) 120.5038 (28.65) 4.4626 (8.17) 4;0 8.3681 (0.04) 9 1
path_count:10**1-block:2**23 120.2478 (33.69) 127.0963 (11.84) 123.9217 (28.68) 2.4462 (3.41) 123.6791 (29.41) 4.4952 (8.23) 4;0 8.0696 (0.03) 9 1
path_count:10**1-block:2**22 121.8187 (34.13) 129.7571 (12.09) 126.1133 (29.19) 2.4491 (3.42) 125.5242 (29.84) 3.3412 (6.11) 3;0 7.9294 (0.03) 9 1
path_count:10**3-block:2**12 122.2652 (34.26) 151.8353 (14.14) 130.1510 (30.12) 8.6133 (12.01) 128.8925 (30.64) 4.0784 (7.46) 1;1 7.6834 (0.03) 9 1
path_count:10**2-block:2**20 126.5354 (35.45) 132.4216 (12.34) 128.7649 (29.80) 1.6913 (2.36) 128.4850 (30.55) 1.1516 (2.11) 2;1 7.7661 (0.03) 8 1
path_count:10**1-block:2**24 129.6147 (36.32) 138.0557 (12.86) 132.1618 (30.59) 2.5367 (3.54) 131.7247 (31.32) 2.4724 (4.52) 2;1 7.5665 (0.03) 9 1
path_count:10**3-block:2**13 138.4002 (38.78) 154.9991 (14.44) 143.7696 (33.27) 4.9943 (6.97) 143.0853 (34.02) 3.4257 (6.27) 2;1 6.9556 (0.03) 8 1
path_count:10**3-block:2**14 161.4001 (45.22) 178.3270 (16.61) 169.5983 (39.25) 6.2414 (8.70) 168.1234 (39.97) 10.4301 (19.09) 3;0 5.8963 (0.03) 7 1
path_count:10**2-block:2**21 186.5466 (52.27) 196.2761 (18.28) 191.4675 (44.31) 3.4654 (4.83) 191.1813 (45.45) 4.3208 (7.91) 2;0 5.2228 (0.02) 6 1
path_count:10**3-block:2**15 212.4209 (59.52) 228.0293 (21.24) 219.9730 (50.91) 6.7749 (9.45) 220.1305 (52.34) 12.2143 (22.35) 2;0 4.5460 (0.02) 5 1
path_count:10**3-block:2**16 238.1790 (66.73) 256.4052 (23.89) 247.1855 (57.21) 6.7433 (9.40) 247.6911 (58.89) 8.6993 (15.92) 2;0 4.0455 (0.02) 5 1
path_count:10**3-block:2**17 254.2035 (71.22) 261.1347 (24.33) 257.6822 (59.64) 2.9044 (4.05) 257.5330 (61.23) 5.0334 (9.21) 2;0 3.8807 (0.02) 5 1
path_count:10**3-block:2**18 413.5110 (115.86) 424.9017 (39.58) 417.7619 (96.68) 4.6394 (6.47) 415.7259 (98.84) 6.6039 (12.09) 1;0 2.3937 (0.01) 5 1
path_count:10**4-block:2**12 626.2353 (175.46) 645.4658 (60.13) 637.6895 (147.58) 7.2120 (10.06) 638.6729 (151.85) 8.4487 (15.46) 2;0 1.5682 (0.01) 5 1
path_count:10**3-block:2**19 690.0580 (193.34) 717.1962 (66.81) 704.0787 (162.95) 9.8685 (13.76) 703.5453 (167.27) 11.4782 (21.01) 2;0 1.4203 (0.01) 5 1
path_count:10**2-block:2**24 709.9042 (198.90) 730.4895 (68.05) 725.0971 (167.81) 8.5810 (11.97) 728.3446 (173.17) 6.8613 (12.56) 1;1 1.3791 (0.01) 5 1
path_count:10**3-block:2**20 716.4240 (200.73) 723.8390 (67.43) 719.7805 (166.58) 3.1494 (4.39) 719.3831 (171.04) 5.4869 (10.04) 2;0 1.3893 (0.01) 5 1
path_count:10**2-block:2**22 718.8491 (201.41) 740.0259 (68.94) 727.4667 (168.36) 7.7014 (10.74) 726.0589 (172.62) 5.5488 (10.16) 2;1 1.3746 (0.01) 5 1
path_count:10**2-block:2**23 725.9193 (203.39) 739.1883 (68.86) 730.7509 (169.12) 5.1446 (7.18) 729.4962 (173.44) 6.1227 (11.21) 1;0 1.3685 (0.01) 5 1
path_count:10**4-block:2**13 737.5183 (206.64) 751.4664 (70.00) 743.5828 (172.09) 5.8101 (8.10) 741.8119 (176.37) 9.5767 (17.53) 2;0 1.3448 (0.01) 5 1
path_count:10**3-block:2**21 738.5819 (206.94) 788.1431 (73.42) 769.3587 (178.05) 18.4207 (25.69) 772.1234 (183.58) 15.1980 (27.81) 2;1 1.2998 (0.01) 5 1
path_count:10**4-block:2**17 890.8131 (249.59) 939.3999 (87.51) 919.2423 (212.74) 20.5046 (28.60) 915.9069 (217.76) 33.0585 (60.50) 1;0 1.0879 (0.00) 5 1
path_count:10**3-block:2**24 896.4155 (251.16) 935.5901 (87.15) 913.0660 (211.31) 16.1264 (22.49) 909.9338 (216.34) 26.2972 (48.13) 2;0 1.0952 (0.00) 5 1
path_count:10**4-block:2**14 917.1603 (256.97) 934.8153 (87.08) 925.0660 (214.09) 7.2632 (10.13) 926.2772 (220.23) 11.7111 (21.43) 2;0 1.0810 (0.00) 5 1
path_count:10**3-block:2**23 957.8564 (268.37) 1,003.1707 (93.45) 977.4003 (226.20) 19.5572 (27.28) 968.3429 (230.23) 32.4836 (59.45) 1;0 1.0231 (0.00) 5 1
path_count:10**3-block:2**22 967.0392 (270.95) 996.8894 (92.86) 980.9629 (227.03) 10.8956 (15.20) 980.9988 (233.24) 12.7929 (23.41) 2;0 1.0194 (0.00) 5 1
path_count:10**4-block:2**18 973.5699 (272.78) 1,015.8974 (94.64) 992.8517 (229.78) 19.0092 (26.51) 994.2853 (236.40) 34.9295 (63.93) 2;0 1.0072 (0.00) 5 1
path_count:10**4-block:2**20 982.3121 (275.23) 1,012.0784 (94.28) 1,003.5476 (232.25) 12.3862 (17.28) 1,008.1037 (239.68) 13.8052 (25.27) 1;0 0.9965 (0.00) 5 1
path_count:10**4-block:2**22 983.1779 (275.47) 1,001.1454 (93.26) 991.8386 (229.54) 7.4145 (10.34) 989.2530 (235.20) 11.9700 (21.91) 2;0 1.0082 (0.00) 5 1
path_count:10**4-block:2**19 992.5228 (278.09) 1,012.5777 (94.33) 1,000.3304 (231.51) 7.9332 (11.06) 999.1544 (237.55) 11.2054 (20.51) 1;0 0.9997 (0.00) 5 1
path_count:10**4-block:2**16 992.8677 (278.18) 1,020.6019 (95.07) 1,006.1335 (232.85) 10.7746 (15.03) 1,005.5734 (239.08) 16.3329 (29.89) 2;0 0.9939 (0.00) 5 1
path_count:10**4-block:2**23 997.6451 (279.52) 1,012.4134 (94.31) 1,005.9867 (232.82) 5.7059 (7.96) 1,006.0513 (239.19) 8.1454 (14.91) 2;0 0.9940 (0.00) 5 1
path_count:10**4-block:2**21 1,001.5284 (280.61) 1,044.2003 (97.27) 1,016.7758 (235.31) 16.3316 (22.78) 1,014.2874 (241.15) 16.3967 (30.01) 1;0 0.9835 (0.00) 5 1
path_count:10**4-block:2**24 1,001.8437 (280.70) 1,018.0594 (94.84) 1,007.6604 (233.20) 7.3750 (10.29) 1,002.9604 (238.46) 11.8112 (21.62) 1;0 0.9924 (0.00) 5 1
path_count:10**4-block:2**15 1,014.8380 (284.34) 1,119.7169 (104.31) 1,063.8347 (246.20) 37.2596 (51.97) 1,061.1784 (252.30) 30.1333 (55.15) 2;0 0.9400 (0.00) 5 1
path_count:10**5-block:2**23 1,219.9193 (341.80) 1,253.0316 (116.73) 1,232.9681 (285.35) 13.2145 (18.43) 1,234.8424 (293.59) 17.9213 (32.80) 1;0 0.8111 (0.00) 5 1
path_count:10**5-block:2**20 1,229.3883 (344.45) 1,260.0414 (117.38) 1,244.6405 (288.05) 11.2831 (15.74) 1,242.5092 (295.41) 13.7477 (25.16) 2;0 0.8034 (0.00) 5 1
path_count:10**5-block:2**19 1,230.6877 (344.82) 1,264.1524 (117.76) 1,248.0619 (288.84) 14.7840 (20.62) 1,244.7876 (295.96) 26.5028 (48.50) 2;0 0.8012 (0.00) 5 1
path_count:10**5-block:2**24 1,238.2362 (346.93) 1,252.8960 (116.71) 1,249.2539 (289.12) 6.2238 (8.68) 1,252.2974 (297.74) 5.0243 (9.20) 1;1 0.8005 (0.00) 5 1
path_count:10**5-block:2**22 1,239.9151 (347.40) 1,282.0471 (119.43) 1,260.4928 (291.72) 15.4859 (21.60) 1,257.8933 (299.07) 18.9618 (34.70) 2;0 0.7933 (0.00) 5 1
path_count:10**5-block:2**21 1,245.8745 (349.07) 1,280.2767 (119.26) 1,262.6188 (292.21) 12.9068 (18.00) 1,262.1577 (300.09) 17.7160 (32.42) 2;0 0.7920 (0.00) 5 1
path_count:10**5-block:2**17 1,255.2531 (351.70) 1,315.2295 (122.52) 1,284.2452 (297.21) 23.6645 (33.01) 1,278.2361 (303.91) 36.0818 (66.04) 2;0 0.7787 (0.00) 5 1
path_count:10**5-block:2**18 1,301.3525 (364.62) 1,330.4368 (123.94) 1,313.7513 (304.04) 11.4189 (15.93) 1,314.3602 (312.50) 16.5828 (30.35) 2;0 0.7612 (0.00) 5 1
path_count:10**5-block:2**16 1,511.7260 (423.56) 1,552.3359 (144.61) 1,524.5343 (352.82) 16.8529 (23.50) 1,517.9699 (360.91) 21.7947 (39.89) 1;0 0.6559 (0.00) 5 1
path_count:10**5-block:2**15 1,911.0349 (535.44) 1,921.4396 (178.99) 1,916.0916 (443.44) 4.6823 (6.53) 1,916.9788 (455.77) 8.6053 (15.75) 2;0 0.5219 (0.00) 5 1
path_count:10**5-block:2**14 2,049.8085 (574.32) 2,132.6056 (198.66) 2,093.7431 (484.56) 35.2843 (49.21) 2,102.8778 (499.97) 61.0557 (111.74) 2;0 0.4776 (0.00) 5 1
path_count:10**5-block:2**13 2,394.8802 (671.00) 2,452.7132 (228.48) 2,419.2337 (559.89) 22.4261 (31.28) 2,422.1119 (575.87) 30.0979 (55.08) 2;0 0.4134 (0.00) 5 1
path_count:10**5-block:2**12 2,772.7668 (776.88) 2,827.6577 (263.41) 2,813.6850 (651.17) 23.1839 (32.33) 2,822.8212 (671.14) 20.2932 (37.14) 1;1 0.3554 (0.00) 5 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean