Skip to content

Commit 1a4a904

Browse files
authored
Merge pull request #65 from XtensibleBinaryFormat/develop
Merge Develop
2 parents 21a3963 + 777a9fa commit 1a4a904

22 files changed

+1579
-246
lines changed

.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
indent.log

docs/.gitignore

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
paper.*
2+
!paper.tex
3+
!paper.pdf
4+
presentation.*
5+
!presentation.tex
6+
!presentation.pdf

docs/format_comparison.md

+23-71
Original file line numberDiff line numberDiff line change
@@ -32,79 +32,31 @@ To quote their website: "[CBOR] is a data format whose design goals include the
3232

3333
### XBF
3434

35-
## Tests
35+
## Test Methodology
3636

37-
### Stock Data
37+
The goal of this test is to measure how many bytes a particular format requires in order to transmit a given set of data, as well as how long a request for the data takes.
3838

39-
### Random Person Data
39+
The dataset is one year of Sony stock history downloaded from Yahoo Finance, in CSV format. Here is the link used to download it: <https://query1.finance.yahoo.com/v7/finance/download/SONY?period1=1659398400&period2=1690934400&interval=1d&events=history&includeAdjustedClose=true>
40+
41+
On the client side, the client measures the time it takes to open a connection to the server, send a request of what format it would like back, wait for the data to be sent back, and count the number of bytes received. This loop is performed 100 times and then the average of the times is taken.
42+
43+
On the server side, the server downloads the stock data from Yahoo and parses it into a vector of Rust native structs using the previously mentioned CSV parser implementation. It then waits for a connection, and depending on the request type received serializes the native list into the requested format and sends it over the wire. This serialization result is not cached, and is performed every time a given format is asked for to ensure that parser performance is included in the measured round trip time.
44+
45+
The server was a Digital Ocean droplet running Ubuntu 20.04.5 located in New York City. The client was a laptop running openSUSE Tumbleweed located in Hoboken, NJ.
4046

4147
## Results
4248

43-
### Stock Data
44-
45-
### Random Person Data
46-
47-
## plaintext output data August 6th, 20:31
48-
49-
original stock csv data size: 17160
50-
original random person data size: 13684
51-
52-
Request Type: Stock
53-
Data Format: Csv
54-
Avg Time: 22.826262ms
55-
Bytes Read: Some(19090)
56-
57-
Request Type: Stock
58-
Data Format: MessagePack
59-
Avg Time: 17.011247ms
60-
Bytes Read: Some(11549)
61-
62-
Request Type: Stock
63-
Data Format: Cbor
64-
Avg Time: 21.628923ms
65-
Bytes Read: Some(18479)
66-
67-
Request Type: Stock
68-
Data Format: Json
69-
Avg Time: 31.645693ms
70-
Bytes Read: Some(22103)
71-
72-
Request Type: Stock
73-
Data Format: Xml
74-
Avg Time: 27.781995ms
75-
Bytes Read: Some(32112)
76-
77-
Request Type: Stock
78-
Data Format: Xbf
79-
Avg Time: 16.833106ms
80-
Bytes Read: Some(10140)
81-
82-
Request Type: Person
83-
Data Format: Csv
84-
Avg Time: 22.642948ms
85-
Bytes Read: Some(24254)
86-
87-
Request Type: Person
88-
Data Format: MessagePack
89-
Avg Time: 17.399429ms
90-
Bytes Read: Some(15187)
91-
92-
Request Type: Person
93-
Data Format: Cbor
94-
Avg Time: 22.86973ms
95-
Bytes Read: Some(24089)
96-
97-
Request Type: Person
98-
Data Format: Json
99-
Avg Time: 22.468048ms
100-
Bytes Read: Some(30255)
101-
102-
Request Type: Person
103-
Data Format: Xml
104-
Avg Time: 28.264575ms
105-
Bytes Read: Some(42309)
106-
107-
Request Type: Person
108-
Data Format: Xbf
109-
Avg Time: 22.101917ms
110-
Bytes Read: Some(21751)
49+
Original stock CSV data size: 17160
50+
51+
Native data size: 14558
52+
53+
| Format | Avg Time (ms) | Bytes Read |
54+
| ----------- | ------------- | ---------- |
55+
| CSV | 18.931802 | 16411 |
56+
| MessagePack | 11.220957 | 15565 |
57+
| CBOR | 16.957873 | 25507 |
58+
| JSON | 21.912745 | 31180 |
59+
| XML | 21.873043 | 43699 |
60+
| XBF | 11.322245 | 14686 |
61+
62+
There is a discrepancy between the original CSV data size and the data received by the client.

docs/paper/justfile

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
build:
2+
latexmk -pdf -pv
3+
4+
build_continuous:
5+
latexmk -pdf -pvc
6+
7+
clean:
8+
latexmk -c
9+
10+
deep_clean:
11+
latexmk -C

docs/paper/paper.pdf

141 KB
Binary file not shown.

0 commit comments

Comments
 (0)