← back

fork/rust_serialization_benchmark

git clone https://git.julia.blue/fork/rust_serialization_benchmark

David KoloskiAdd basic log benchmark for bebop277eb9b

bebop
5 folders5 files

files

.cargo/
benches/
benchmark_results/
src/
tools/
.gitignore28 B
Cargo.toml2.2 KiB
README.md14.4 KiB
build.rs1.7 KiB
how_to_profile.md1.2 KiB

Rust serialization benchmark

The goal of these benchmarks is to provide thorough and complete benchmarks for various rust serialization frameworks.

These benchmarks are a work in progress

These benchmarks are still being developed and pull requests to improve benchmarks are welcome.

You can use this horrible tiny webpage to turn a benchmark log into nicely-formatted markdown tables.

Format

All tests benchmark the following properties (time or size):

Zero-copy deserialization libraries have an additional set of benchmarks:

Some benchmark results may be italicized and followed by an asterisk. Mouse over these for more details on what situation was benchmarked. Other footnotes are located at the bottom.

Last updated: 7/29/2021

log

This data set is composed of HTTP request logs that are small and contain many strings.

Raw Data

For operations, time per iteration; for size, bytes. Lower is better.

Serialize / deserialize speed and size

Format / LibSerializeDeserializeSizeZlib
abomonation237.86 us<span title="unvalidated">3.2487 ms*</span>1705800507148
bincode527.80 us4.0328 ms1045784374305
borsh526.96 us3.8704 ms885780363280
capnp1.2341 ms†1443216509618
cbor1.8670 ms8.2403 ms1407835407372
flatbuffers2.7526 ms†1276368469962
nachricht4.5599 ms6.9329 ms926221365209
postcard719.88 us4.5856 ms765778312739
prost<span title="populate + encode">5.4255 ms*</span> <span title="encode">1.5852 ms*</span>5.3388 ms764951269811
rkyv306.63 us<span title="unvalidated">3.2056 ms*</span> <span title="validated upfront with error">3.9919 ms*</span>1011488269353
rmp1.5243 ms5.1389 ms784997326654
ron20.290 ms16.247 ms1607459452648
serde_json4.8639 ms9.7947 ms1827461474358
speedy268.85 us3.5334 ms885780363280
alkahest264.70 us†1045784454748

Zero-copy deserialization speed

Format / LibAccessReadUpdate
abomonation<span title="unvalidated">36.589 us*</span><span title="unvalidated">57.773 us*</span>‡
capnp<span title="validated on-demand with error">146.66 ns*</span><span title="validated on-demand with error">496.42 us*</span>‡
flatbuffers<span title="unvalidated">2.9546 ns*</span> <span title="validated upfront with error">2.0092 ms*</span><span title="unvalidated">137.99 us*</span> <span title="validated upfront with error">2.1892 ms*</span>‡
rkyv<span title="unvalidated">1.3871 ns*</span> <span title="validated upfront with error">756.30 us*</span><span title="unvalidated">16.632 us*</span> <span title="validated upfront with error">776.72 us*</span>66.600 us
alkahest<span title="validated on-demand with panic">2.0442 ns*</span><span title="validated on-demand with panic">81.230 us*</span>‡

Comparison

Relative to best. Higher is better.

Serialize / deserialize speed and size

Format / LibSerializeDeserializeSizeZlib
abomonation100.00%<span title="unvalidated">98.67%*</span>44.84%53.11%
bincode45.07%79.49%73.15%71.96%
borsh45.14%82.82%86.36%74.14%
capnp19.27%†53.00%52.85%
cbor12.74%38.90%54.34%66.12%
flatbuffers8.64%†59.93%57.31%
nachricht5.22%46.24%82.59%73.75%
postcard33.04%69.91%99.89%86.13%
prost<span title="populate + encode">4.38%*</span> <span title="encode">15.01%*</span>60.04%100.00%99.83%
rkyv77.57%<span title="unvalidated">100.00%*</span> <span title="validated upfront with error">80.30%*</span>75.63%100.00%
rmp15.60%62.38%97.45%82.46%
ron1.17%19.73%47.59%59.51%
serde_json4.89%32.73%41.86%56.78%
speedy88.47%90.72%86.36%74.14%
alkahest89.86%†73.15%59.23%

Zero-copy deserialization speed

Format / LibAccessReadUpdate
abomonation<span title="unvalidated">0.00%*</span><span title="unvalidated">28.79%*</span>‡
capnp<span title="validated on-demand with error">0.95%*</span><span title="validated on-demand with error">3.35%*</span>‡
flatbuffers<span title="unvalidated">46.95%*</span> <span title="validated upfront with error">0.00%*</span><span title="unvalidated">12.05%*</span> <span title="validated upfront with error">0.76%*</span>‡
rkyv<span title="unvalidated">100.00%*</span> <span title="validated upfront with error">0.00%*</span><span title="unvalidated">100.00%*</span> <span title="validated upfront with error">2.14%*</span>100.00%
alkahest<span title="validated on-demand with panic">67.86%*</span><span title="validated on-demand with panic">20.48%*</span>‡

mesh

The data set is a single mesh. The mesh contains an array of triangles, each of which has three vertices and a normal vector.

Raw Data

For operations, time per iteration; for size, bytes. Lower is better.

Serialize / deserialize speed and size

Format / LibSerializeDeserializeSizeZlib
abomonation238.00 us<span title="unvalidated">1.5709 ms*</span>60000245380836
bincode7.1569 ms7.1089 ms60000085380823
borsh6.3575 ms9.1960 ms60000045380818
capnp21.372 ms†140000886729881
cbor43.461 ms63.095 ms131223247527423
flatbuffers2.6383 ms†60000245380800
nachricht65.692 ms40.529 ms101250307160144
postcard7.4290 ms9.9395 ms60000035380817
prost<span title="populate + encode">34.207 ms*</span> <span title="encode">27.866 ms*</span>21.499 ms87500006683814
rkyv1.0296 ms<span title="unvalidated">1.8563 ms*</span> <span title="validated upfront with error">1.8722 ms*</span>60000084263104
rmp19.365 ms17.591 ms81250066496879
ron199.29 ms279.11 ms221928859009575
serde_json109.08 ms89.157 ms261928839612105
speedy928.63 us2.3347 ms60000045380818
alkahest700.77 us†60000085380823

Zero-copy deserialization speed

Format / LibAccessReadUpdate
abomonation<span title="unvalidated">2.3509 ns*</span><span title="unvalidated">204.32 us*</span>‡
capnp<span title="validated on-demand with error">233.54 ns*</span><span title="validated on-demand with error">6.7789 ms*</span>‡
flatbuffers<span title="unvalidated">2.9247 ns*</span> <span title="validated upfront with error">40.962 ns*</span><span title="unvalidated">184.47 us*</span> <span title="validated upfront with error">186.86 us*</span>‡
rkyv<span title="unvalidated">1.3646 ns*</span> <span title="validated upfront with error">10.994 ns*</span><span title="unvalidated">40.600 us*</span> <span title="validated upfront with error">40.870 us*</span>624.86 us
alkahest<span title="validated on-demand with panic">2.0153 ns*</span><span title="validated on-demand with panic">74.309 us*</span>‡

Comparison

Relative to best. Higher is better.

Serialize / deserialize speed and size

Format / LibSerializeDeserializeSizeZlib
abomonation100.00%<span title="unvalidated">100.00%*</span>100.00%79.23%
bincode3.33%22.10%100.00%79.23%
borsh3.74%17.08%100.00%79.23%
capnp1.11%†42.86%63.35%
cbor0.55%2.49%45.72%56.63%
flatbuffers9.02%†100.00%79.23%
nachricht0.36%3.88%59.26%59.54%
postcard3.20%15.80%100.00%79.23%
prost<span title="populate + encode">0.70%*</span> <span title="encode">0.85%*</span>7.31%68.57%63.78%
rkyv23.12%<span title="unvalidated">84.63%*</span> <span title="validated upfront with error">83.91%*</span>100.00%100.00%
rmp1.23%8.93%73.85%65.62%
ron0.12%0.56%27.04%47.32%
serde_json0.22%1.76%22.91%44.35%
speedy25.63%67.28%100.00%79.23%
alkahest33.96%†100.00%79.23%

Zero-copy deserialization speed

Format / LibAccessReadUpdate
abomonation<span title="unvalidated">58.05%*</span><span title="unvalidated">19.87%*</span>‡
capnp<span title="validated on-demand with error">0.58%*</span><span title="validated on-demand with error">0.60%*</span>‡
flatbuffers<span title="unvalidated">46.66%*</span> <span title="validated upfront with error">3.33%*</span><span title="unvalidated">22.01%*</span> <span title="validated upfront with error">21.73%*</span>‡
rkyv<span title="unvalidated">100.00%*</span> <span title="validated upfront with error">12.41%*</span><span title="unvalidated">100.00%*</span> <span title="validated upfront with error">99.34%*</span>100.00%
alkahest<span title="validated on-demand with panic">67.71%*</span><span title="validated on-demand with panic">54.64%*</span>‡

minecraft_savedata

The data set is composed of Minecraft player saves that contain highly-structured data.

Raw Data

For operations, time per iteration; for size, bytes. Lower is better.

Serialize / deserialize speed and size

Format / LibSerializeDeserializeSizeZlib
abomonation365.54 us<span title="unvalidated">2.4293 ms*</span>1290592389259
bincode755.51 us3.0414 ms569975240897
borsh752.56 us2.9753 ms446595234395
capnp828.77 us†803896336655
cbor2.0224 ms7.8268 ms1109821347562
flatbuffers33.323 ms†844168346957
nachricht4.9511 ms6.0666 ms535881281994
postcard885.43 us3.6000 ms356311213270
prost<span title="populate + encode">6.0083 ms*</span> <span title="encode">2.8965 ms*</span>5.2160 ms596811306728
rkyv538.85 us<span title="unvalidated">2.4413 ms*</span> <span title="validated upfront with error">3.0630 ms*</span>596952241952
rmp1.8155 ms4.2848 ms418025244771
ron10.106 ms16.938 ms1465229439673
serde_json4.7927 ms10.320 ms1623197472162
speedy536.45 us2.6385 ms449595235136
alkahest416.27 us†667570325536

Zero-copy deserialization speed

Format / LibAccessReadUpdate
abomonation<span title="unvalidated">42.169 us*</span><span title="unvalidated">42.559 us*</span>‡
capnp<span title="validated on-demand with error">148.53 ns*</span><span title="validated on-demand with error">5.1845 us*</span>‡
flatbuffers<span title="unvalidated">2.9109 ns*</span> <span title="validated upfront with error">2.5496 ms*</span><span title="unvalidated">3.5856 us*</span> <span title="validated upfront with error">2.5603 ms*</span>‡
rkyv<span title="unvalidated">1.3600 ns*</span> <span title="validated upfront with error">612.41 us*</span><span title="unvalidated">167.71 ns*</span> <span title="validated upfront with error">600.39 us*</span>6.2397 us
alkahest<span title="validated on-demand with panic">2.0319 ns*</span><span title="validated on-demand with panic">31.299 us*</span>‡

Comparison

Relative to best. Higher is better.

Serialize / deserialize speed and size

Format / LibSerializeDeserializeSizeZlib
abomonation100.00%<span title="unvalidated">100.00%*</span>27.61%54.79%
bincode48.38%79.87%62.51%88.53%
borsh48.57%81.65%79.78%90.99%
capnp44.11%†44.32%63.35%
cbor18.07%31.04%32.11%61.36%
flatbuffers1.10%†42.21%61.47%
nachricht7.38%40.04%66.49%75.63%
postcard41.28%67.48%100.00%100.00%
prost<span title="populate + encode">6.08%*</span> <span title="encode">12.62%*</span>46.57%59.70%69.53%
rkyv67.84%<span title="unvalidated">99.51%*</span> <span title="validated upfront with error">79.31%*</span>59.69%88.15%
rmp20.13%56.70%85.24%87.13%
ron3.62%14.34%24.32%48.51%
serde_json7.63%23.54%21.95%45.17%
speedy68.14%92.07%79.25%90.70%
alkahest87.81%†53.37%65.51%

Zero-copy deserialization speed

Format / LibAccessReadUpdate
abomonation<span title="unvalidated">0.00%*</span><span title="unvalidated">0.39%*</span>‡
capnp<span title="validated on-demand with error">0.92%*</span><span title="validated on-demand with error">3.23%*</span>‡
flatbuffers<span title="unvalidated">46.72%*</span> <span title="validated upfront with error">0.00%*</span><span title="unvalidated">4.68%*</span> <span title="validated upfront with error">0.01%*</span>‡
rkyv<span title="unvalidated">100.00%*</span> <span title="validated upfront with error">0.00%*</span><span title="unvalidated">100.00%*</span> <span title="validated upfront with error">0.03%*</span>100.00%
alkahest<span title="validated on-demand with panic">66.93%*</span><span title="validated on-demand with panic">0.54%*</span>‡

Footnotes:

* mouse over for situational details

† do not provide deserialization capabilities, but the user can write their own

‡ do not support buffer mutation (capnp and flatbuffers may but not for rust)