Benchmarks · committed v1.0.1 full run
How Surp compares,
measured, not claimed.
A deterministic regression suite (surp-bench) encodes and decodes six fixed datasets with Surp, Surp + Dedup, JSON, MessagePack, CBOR, and Protocol Buffers, then writes the raw numbers below to docs/assets/bench/v1.0.1/. No vendor cherry-picking — every dataset, every operation, every format is shown.
Datasets
6
Formats
6
Iterations
10
Regressions
0
Methodology
The harness is deterministic on purpose.
Same seeds, same datasets, same iteration count. If a number moves, something in the codec moved.
- Mode
- full
- Iterations per measurement
- 10
- OS / arch
- macos / aarch64
- CPU cores
- 10
- Rust
- rustc 1.94.1 (2026-03-25)
- Dataset version
- 1.0.0 (sha256-pinned)
- PRNG
- xorshift64, fixed seeds
- Allocator
- system (jemalloc optional)
- Output dir
- docs/assets/bench/v1.0.1/
- Regressions detected
- none
The Protobuf comparison uses a generic
Value schema so it can represent the same schema-less payloads as Surp and JSON. Numbers shown are median with p95 and coefficient of variation, taken from summary.csv for the committed v1.0.1 run.Datasets
Six shapes that stress different parts of a codec.
small_objects
100,000 records100k flat objects with 6 fields each (id, name, email, active, score, level). Exercises per-record overhead and small-string handling.
string_heavy
10,000 records10k records where ~60% of values are drawn from a 50-string pool. Designed to measure the win from string deduplication.
nested_deep
10-deep tree + 50-deep linearA branching tree (depth 10, fanout 3) plus a 50-level linear chain. Stresses recursion, depth limits, and small-object framing.
binary_blobs
100 × ~64 KiBRecords carrying ~64 KiB random byte payloads. Exposes how each format treats raw bytes (native vs base64).
mixed_api_events
5,000 eventsSynthetic GitHub-event-shaped objects with mixed scalar and nested fields. Models a realistic API payload.
numeric_heavy
dense numericsArrays dominated by integers and floats. Exposes varint efficiency and float framing.
Committed charts · v1.0.1
The same SVGs the harness commits to the repo.
Rendered straight from docs/assets/bench/v1.0.1/charts/. No re-projection, no re-aggregation — exactly what surp-bench wrote.
Chart · size
Serialized size
Chart · encode
Encode throughput
Chart · decode
Decode throughput
Serialized size
Bytes on disk, per dataset.
Surp lands smaller than JSON on every dataset; ratio shown in the right-most column. Surp + Dedup pulls ahead on string-heavy and ties Surp on raw binary.
small_objects
surp/json = 0.82×Surp
8.6 MB
Surp+Dedup
12.0 MB
JSON
10.5 MB
MsgPack
7.8 MB
CBOR
7.9 MB
Protobuf
11.4 MB
string_heavy
surp/json = 0.96×Surp
1.0 MB
Surp+Dedup
668.3 KB
JSON
1.1 MB
MsgPack
925.8 KB
CBOR
927.0 KB
Protobuf
1.2 MB
nested_deep
surp/json = 0.87×Surp
1.0 MB
Surp+Dedup
1.5 MB
JSON
1.2 MB
MsgPack
835.1 KB
CBOR
835.3 KB
Protobuf
1.4 MB
binary_blobs
surp/json = 0.75×Surp
6.4 MB
Surp+Dedup
6.4 MB
JSON
8.5 MB
MsgPack
8.5 MB
CBOR
8.5 MB
Protobuf
6.4 MB
mixed_api_events
surp/json = 0.92×Surp
1.9 MB
Surp+Dedup
2.8 MB
JSON
2.0 MB
MsgPack
1.7 MB
CBOR
1.7 MB
Protobuf
2.2 MB
numeric_heavy
surp/json = 0.63×Surp
3.7 MB
Surp+Dedup
3.7 MB
JSON
6.0 MB
MsgPack
3.5 MB
CBOR
3.5 MB
Protobuf
5.0 MB
Encode throughput
Bytes written per second.
MsgPack is fastest to encode on the smallest payloads; Surp is competitive everywhere and notably wins decode (next chart) by a healthy margin.
small_objects
higher is better · MB/sSurp
505 MB/s · 17.90 ms
Surp + Dedup
174 MB/s · 72.27 ms
JSON
630 MB/s · 17.54 ms
MessagePack
566 MB/s · 14.51 ms
CBOR
524 MB/s · 15.83 ms
Protobuf
467 MB/s · 25.57 ms
string_heavy
higher is better · MB/sSurp
999 MB/s · 1.07 ms
Surp + Dedup
143 MB/s · 4.79 ms
JSON
917 MB/s · 1.21 ms
MessagePack
940 MB/s · 1.01 ms
CBOR
892 MB/s · 1.06 ms
Protobuf
615 MB/s · 1.98 ms
nested_deep
higher is better · MB/sSurp
451 MB/s · 2.42 ms
Surp + Dedup
154 MB/s · 10.52 ms
JSON
517 MB/s · 2.43 ms
MessagePack
380 MB/s · 2.25 ms
CBOR
302 MB/s · 2.84 ms
Protobuf
47 MB/s · 32.35 ms
binary_blobs
higher is better · MB/sSurp
2386 MB/s · 2.80 ms
Surp + Dedup
1896 MB/s · 3.52 ms
JSON
1726 MB/s · 5.16 ms
MessagePack
6290 MB/s · 1.42 ms
CBOR
6550 MB/s · 1.36 ms
Protobuf
35246 MB/s · 189.8 µs
mixed_api_events
higher is better · MB/sSurp
1120 MB/s · 1.74 ms
Surp + Dedup
352 MB/s · 8.26 ms
JSON
1016 MB/s · 2.09 ms
MessagePack
1225 MB/s · 1.50 ms
CBOR
1003 MB/s · 1.83 ms
Protobuf
544 MB/s · 4.22 ms
numeric_heavy
higher is better · MB/sSurp
618 MB/s · 6.35 ms
Surp + Dedup
685 MB/s · 5.72 ms
JSON
744 MB/s · 8.41 ms
MessagePack
747 MB/s · 4.88 ms
CBOR
642 MB/s · 5.65 ms
Protobuf
457 MB/s · 11.41 ms
Decode throughput
Bytes parsed per second.
Surp's decode path is the most consistent across shapes — CBOR loses ground on small_objects, Protobuf rebounds on binary_blobs because it stores bytes natively.
small_objects
higher is better · MB/sSurp
452 MB/s · 19.98 ms
Surp + Dedup
240 MB/s · 52.25 ms
JSON
171 MB/s · 64.72 ms
MessagePack
133 MB/s · 61.63 ms
CBOR
63 MB/s · 131.49 ms
Protobuf
156 MB/s · 76.72 ms
string_heavy
higher is better · MB/sSurp
560 MB/s · 1.91 ms
Surp + Dedup
222 MB/s · 3.08 ms
JSON
215 MB/s · 5.16 ms
MessagePack
214 MB/s · 4.44 ms
CBOR
125 MB/s · 7.59 ms
Protobuf
201 MB/s · 6.05 ms
nested_deep
higher is better · MB/sSurp
230 MB/s · 4.75 ms
Surp + Dedup
201 MB/s · 8.09 ms
JSON
127 MB/s · 9.94 ms
MessagePack
94 MB/s · 9.08 ms
CBOR
58 MB/s · 14.71 ms
Protobuf
137 MB/s · 10.98 ms
binary_blobs
higher is better · MB/sSurp
15547 MB/s · 430.0 µs
Surp + Dedup
15403 MB/s · 433.9 µs
JSON
6466 MB/s · 1.38 ms
MessagePack
19295 MB/s · 461.8 µs
CBOR
9459 MB/s · 942.1 µs
Protobuf
15965 MB/s · 418.9 µs
mixed_api_events
higher is better · MB/sSurp
745 MB/s · 2.62 ms
Surp + Dedup
537 MB/s · 5.41 ms
JSON
270 MB/s · 7.88 ms
MessagePack
272 MB/s · 6.75 ms
CBOR
145 MB/s · 12.64 ms
Protobuf
228 MB/s · 10.06 ms
numeric_heavy
higher is better · MB/sSurp
520 MB/s · 7.55 ms
Surp + Dedup
480 MB/s · 8.16 ms
JSON
246 MB/s · 25.46 ms
MessagePack
192 MB/s · 18.95 ms
CBOR
85 MB/s · 42.49 ms
Protobuf
196 MB/s · 26.67 ms
Full numbers
Every measurement, every format.
Median, p95, coefficient of variation, throughput, and serialized size for all 6 datasets × 6 formats × 3 operations. Mirrored from summary.csv.
| Format | Dataset | Op | Median | p95 | CV% | MB/s | Size |
|---|---|---|---|---|---|---|---|
| surp | small_objects | encode | 17.90 ms | 20.82 ms | 6.9 | 504.6 | 8.6 MB |
| surp | small_objects | decode | 19.98 ms | 22.46 ms | 6.2 | 452.2 | 8.6 MB |
| surp | small_objects | roundtrip | 37.46 ms | 40.96 ms | 5.7 | — | — |
| surp_dedup | small_objects | encode | 72.27 ms | 90.69 ms | 11.8 | 173.6 | 12.0 MB |
| surp_dedup | small_objects | decode | 52.25 ms | 65.38 ms | 12.2 | 240.1 | 12.0 MB |
| json | small_objects | encode | 17.54 ms | 20.01 ms | 6.3 | 629.5 | 10.5 MB |
| json | small_objects | decode | 64.72 ms | 71.76 ms | 4.4 | 170.6 | 10.5 MB |
| json | small_objects | roundtrip | 85.05 ms | 95.08 ms | 5.3 | — | — |
| msgpack | small_objects | encode | 14.51 ms | 14.91 ms | 2.8 | 566.3 | 7.8 MB |
| msgpack | small_objects | decode | 61.63 ms | 85.14 ms | 11.4 | 133.4 | 7.8 MB |
| msgpack | small_objects | roundtrip | 67.95 ms | 77.35 ms | 5.8 | — | — |
| cbor | small_objects | encode | 15.83 ms | 16.33 ms | 1.1 | 523.8 | 7.9 MB |
| cbor | small_objects | decode | 131.49 ms | 196.48 ms | 18.6 | 63.1 | 7.9 MB |
| cbor | small_objects | roundtrip | 132.00 ms | 140.96 ms | 3.3 | — | — |
| protobuf | small_objects | encode | 25.57 ms | 29.56 ms | 6.7 | 466.8 | 11.4 MB |
| protobuf | small_objects | decode | 76.72 ms | 92.80 ms | 7.3 | 155.5 | 11.4 MB |
| protobuf | small_objects | roundtrip | 111.78 ms | 140.83 ms | 11.3 | — | — |
| surp | string_heavy | encode | 1.07 ms | 1.10 ms | 1.0 | 999.4 | 1.0 MB |
| surp | string_heavy | decode | 1.91 ms | 1.94 ms | 0.9 | 559.5 | 1.0 MB |
| surp | string_heavy | roundtrip | 2.98 ms | 3.09 ms | 1.3 | — | — |
| surp_dedup | string_heavy | encode | 4.79 ms | 4.91 ms | 1.3 | 142.9 | 668.3 KB |
| surp_dedup | string_heavy | decode | 3.08 ms | 3.19 ms | 1.5 | 221.9 | 668.3 KB |
| json | string_heavy | encode | 1.21 ms | 1.29 ms | 3.3 | 916.6 | 1.1 MB |
| json | string_heavy | decode | 5.16 ms | 5.40 ms | 1.8 | 214.5 | 1.1 MB |
| json | string_heavy | roundtrip | 6.55 ms | 6.77 ms | 2.2 | — | — |
| msgpack | string_heavy | encode | 1.01 ms | 1.72 ms | 22.5 | 939.5 | 925.8 KB |
| msgpack | string_heavy | decode | 4.44 ms | 4.53 ms | 0.8 | 213.5 | 925.8 KB |
| msgpack | string_heavy | roundtrip | 5.42 ms | 5.50 ms | 0.7 | — | — |
| cbor | string_heavy | encode | 1.06 ms | 1.08 ms | 0.5 | 892.1 | 927.0 KB |
| cbor | string_heavy | decode | 7.59 ms | 7.99 ms | 1.8 | 125.1 | 927.0 KB |
| cbor | string_heavy | roundtrip | 8.76 ms | 8.87 ms | 0.7 | — | — |
| protobuf | string_heavy | encode | 1.98 ms | 1.99 ms | 0.2 | 614.5 | 1.2 MB |
| protobuf | string_heavy | decode | 6.05 ms | 6.07 ms | 0.3 | 201.4 | 1.2 MB |
| protobuf | string_heavy | roundtrip | 8.11 ms | 8.18 ms | 0.8 | — | — |
| surp | nested_deep | encode | 2.42 ms | 5.11 ms | 31.4 | 451.0 | 1.0 MB |
| surp | nested_deep | decode | 4.75 ms | 4.80 ms | 0.4 | 229.9 | 1.0 MB |
| surp | nested_deep | roundtrip | 6.47 ms | 7.16 ms | 4.5 | — | — |
| surp_dedup | nested_deep | encode | 10.52 ms | 10.77 ms | 1.0 | 154.1 | 1.5 MB |
| surp_dedup | nested_deep | decode | 8.09 ms | 9.07 ms | 5.5 | 200.5 | 1.5 MB |
| json | nested_deep | encode | 2.43 ms | 3.28 ms | 13.8 | 516.7 | 1.2 MB |
| json | nested_deep | decode | 9.94 ms | 10.04 ms | 0.7 | 126.6 | 1.2 MB |
| json | nested_deep | roundtrip | 13.02 ms | 14.32 ms | 3.6 | — | — |
| msgpack | nested_deep | encode | 2.25 ms | 2.46 ms | 3.5 | 379.7 | 835.1 KB |
| msgpack | nested_deep | decode | 9.08 ms | 9.21 ms | 1.6 | 94.2 | 835.1 KB |
| msgpack | nested_deep | roundtrip | 11.74 ms | 17.50 ms | 15.3 | — | — |
| cbor | nested_deep | encode | 2.84 ms | 3.08 ms | 5.3 | 301.5 | 835.3 KB |
| cbor | nested_deep | decode | 14.71 ms | 14.83 ms | 0.8 | 58.1 | 835.3 KB |
| cbor | nested_deep | roundtrip | 17.94 ms | 18.48 ms | 1.2 | — | — |
| protobuf | nested_deep | encode | 32.35 ms | 33.53 ms | 3.9 | 46.6 | 1.4 MB |
| protobuf | nested_deep | decode | 10.98 ms | 11.18 ms | 1.2 | 137.2 | 1.4 MB |
| protobuf | nested_deep | roundtrip | 44.35 ms | 45.29 ms | 0.7 | — | — |
| surp | binary_blobs | encode | 2.80 ms | 3.22 ms | 14.6 | 2385.7 | 6.4 MB |
| surp | binary_blobs | decode | 430.0 µs | 432.5 µs | 0.2 | 15547.0 | 6.4 MB |
| surp | binary_blobs | roundtrip | 3.62 ms | 3.66 ms | 2.0 | — | — |
| surp_dedup | binary_blobs | encode | 3.52 ms | 3.64 ms | 7.7 | 1896.2 | 6.4 MB |
| surp_dedup | binary_blobs | decode | 433.9 µs | 436.5 µs | 0.5 | 15402.6 | 6.4 MB |
| json | binary_blobs | encode | 5.16 ms | 5.39 ms | 2.8 | 1725.8 | 8.5 MB |
| json | binary_blobs | decode | 1.38 ms | 1.43 ms | 1.2 | 6466.2 | 8.5 MB |
| json | binary_blobs | roundtrip | 7.06 ms | 9.75 ms | 12.0 | — | — |
| msgpack | binary_blobs | encode | 1.42 ms | 1.48 ms | 3.7 | 6289.9 | 8.5 MB |
| msgpack | binary_blobs | decode | 461.8 µs | 472.6 µs | 0.8 | 19295.3 | 8.5 MB |
| msgpack | binary_blobs | roundtrip | 2.23 ms | 2.54 ms | 4.9 | — | — |
| cbor | binary_blobs | encode | 1.36 ms | 1.49 ms | 5.8 | 6549.7 | 8.5 MB |
| cbor | binary_blobs | decode | 942.1 µs | 982.6 µs | 1.8 | 9458.8 | 8.5 MB |
| cbor | binary_blobs | roundtrip | 2.76 ms | 3.23 ms | 6.4 | — | — |
| protobuf | binary_blobs | encode | 189.8 µs | 194.8 µs | 3.4 | 35245.8 | 6.4 MB |
| protobuf | binary_blobs | decode | 418.9 µs | 463.0 µs | 3.4 | 15964.7 | 6.4 MB |
| protobuf | binary_blobs | roundtrip | 649.1 µs | 661.9 µs | 1.7 | — | — |
| surp | mixed_api_events | encode | 1.74 ms | 1.75 ms | 0.3 | 1120.2 | 1.9 MB |
| surp | mixed_api_events | decode | 2.62 ms | 2.66 ms | 0.8 | 744.5 | 1.9 MB |
| surp | mixed_api_events | roundtrip | 4.37 ms | 4.40 ms | 0.4 | — | — |
| surp_dedup | mixed_api_events | encode | 8.26 ms | 8.35 ms | 0.8 | 352.1 | 2.8 MB |
| surp_dedup | mixed_api_events | decode | 5.41 ms | 5.65 ms | 1.5 | 537.3 | 2.8 MB |
| json | mixed_api_events | encode | 2.09 ms | 2.11 ms | 0.6 | 1016.3 | 2.0 MB |
| json | mixed_api_events | decode | 7.88 ms | 7.94 ms | 0.4 | 269.5 | 2.0 MB |
| json | mixed_api_events | roundtrip | 9.97 ms | 10.09 ms | 0.6 | — | — |
| msgpack | mixed_api_events | encode | 1.50 ms | 1.53 ms | 1.0 | 1225.4 | 1.7 MB |
| msgpack | mixed_api_events | decode | 6.75 ms | 6.79 ms | 0.4 | 271.5 | 1.7 MB |
| msgpack | mixed_api_events | roundtrip | 8.25 ms | 11.49 ms | 11.9 | — | — |
| cbor | mixed_api_events | encode | 1.83 ms | 1.83 ms | 0.2 | 1002.9 | 1.7 MB |
| cbor | mixed_api_events | decode | 12.64 ms | 12.75 ms | 0.4 | 145.0 | 1.7 MB |
| cbor | mixed_api_events | roundtrip | 14.65 ms | 19.25 ms | 11.3 | — | — |
| protobuf | mixed_api_events | encode | 4.22 ms | 4.51 ms | 3.3 | 543.5 | 2.2 MB |
| protobuf | mixed_api_events | decode | 10.06 ms | 10.55 ms | 2.0 | 228.2 | 2.2 MB |
| protobuf | mixed_api_events | roundtrip | 14.47 ms | 14.83 ms | 2.3 | — | — |
| surp | numeric_heavy | encode | 6.35 ms | 6.40 ms | 2.2 | 617.8 | 3.7 MB |
| surp | numeric_heavy | decode | 7.55 ms | 11.36 ms | 15.8 | 519.5 | 3.7 MB |
| surp | numeric_heavy | roundtrip | 14.81 ms | 15.40 ms | 2.4 | — | — |
| surp_dedup | numeric_heavy | encode | 5.72 ms | 6.23 ms | 3.3 | 685.2 | 3.7 MB |
| surp_dedup | numeric_heavy | decode | 8.16 ms | 8.34 ms | 1.2 | 480.2 | 3.7 MB |
| json | numeric_heavy | encode | 8.41 ms | 9.49 ms | 5.0 | 743.8 | 6.0 MB |
| json | numeric_heavy | decode | 25.46 ms | 25.72 ms | 1.1 | 245.8 | 6.0 MB |
| json | numeric_heavy | roundtrip | 34.74 ms | 34.99 ms | 0.8 | — | — |
| msgpack | numeric_heavy | encode | 4.88 ms | 4.92 ms | 0.5 | 746.5 | 3.5 MB |
| msgpack | numeric_heavy | decode | 18.95 ms | 19.10 ms | 0.5 | 192.3 | 3.5 MB |
| msgpack | numeric_heavy | roundtrip | 24.55 ms | 24.70 ms | 0.8 | — | — |
| cbor | numeric_heavy | encode | 5.65 ms | 6.29 ms | 4.9 | 642.2 | 3.5 MB |
| cbor | numeric_heavy | decode | 42.49 ms | 42.87 ms | 4.6 | 85.4 | 3.5 MB |
| cbor | numeric_heavy | roundtrip | 49.61 ms | 51.33 ms | 4.5 | — | — |
| protobuf | numeric_heavy | encode | 11.41 ms | 11.71 ms | 4.6 | 457.3 | 5.0 MB |
| protobuf | numeric_heavy | decode | 26.67 ms | 27.21 ms | 2.7 | 195.7 | 5.0 MB |
| protobuf | numeric_heavy | roundtrip | 38.18 ms | 39.01 ms | 3.5 | — | — |
Reproduce locally
Run it yourself.
All datasets are seeded; numbers will scale to your hardware but the rank order should hold.
# CI fast mode — minutes
cargo run -p surp-bench --release -- \
--mode ci --output bench/results
# Full mode — committed in repo
cargo run -p surp-bench --release -- \
--mode full --output bench/results/full
# Python harness
python bench/python/bench_surp.py