Comparing sizes of protobuf vs json

Google Protobuffer is a binary format claiming to much more compact than json and other text-formats, but just how much less space does it require? Does it hold for large arrays of data?

In this blogpost I will compare the sizes of the two formats.

Test-data with array of tickers

I will generate test-data with a home made tool, you can find it on github: https://github.com/nilsmagnus/protobuf-json-xml-size-comparison

The content of the data is defined in a proto-message:

syntax = "proto3";

package sample;

message Test {
  string query = 1;
  int32 page_number = 2;
  int32 result_per_page = 3;
  repeated Ticker tickers = 4;
}

message Ticker {
  string name = 1;
  float value = 2;
}

There is 1 string, 2 ints and an array of possibly unlimited array of tickers.

The ticker-name is a random string of size 3 and the value is a random float value between 0.0 and 9.99. In json could looks like this with 2 tickers in the array:

{
  "query": "myQuery",
  "page_number": 42,
  "result_per_page": 100,
  "tickers": [
    {
      "name": "rPs",
      "value": 9.768923
    },
    {
      "name": "WEo",
      "value": 6.067048
    }
  ]
}

The numbers

Comparing the raw-size only of json would be unfair, since it is usually gzipped before transfer between client/servers.

However, take into account that when gzipping content you are using additional cpu to zip the content which is not needed when using protobuf.

The size-numbers are in bytes.

Raw json

Protobuf is clearly the winner for all sizes of the ticker-list, but is best when the ticker-list is smaller.

gzipped json and gzipped protobuf

The common case for transferring json-messages is to gzip them first, so lets see how that affects our numbers.

Protobuf clearly wins on the smaller map-sizes, but loses its clear advantage when the ticker-list grows in size.

Protobuf is still the winner on all sizes.

Conclusion

Protocol buffers is a clear winner for small messages where the protobuf size is as small as 16% of the gzipped json size.

However, when large arrays of data is transferred, gzipped protobuf is still smaller but seems to lose its clear advantage in message size to gzipped json.