MUS Serialization Format

The MUS (Marshal, Unmarshal, Size) format will not take you long to figure it out. Its specification defines encoding rules for basic data types (byte, uint, int, float, bool, string, list, map, pointer) and the DTM definition. That’s all. “Keep it small and simple!” — as one well-known design principle suggests.

Key Characteristics

The format uses almost no metadata, resulting in compact encoding.

DTM (Data Type Metadata)

DTM acts as a prefix that indicates the type of the subsequent data.

DTM + data

Structural Data Types

Fields of structural data types are encoded sequentially: first, second, third, and so on. Decoding follows the same logic, with the types and order of fields determined by the type of the encoded structure.

Data Versioning

The MUS format suggests not to change the data (like adding or deleting a field) but to create a new version of it + use DTM to distinguish one version from another. DTM can simultaneously define both the data type and its version:

const FooV1DTM = 1 // type Foo, version V1

For example, with this approach, decoding of FooV1 and FooV2 (current version), can be done like:

type Foo FooV2 // Foo denotes the current version.

func DecodeFoo(...) (Foo, error) {
  // 1. Decode DTM.
  // 2. Use DTM to distinguish one version from another:
  //    - For FooV1: Decode, migrate to FooV2, and return the result.
  //    - For FooV2: Simply decode and return as the current version.
  ...
}

Such functionality can be encapsulated in a separate package or module, so the rest of our code won’t even be aware of the different versions — only Foo will be used everywhere.

Serializers

The MUS serializer and DTM support module are currently implemented only for Golang. The serializer, by the way, is quite fast and simple at the same time.

Streaming Support

The MUS format is also ideal for streaming — there’s no need to encode the data length beforehand, all that’s required on the receiving end is the data type.