Announcing go-msgpack

Announcing go-msgpack, a rich msgpack codec for Go. Supports encoding/decoding to msgpack binary format, and use for net/rpc communication.

https://github.com/ugorji/go-msgpack
http://gopkgdoc.appspot.com/pkg/github.com/ugorji/go-msgpack

It provides features similar to encoding packages in the standard library (ie json, xml, gob, etc).

Supports:

• Standard Marshal/Unmarshal interface.
• Standard field renaming via tags
• Encoding from any value (struct, slice, map, primitives, pointers, interface{}, etc)
• Decoding into a pointer to any non-nil value (struct, slice, map, int, float32, bool, string, etc)
• Decoding into a nil interface{} (big)
• Handles time.Time transparently.
• Provides a Server and Client Codec so msgpack can be used as communication protocol for net/rpc.

Usage:

  // v can be interface{}, int32, bool, map[string]bool, etc
  dec = msgpack.NewDecoder(r, nil)
  err = dec.Decode(&v)

  enc = msgpack.NewEncoder(w, nil)
  err = enc.Encode(v)

  //methods below are convenience methods over functions above.
  data, err = msgpack.Marshal(v, nil)
  err = msgpack.Unmarshal(data, &v, nil)
  //RPC Communication
  conn, err = net.Dial("tcp", "localhost:5555")
  rpcCodec := msgpack.NewRPCCodec(conn)
  client := rpc.NewClientWithCodec(rpcCodec)
  ...

Why?
I was initially looking at different binary serialization formats for an application I'm developing. I looked at Thrift, Protocol Buffers, BSON, Msgpack, etc.

I finally decided on msgpack:

• compact
• supports basic types (just like JSON. Numbers, Bool, Null, Bytes/String, Map, List) from which other data structures else can be assembled
• raw bytes support (which can represent binary data or strings)
• no schema needed (just like JSON)
• cross-language support
• has pretty good mindshare

Unfortunately, the Go library on the msgpack.org site is old, does not build, and is not "smart" like the encoding/json package. Specifically, it doesn't allow me decode into a typed object (e.g. struct, bool, etc).

I wrote go-msgpack to give the same conveniences we've gotten used to (spoiled by using the encoding/json package), while being really performant.

Summary of my testing:

• Decodes significantly faster (120% faster) and encodes only slightly slower (20% slower) than encoding/json
• Uses less disk space (40% less)
• May not require compression (compression only gave a 10% reduction in size).
  Since compression/decompression time may be significant, this may be an important win.

Hope folks use it and enjoy using it. I know I will. Please feel free to send me feedback.