Textual vs. Binary Protocols

There are roughly these classes of protocol encodings:

• textual, with an RFC-822-like syntax, as in HTTP, SMTP, SIP, RTSP;
• textual, with an XML-like syntax, such as HTML, SMIL or the TIPHON telephony charging proposal.
• binary, with a fixed structure more-or-less directly mappable into C struct, such as the (fixed) headers for UDP, TCP, RTP;
• binary, with a flat list of attribute-value pairs, as in Radius, DHCP, Diameter, BGP, or the RTCP items.
• a fully specified binary syntax, specified for example through a description language such as IDL, XDR or ASN.1, and implemented through compilers, as used in X.500, X.500, SNMP (all ASN.1 BER), Sun RPC (XDR), H.323 (ASN.1 PER).

Cost of development:

ASN.1 PER and BER require the use of compilers or very lengthy development periods due to the variety of input formats that are possible. Unfortunately, there are no freely available ASN.1 compilers that understand or generate PER.

Ease of use:

Languages such as Java, VisualBasic, Tcl, Python and Perl are designed to operate on text rather than binary data. They often have difficulties with the bit-shifting and masking operations required, for example, for decoding ASN.1. (No PER or BER decoder written in these languages is known to the author, but servers, clients and test tools for SMTP, HTTP and SIP written in all of the languages mentioned above are common.)

Use of textual protocols also makes APIs like cgi-bin much easier, since the server does not have to understand every single part of the request and transcode it through an API.

Ease of debugging:

Without special tools, it is much easier to debug applications. For example, one can telnet to the server port and type in the request if things don't work. This is still a very common mode of operation for debugging email and web servers, for example. Text codings also tend to be more robust in that they often recover from coding mistakes. A single off-by-one data item length in a binary encoding will usually render the whole message unintelligible.

Self-describing:

Binary protocols often are not-describing, so that data type and description are not obvious without access to the specification. This makes upgrades and coordination more difficult. People are less likely to concurrently re-use a name, for example, when the next option number. (UUIDs can solve this problem, but at space expense greater than textual description.)

Extensible:

If protocol fields are orthogonal and mutually independent, text protocols may it easy to upgrade protocols, while not breaking existing implementations. Despite some limitations, HTML, HTTP and SMTP have shown this to be the case.

Architecture-independence:

Text protocols largely avoid issues of floating point formats and byte ordering (little vs. big-endian).

• Protocol messages encoded in binary protocols can be smaller, particularly if dominated by lots of binary information. However, if most of the information is text, including DNS names, URLs and similar identifiers, the advantage tends to be smaller.
• Text-based protocols are also a bad fit if much of the information is opaque binary information. (Some text protocols such as HTTP, RTSP, SIP and SMTP allow a single large ``blob'' of binary data as part of the message.)
• RFC-822-style protocols do not work well with deeply nested structures. They work best if the header consists of at most two levels of nesting, namely the header name and possibly ;-designated header parameters. Attempts at deeper nesting, such as in HTTP content negotiation, tend to get ugly.

Binary protocols cause some operational problems, or example due to less-than-perfect compilers, and also because any change in the specification can cause non-interoperability. Look at the lengthy debugging required for X.400, X.500 or more recently H.323 for evidence. The packed encoding rules in fact exacerbate the problem, because they remove all the self describing aspect of the ASN.1 BER/DER encodings.

A classic implementation trick in text-based protocols is to just precompile a set of protocol elements. For example, you can have an SDP profile where the only element that ever changes from connection to connection is the port number -- you will easily store a text string and just insert the port number. You can do some amount of that with single level attribute-value pair protocols, but it is more difficult -- you have to use partial precompilation, update the length fields, etc. Now, if you try to do this kind of things with ASN.1/BER you get interesting effects where by changing the length of one element you also change the length of enveloping elements, etc.

The other classic text advantage is partial decompilation. If a program module is only interested in one parameter, it just goes fetch the specified line, and look at the specified variables. Again, this is something you can do, to a degree, with a single level attribute-value pair protocol, but it is something that is essentially hard with complex "compiler-based" protocol. Basically, the compiler will have to decompile the whole message, which you will access through some local memory representation. The cost can in fact be quite large, because compilers generally perform memory allocation, etc.

As a data point, the sipe 'embedded' SIP/SDP UA parser for a basic SIP audio call with the required parameters. Parsing a request takes about 120 microseconds on a Solaris Ultra 1/170. The SIP and SDP parser consumes about 30 kB of object memory.

Protocol design often has a larger influence than the encoding on protocol space efficiency. For example, a standard H.323 setup using NetMeeting produces roughly 1,000 bytes of payload, i.e., without counting TCP headers. A roughly equivalent set of SIP messages (INVITE, 200, ACK), with full rather than abbreviated SIP headers, needs less than that.

This is based on discussions on the navdec mailing list, with contributions by Christian Huitema and others.

Last updated by Henning Schulzrinne