Internet Engineering Task Force Greg Herlein Internet Draft Jean-Marc Valin draft-herlein-speex-rtp-profile-00 Simon Morlat February, 2002 Expires: July, 2003 RTP Payload Format for the Speex Codec Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress". The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt To view the list Internet-Draft Shadow Directories, see http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The Internet Society (2002). All Rights Reserved. Abstract Speex is an open-source, patent-free voice codec suitable for use in Voice over IP (VoIP) type applications. The Speex codec supports three modes of operation: narrowband at a nominal 8kHz sample rate, wideband at a nominal 16kHz sample rate, and ultra-wideband at a nominal 32kHz sample rate. Speex supports Voice Activity Detection (VAD) and Variable Bit Rate (VBR). This document describes the payload format for Speex generated bit streams within an RTP packet. Also included here are the necessary details for the use of Speex with the Session Description Protocol (SDP) [4] and a preliminary method of using Speex within H.323 applications. Use of Speex with MIME will be covered as part of the Ogg Vorbis MIME definitions and is covered only minimally here. Herlein, Valin, etc [Page 1] ^L Internet-Draft RTP Payload Format for the Speex Codec Nov 2002 1. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [5]. 2. Overview of the Speex Codec Speex is based on the CELP encoding technique with support for either narrowband (nominal 8kHz), wideband (nominal 16kHz) or ultra-wideband (nominal 32kHz) sampling. The main characteristics can be summerized as follows: o Free software/open-source, royalty-free o Integration of wideband and narrowband in the same bit-stream o Wide range of bit-rates available o Dynamic bit-rate switching and variable bit-rate (VBR) o Voice Activity Detection (VAD, integrated with VBR) o Variable complexity 3. RTP payload format for Speex Speex uses 20 ms frames and a variable sampling rate clock. The RTP timestamp MUST be in units of 1/X of a second where X is the sample rate used. Speex uses a nominal 8kHz sampling rate for narrowband use, a nominal 16kHz sampling rate for wideband use, and a nominal 32kHz sampling rate for ultra-wideband use. The RTP payload for Speex has the format shown in Figure 1. No additional header specific to this payload format is required. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header [2] | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | | + one or more frames of Speex | | .... |p| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1: RTP payload for Speex The encoding and decoding algorithm can change the bit rate at any 20ms frame boundary but the bit rate change notification is provided in-band with the bit stream. Each frame contains both "mode" (narrowband,wideband or ultra-wideband) and "sub-mode" (bit-rate) information in the bit stream. No out-of-band notification is required for the decoder to process changes in the bit rate sent by the encoder. Herlein, Valin, etc [Page 2] ^L Internet-Draft RTP Payload Format for the Speex Codec Nov 2002 For the purposes of packetizing the bit stream in RTP, it is only necessary to consider the sequence of bits as output by the Speex encoder, and present the same sequence to the decoder. The payload format described here maintains this sequence. An RTP packet MAY contain Speex frames of the same bit rate or of varying bit rates, since the bit-rate for a frame is conveyed in band with the signal. It is RECOMMENDED that values of 8000 or 16000 be used for normal internet telephony applications, though the sample rate is supported at rates as low as 6000 Hz and as high as 32 kHz. The RTP payload MUST be padded to provide an integer number of octets as the payload length. These padding bits MUST be all zero. This padding is only required for the last frame in the packet, and only to ensure the packet contents ends on an octet boundary. 3.1 RTP Payload Type Codes The RTP Audio-Visual Working Group will no longer issue static payload type codes for RTP (beyond those already assigned). Dynamic payload type codes MUST be negotiated 'out-of-band' for the assignment of a dynamic payload type from the range of 96-127. Examples of this are shown in the section discussing the Session Description Protocol (SDP) below. 3.2 Multiple Speex frames in a RTP packet By default only one Speex frame is permitted in a single RTP packet. When operating with multiple frames per packet then the end points MUST use out-of-band negotiation to determine the number of frames per packet. See section 5 below for an example of how to do this with SDP [4]. 3.3 Computing the number of Speex frames If using SDP [4] (see section 5 below for an example) this can be done using the "ptime" variable to denote the packetization interval (ie, how many milliseconds of audio is encoded in a single RTP packet). Since Speex uses 20ms frames, ptime values of multiples of 20 denote multiple Speex frames per packet. Values of ptime in other than multiples of 20 SHOULD be ignored and SHOULD use the default value of one instead. 4. MIME registration of Speex Full definition of the MIME type for Speex will be part of the Ogg Vorbis MIME type definition application. MIME media type name: audio MIME subtype: speex Required parameters: to be included in the Ogg MIME specification. Herlein, Valin, etc [Page 3] ^L Internet-Draft RTP Payload Format for the Speex Codec Nov 2002 Optional parameters: Encoding considerations: Security Considerations: See Section 6 of RFC 3047. Interoperability considerations: none Published specification: Applications which use this media type: Additional information: none Person & email address to contact for further information: Greg Herlein Jean-Marc Valin Intended usage: COMMON Author/Change controller: Author: Greg Herlein Change controller: Greg Herlein Herlein, Valin, etc [Page 4] ^L Internet-Draft RTP Payload Format for the Speex Codec Nov 2002 5. SDP usage of Speex When conveying information by SDP [4], the encoding name SHALL be "speex". An example of the media representation in SDP for offering a single channel of Speex at 8000 samples per second might be: m=audio 8088 RTP/AVP 97 a=rtpmap:97 speex/8000 Note that the RTP payload type code of 97 is defined in this media definition to be 'mapped' to the speex codec at an 8kHz sampling frequency using the 'a=rtpmap' line. Any number from 96 to 127 could have been chosen (the allowed range for dynamic types). The value of the sampling frequency is typically 8000 for narrow band operation, 16000 for wide band operation, and 32000 for ultra-wide band operation. If for some reason the offerer has bandwith limitations, he may use the "b=" header, as explained in SDP [4]. The following example illustrates the case where the offerer cannot receive more than 10 kbit/s. m=audio 8088 RTP/AVP 97 b=AS:10 a=rtmap:97 speex/8000 In this case, if the remote part agrees, it should configure its speex encoder so that it does not use modes that produce more than 10 kbit/s. Note that the "b=" constraint also applies on all payload types that may be proposed in the media line ("m="). An other way to make recommendations to the remote speex encoder is to use its specific parameters via the a=fmtp: directive. The following parameters are defined for use in this way: ptime: duration of each packet in milliseconds. sr: actual sample rate in Hz. ebw: encoding bandwidth - either 'narrow' or 'wide' or 'ultra' (corresponds to nominal 8000, 16000, and 32000 Hz sampling rates). vbr: variable bit rate - either 'on' 'off' or 'vad' (defaults to off). If on, variable bit rate is enabled. If off, disabled. If set to 'vad' then constant bit rate is used but silence will be encoded with special short frames to indicate a lack of voice for that period. cng: comfort noise generation - either 'on' or 'off'. If off then silence frames will be silent; if 'on' then those frames will be filled with comfort noise. mode: speex encoding mode. Can be {1,2,3,4,5,6,any} defaults to 3 in narrowband, 6 in wide and ultra-wide. penh: use of perceptual enhancement. 1 indicates to the decoder that perceptual enhancement is recommended, 0 indicates that it is not. Defaults to on (1). Herlein, Valin, etc [Page 5] ^L Internet-Draft RTP Payload Format for the Speex Codec Nov 2002 Examples: m=audio 8008 RTP/AVP 97 a=rtpmap:97 speex/8000 a=fmtp:97 mode=4 This examples illustrate an offerer that wishes to receive a speex stream at 8000Hz, but only using speex mode 3. The offerer may suggest to the remote decoder to activate its perceptual enhancement filter like this: m=audio 8088 RTP/AVP 97 a=rtmap:97 speex/8000 a=fmtp:97 penh=1 Several speex specific parameters can be given in a single a=fmtp line provided that they are separated by a semi-colon: a=fmtp:97 mode=any;penh=1 The offerer may indicate that it wishes to send variable bit rate frames with comfort noise: m=audio 8088 RTP/AVP 97 a=rtmap:97 speex/8000 a=fmtp:97 vbr=on;cng=on The use of a particular packetization interval may be suggested to the remote encoder using the ptime parameter: m=audio 8008 RTP/AVP 97 a=rtpmap:97 speex/8000 a=ptime:40 Note that the ptime parameter applies to all payloads listed in the media line and is not used as part of an a=fmtp directive. Speex can encode frames of 20 ms. Values of ptime not multiple of 20 ms are meaningless, so the receiver of such ptime values SHOULD ignore them. Herlein, Valin, etc [Page 6] ^L Internet-Draft RTP Payload Format for the Speex Codec Nov 2002 6. ITU H.323/H.245 Use of Speex Application is underway to make Speex a standard ITU codec. However, until that is finalized, Speex MAY be used in H.323 [6] by using a non-standard codec block definition in the H.245 [7] codec capability negotiations. 6.1 NonStandardMessage format For Speex use in H.245 [7] based systems, the fields in the NonStandardMessage should be: t35CountryCode = Hex: B5 t35Extension = Hex: 00 manufacturerCode = Hex: 0026 [Length of the Binary Sequence (8 bit number)] [Binary Sequence consisting of an ASCII string, no NULL terminator] The binary sequence is an ascii string merely for ease of use. The string is not null terminated. The format of this string is speex [optional variables] The optional variables are identical to those used for the SDP a=fmtp strings discussed in section 5 above. The string is built to be all on one line, each key-value pair seperated by a semi-colon. The optional variables MAY be ommited, which causes the default values to be assumed. They are: ebw=narrow;mode=3;vbr=off;cng=off;ptime=20;sr=8000;penh=no; The fifth byte of the block is the length of the binary sequence. NOTE: this method can result in the advertising of a large number of Speex 'codecs' based on the number of variables possible. For most VoIP applications, use of the defailt binary sequence of 'speex' is RECOMMENDED to be used in addition to all other options. This maximizes the chances that two H.323 based applications that support Speex can find a mutual codec. 6.2 RTP Payload Types Dynamic payload type codes MUST be negotiated 'out-of-band' for the assignment of a dynamic payload type from the range of 96-127. H.323 applications MUST use the H.245 H2250LogicalChannelParameters encoding to accomplish this. 7. Security Considerations RTP packets using the payload format defined in this specification are subject to the security considerations discussed in the RTP specification [2], and any appropriate RTP profile. This implies that confidentiality of the media streams is achieved by encryption. Because the data compression used with this payload format is applied end-to-end, encryption may be performed after compression so there is no conflict between the two operations. A potential denial-of-service threat exists for data encodings using compression techniques that have non-uniform receiver-end computational load. The attacker can inject pathological datagrams into the stream which are complex to decode and cause the receiver to be overloaded. However, this encoding does not exhibit any significant non-uniformity. As with any IP-based protocol, in some circumstances a receiver may be overloaded simply by the receipt of too many packets, either desired or undesired. Network-layer authentication may be used to discard packets from undesired sources, but the processing cost of the authentication itself may be too high. 8. References 1. Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. 2. Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson, "RTP: A Transport Protocol for real-time applications", RFC 1889, January 1996. (Updated by a Work in Progress.) 3. Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996. 4. Handley, M. and V. Jacobson, "SDP: Session Description Protocol", RFC 2327, April 1998. 5. Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 6. ITU-T Recommendation H.323. "Packet-based Multimedia Communications Systems," 1998. 7. ITU-T Recommendation H.245 (1998), "Control of communications between Visual Telephone Systems and Terminal Equipment". 9. Acknowledgments The authors would like to thank Equivalence Pty Ltd of Australia for their assistance in attempting to standardize the use of Speex in H.323 applications, and for implementing Speex in their open source OpenH323 stack. The authors would also like to thank Brian C. Wiles of StreamComm for his assistance in developing the proposed standard for Speex use in H.323 applications. 10. Author's Address Greg Herlein 2034 Filbert Street San Francisco, CA United States 94123 Jean-Marc Valin Department of electrical and computer engineering University of Sherbrooke 2500 blvd Universit‰ Sherbrooke, Quebec, Canada, J1K 2R1 Simon MORLAT 35, av de Vizille App 42 38000 GRENOBLE FRANCE Roger Hardiman 49 Nettleton Road Cheltenham Gloucestershire GL51 6NR England Herlein, Valin, etc [Page 7] ^L Internet-Draft RTP Payload Format for the Speex Codec Nov 2002 10. Full Copyright Statement Copyright (C) The Internet Society (2001). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. Herlein, Valin, etc [Page 8] ^L