Columbia VoIP Testbed

Overview

Architecture

SIP End Clients

Interoperability

Home | Issues 1 2 3 4 5 6 | Robustness Tests |

Robustness Tests

We evaluated the performances of SIP end-clients against two of the widely used robustness tests. The SIP Torture Test Messages (RFC 4475) contains a set of 49 SIP messages, developed and refined at SIPit events. These tests primarily focus on areas that have caused interoperability problems or that have particularly unfavorable characteristics if handled improperly. The second was the PROTOS SIP Test Suite developed by the University of Oulu, which contains more than 4500 SIP INVITE requests. Both these test suites are designed to exercise and torture the SIP parser and the application above the SIP implementation.

Test Setup

1. All the soft SIP end-clients are run on an Intel 3GHz CPU with 1GB RAM, running Windows XP.

2. All SIP messages are relayed between UAC and UAS directly, without involving any SIP proxy.

Results

SIP end-clients	Failed test-cases	Behavior
WengoPhone	General	CPU consumption is very high (varies between 50% and 90%), throughout the testing
SIPc	General	Sends response after about 3s, for most of the INVITEs
	PROTOS: 368, 369	Software crashes (with a message “Hostname variable too long”)
ExpressTalk	General	Software crashes, if an INVITE is received, when all the 6 lines are busy.
	PROTOS: 402, 675, 1243, 1260-1263, 1748-1753	Sends back malformed “180 Ringing” responses.
	PROTOS: 2363, 2366, 2371, 2374, 2383 – 2406	Software crashes. It can be concluded that this end-client is unable to handle improper values for “Content-length” field.

References

1. PROTOS Test-suite for SIP (http://www.ee.oulu.fi/research/ouspg/protos/testing/c07/sip/index.html)

2. SIP Torture Test Messages (http://www.ietf.org/rfc/rfc4475.txt)