Ain't SIP is an effort to create a publicly available automated test infrastructure to identify, test and document SIP interoperability issues. This project follows an earlier study at the IRT Lab using Columbia VoIP testbed that identified a host of interoperability issues, even at the basic level of SIP usage . Ain't SIP has three significant characteristics that make it different from earlier attempts at SIP interoperability testing.
1. Semi-automated Testing
The primary goal is to provide an automated way of testing SIP devices against known interoperability issues. The testing system cannot be fully automated, because most of the test-cases require human intervention as it interacts with the testing system (for carrying out actions on the testing device e.g., registering, making a call, verifying if audio is being heard etc). But on the other hand, no dedicated support personnel can be expected to be available at the Columbia VoIP infrastructure, during any of the tests - so the tests have to be semi-automatic.
2. Remote Accessibility through a web interface
Anyone interested in testing their SIP devices should be able to remotely connect to the system, initiate a test, perform the required actions and find out the results. The system should provide a web interface to the testing mechanism, so that accessibility is independent of the OS platforms. Also, basic security mechanisms are required to prevent abuse of the system/resources.
3. Extensible Framework
Not all interoperability issues can be discovered by just one person or one laboratory. So, we envision this to be a shared platform where the community members report new interoperability issues, develop tests that can be used by others and can publish where they stand with respect to the existing repository of interoperability tests. Of significance, is the ease with which new interoperability tests could be plugged in, and its independence from the languages/tools of test-case creation.
The following diagram shows the control and data flow in the Ain't SIP system. On the testing side, a SIP device and a human operator with web connectivity are present, whereas on the testbed side, the Ain't SIP server with test repository exists. The connectivity between the two ends is through the Ain't SIP web interface.