Columbia Video Network Tool (CoVit) Program Documentation
Subhrendu Sarkar
Columbia University
New York, NY 10027
USA
ss3295@columbia.edu
Abstract
This document describes the different data structures and libraries used by the CoVit software tool.
It also provides an overview of the installation procedure and running of the program.
Software Requirements
The software tool has been developed on a Linux platform and compiled under gcc. It comprises of a set of C++ source and header files.
The software has used the RTP library (RTPlib) for the RTP implementation.
and XML-RPC-C
for the XML-RPC Server. The software also uses the unicap library which provides a uniform interface to video
capture devices. It allows applications to use any supported video capture device via a single API.
Instructions to download and install the library Unicap must be followed as documented.
Instructions to download and install the library Xml-Rpc-C must be followed as documented.
run "make" from the shell prompt to compile CoVit and create the executable.
The make command will also create a sample user program "client" which can be used to interact with the covit tool.
running CoVit:
./covit 8080 (where 8080 is a command line argument to specify the port number at which the XML-RPC Server will be listening.
running client program:
./client http://localhost:8080/RPC2
Program Details
XML-RPC Server
The program for the XML-RPC server is in the file < xmlrpc_server.cpp >
To add a new method support at the server we need to add the method to the xmlrpc registry and also implement a corresponding method for that.
The prototype for adding a new method is shown below:
xmlrpc_registry_add_method(
&env, registryP, NULL, "QueryInterface", &QueryInterface, NULL);
Following are the set of methods currently implemented at the XML-RPC Server.
- QueryInterface (returns the set of supported methods by the server)
- no input arguments
- CreateSendStream (creates a new video send stream with the input identifier, identifier value has to be positive)
- input arguments: video send stream identifier (a positive number)
- CreateRecvStream (creates a new video receive stream with the input identifier, identifier value has to be positive)
- input arguments: video receive stream identifier (a positive number)
- StartStreamingRTP (starts streaming video, outbound)
- input arguments: valid video send stream identifier, IP address, port number.
- StartReceivingRTP (starts receiving video, inbound)
- input arguments: valid video receive stream identifier, IP address, port number.
- setCameraProps (sets Camera Properties)
- input arguments: camera device number, capture width, capture height.
- setEncodingFormats (sets the encoding formats)
- input arguments: valid video send stream identifier, codec type (MPEG4, MPEG2, MPEG1, etc),
encoding width, encoding height, bitrate, frame rate.
- setDecodingFormats (sets the encoding formats)
- input arguments: valid video receive stream identifier, codec type (MPEG4, MPEG2, MPEG1, etc),
decoding width, decoding height.
- StopSend (Stops the RTP Sender and capture threads)
- input arguments: valid video send stream identifier (a positive number)
- StopRecv (Stops the RTP receiver)
- input arguments: valid video send stream identifier (a positive number)
RTP Sender
The RTP sender thread details are mostly embedded in the file < send_rtp.cpp >. "sendrtp" is the thread function.
RTP Receiver
The RTP receiver thread details are mostly embedded in the file < recv_rtp.cpp >. "recvrtp" is the thread function.
Capture Thread
The Capture thread details are mostly embedded in the file < capture.cpp >. "capthread" is the thread function.
Data Stuctures
The program uses a couple of data strcutures and uses an object oriented model.
The objects of importance are as follows:
- VideoCapture (defined in < covit.h > as CVideoCapture) (this is the Object at the RTP Sender layer responsible for RTP sending)
- MediaBuffer (defined in < covit.h > as CMediaBuffer)
- UnicapVidCap (defined in < unicapVidCap.h > as CUnicapVidCap) (this is the Object at the Capture layer responsible for putting buffers
into the Media Buffers)
The encoding format for each video send stream has the following structure:
typedef struct encodeformat
{
CodecID encode_fmt;
int encode_width;
int encode_height;
int encode_bitrate;
int fps;
int gopSize;
};
The decoding format for each video receive stream has the following structure:
typedef struct decodeformat
{
CodecID decode_fmt;
int decode_width;
int decode_height;
};
CoVit maintains a linked list of all the send and receive video stream resources.
There are three such linked lists:
-capList (List for capture video stream resources)
-sendList (List for send video stream resources)
-recvList (List for receive video stream resources)
Generally there will be same number of capture List nodes and send List nodes sharing the same video send stream identifiers.
The capList node is as follows:
typedef struct capNode
{
int id;
pthread_mutex_t capthr_cond_mutex;
pthread_cond_t capthr_cond;
pthread_mutex_t capstop_cond_mutex;
pthread_cond_t capstop_cond;
pthread_mutex_t capstatus_mutex;
int stopCapStatus;
encodeformat encodeFmtNode;
};
The sendList node is as follows:
typedef struct sendNode
{
int id;
pthread_mutex_t sendthr_cond_mutex;
pthread_cond_t sendthr_cond;
pthread_mutex_t sendstop_cond_mutex;
pthread_cond_t sendstop_cond;
pthread_mutex_t sendstatus_mutex;
int send_quit;
};
The recvList node is as follows:
typedef struct recvNode
{
int id;
pthread_mutex_t recvthr_cond_mutex;
pthread_cond_t recvthr_cond;
pthread_mutex_t recvstop_cond_mutex;
pthread_cond_t recvstop_cond;
int recv_quit;
decodeformat decodeFmtNode;
};
VideoCapture implements the IVideoCapture interface defined in "interfaces.h"
The method "Process_Raw_Video" of the above interface is the most important as this method is responsible
for the communication between the Object at the Capture layer and the Object at the RTP Sender Layer.
An object of CUnicapVidCap (ObjVidCap) is aggregated within CVideoCapture.
The declaration of CVideoCapture looks like:
class CVideoCapture : public IVideoCapture
{
protected :
CUnicapVidCap *ObjVidCap;
SDL_Rect rect;
SDL_Surface *m_screen;
int m_stopStatusBase;
pthread_mutex_t *m_capstatus_mutex;
pthread_mutex_t *m_thrstop_mutex;
pthread_cond_t *m_thrstop_cond;
pthread_cond_t *m_MediaBuffer_Empty_cond[30];
pthread_cond_t *m_MediaBuffer_Full_cond[30];
pthread_mutex_t *m_MediaBuffer_mutex[30];
pthread_cond_t *m_delMediaBuffer_Empty_cond;
pthread_cond_t *m_delMediaBuffer_Full_cond;
pthread_mutex_t *m_delMediaBuffer_mutex;
int m_encode_bitrate;
int m_fps;
int m_gopSize;
CodecID m_encode_fmt;
public :
int m_threadid;
CMediaBuffer *m_MediaBuffer;
CVideoEnc *VideoEnc;
CVideoCapture(int threadid);
virtual ~CVideoCapture();
int getStopStatus();
int setStopStatus();
virtual void SetSettings(int device_no);
void Process_Raw_Video(int buffers_ready,int PixFormat, int bufsize,int width, int height,unsigned char *data, int tsinc_usec);
virtual void getVideoFormat();
virtual void setVideoFormat();
virtual void getVideoDeviceProp();
virtual void setMediaBuffer(CCapThreadParams*);
virtual void start_capture(CCapThreadParams *cparam);
virtual int stop_capture();
};
ObjVidCap is an object of type CUnicapVidCap aggregated within the CVideoCapture. This object acts as the interface with the capture device and
internally it uses the unicap imaging library to interact with the capture devices.
The declaration of CUnicapVidCap is as follows:
class CUnicapVidCap
{
public :
CUnicapVidCap(IVideoCapture *, int streamid);
virtual ~CUnicapVidCap();
void open_device(int device_no);
int enumerate_devices();
void setformats();
void start_capture(int *stop_status, int fps);
void stop_capture();
pthread_mutex_t *m_capstatus_mutex;
pthread_mutex_t *m_thrstop_mutex;
pthread_cond_t *m_thrstop_cond;
int m_threadid;
private :
int m_device_no;
unicap_handle_t m_handle;
unicap_device_t m_device;
unicap_format_t m_format_spec;
unicap_format_t m_format;
unicap_data_buffer_t m_buffers[BUFFERS];
unicap_data_buffer_t *m_returned_buffer;
int m_PixFmt;
CCameraDevice *m_Cam;
IVideoCapture *BaseVidCapture;
};
The XML RPC Server needs to configure the capture devices present in a system. CCameraDevice is a singleton object, hence only one instance
of CCameraDevice exists in CoVit. Each instance of CUnicapVidCap contains a pointer to that one instance of CCameraDevice object. This object is
responsible for initializing the capture devices, creating capture device handles,
set formats on the capture devices and maintaining the reference count for each of the capture devices. The object also maintains a reference
count for itself so that it can delete the object when the object is no longer referenced and used.
class CCameraDevice
{
public:
static CCameraDevice* Instance();
CCameraDevice();
virtual ~CCameraDevice();
int getNumberofCams();
int getCamHandle(unicap_handle_t *handle, int device_no = 0);
int closeCamHandle(int device_no);
void destroy();
int setformats(int device_no, unicap_format_t *format);
int start_capture(int device_no);
void stop_capture();
int initCamera();
void deinitCams();
static CCameraDevice* m_CameraInstance;
int m_refCount;
private:
int m_NumCams;
unicap_data_buffer_t **m_buffers;
int *m_start_capture;
int *m_hndRefCount;
unicap_handle_t *m_handle;
unicap_device_t *m_device;
unicap_format_t *m_format_spec;
unicap_format_t *m_format;
};
Program Flow
The following diagram (Figure 5) tries to explain the flow of the program for sending video.
Figure 5: The Program Flow
Client Program
The source code for the client program can be found at < xmlrpc_sample_client.cpp >. The client program is an user interactive program which waits for user input. Depending on the
method name provided by the user in the input, the program asks for the input arguments from the user in an interactive manner. The client program aggregates all the input arguments
neccessary for a method and then makes the XML RPC method call on the listening XML RPC server.
To make a remote procedure call we can look at the following method:
/* Make the remote procedure call */
result = xmlrpc_client_call(&env, serverUrl, methodName,
"(i)", (xmlrpc_int32) 0);
serverUrl is a string e.g "http://localhost:8080/RPC2".
methodName is a string e.g "QueryInterface".