Load Management for Internet Services

German Goldszmidt
IBM Research



As Internet traffic increases, many Web sites are often unable to serve their workload, particularly during peak activity periods. In this presentation, I will describe some of the techniques that we used to improve the scalability and performance of large Web sites. NetDispatcher is a software tool that ``routes'' TCP connections to multiple servers that share their workload. It is implemented as a stack extension and a user level management process that controls the allocation of requests among servers according to their real-time load and responsiveness. NetDispatcher forwards packets without performing any TCP/IP header translations. Outgoing server-to-client packets are not handled, and can follow a separate network route to the clients. This allocation method was proven to be efficient in live tests, supporting some of the busiest Internet sites, and, in particular, the 1996 Olympic Games Web site. This site had more than 60 servers in 4 countries, and registered over 192 million requests (including multimedia content) during the games. I will briefly describe some of the challenges of building, operating, securing, and managing such a large distributed system.

Luis Gravano