Release Version: 1.0.2
Date: 1/19/2006
- Introduction
- Dots_Apache
- External Components for DotSlash
- DotSlash Configurations
- Running Apache with DotSlash
- Dots_PHP
- Dots_MySQL
- Summary
Web hotspots, also known as flash crowds or the Slashdot effect, are
short-term dramatic load spikes that can seriously degrade the
service quality of affected web sites. In order to handle
web hotspots effectively, we developed DotSlash, an automated web
hotspot rescue system. The main advantage of using DotSlash is
that it can operate in a fully automated way without human intervention.
As a result, DotSlash-enabled web sites can react quickly to load
spikes and improve their availability during critical periods of their
"15 minutes of fame".
This is a prototype implementation of
DotSlash
for the widely used LAMP (Linux, Apache, MySQL, and PHP) configuration.
This software release includes three components: Dots_Apache,
Dots_PHP, and Dots_MySQL. Their source files are under
three separate directories: dots_apache, dots_php, and
dots_mysql.
- Dots_Apache is an Apache module that supports basic
DotSlash functions including workload monitoring, rescue server
discovery, rescue relationship management, request redirection,
dynamic virtual hosting, and dynamic DNS update.
- Dots_PHP is an extension for the PHP module of Apache
that supports replicating PHP scripts dynamically.
- Dots_MySQL is a cache-enhanced PHP data driver
for MySQL databases that supports caching database query results on demand.
To use DotSlash, Dots_Apache is required but Dots_PHP and
Dots_MySQL are optional. Thus, there are three configurations
in using DotSlash as shown in Table 1.
Table 1. Three configurations in using DotSlash
| Configuration | Bottlenecks Addressed | Used By
|
|---|
| Dots_Apache
| Network and web server
| Static/dynamic content web sites
| Dots_Apache + Dots_PHP
| Network, web server, and application server
| Dynamic content web sites
| Dots_Apache + Dots_PHP + Dots_MySQL
| Network, web server, application server, and database server
| Dynamic content web sites
| | |
In addition, DotSlash needs two external components, dynamic DNS and mSLP DA.
DotSlash uses dynamic DNS to add new aliases for HTTP redirects and to
update IP addresses for DNS round robin. DotSlash uses
mSLP DAs
to serve as its service registries.
Dots_Apache is an Apache module that supports basic
DotSlash functions including workload monitoring, rescue server
discovery, rescue relationship management, request redirection,
dynamic virtual hosting, and dynamic DNS update.
This is the only required component for using DotSlash.
- mod_dots.c:
DotSlash module for Apache, supporting functions
related to client request processing such as accounting for each
response, HTTP redirect, and dynamic virtual hosting
- dotsd.c:
DotSlash daemon for rescue relationship management,
which also invokes SLP client and DNS client
- slpc.c:
SLP client library for performing service registrations
and service queries
- dnsc.c:
DNS client library for performing dynamic DNS updates,
either TSIG-signed or non-signed
- util.c: a few basic routines
- mod_dots.h:
data structure and constant definitions
- dots_ddns: shell script
for performing TSIG-signed dynamic DNS updates using nsupdate
(the BIND tool for dynamic DNS updates)
- httpd.conf.dots:
sample DotSlash configurations using the Apache configuration file httpd.conf
- _dots_404.php.txt:
"File not found" handler for PHP scripts, which invokes Dots_PHP to
replicate missing PHP scripts dynamically
- Red Hat 9.0 or Red Hat Enterprise Linux AS v.3
- Apache 2.0.49
- BIND 9.2.2
- mSLP DA
- Compiling and installing Dots_Apache (go to directory
dots_apache)
- ./configure --with-apache-install=${apache_install} --with-apache-source=${apache_source} --with-bind-install=${bind_install}
where ${apache_install} is the root directory of your Apache installation,
${apache_source} is the root directory of your Apache source code,
and ${bind_install} is the root directory of your BIND installation.
- make
which will compile the source code, and install mod_dots.so under
${apache_install}/modules, install dots_ddns under
${apache_install}/bin, and install _dots_404.php.txt as
${apache_install}/htdocs/_dots_404.php.
DotSlash needs two external components, dynamic DNS and mSLP DA.
DotSlash uses dynamic DNS to add new aliases for HTTP redirects and to
update IP addresses for DNS round robin. DotSlash uses mSLP DAs to serve as
its service registries.
- Dynamic DNS is needed to accept dynamic DNS updates
for aliases and IP addresses. DotSlash supports TSIG-signed
and non-signed dynamic DNS updates.
- You need to register new aliases for performing HTTP redirects
(required). Aliases are constructed from an alias base name
sequentially. For example, if you use foo.example.com as your alias
base name, then all your aliases are in the form of
vh<seqno>.foo.example.com, where <seqno> is
a sequence number.
- You may enable DNS round robin (optional). To enable DNS round
robin, you need to configure your own DNS server to accept your dynamic
DNS updates for IP addresses.
- If you only need to register aliases dynamically, you may use our
dot-slash.net domain, which only accepts TSIG-signed dynamic DNS
updates. To register aliases in our dot-slash.net domain,
you need to obtain a key, a secret, and a unique alias base
name such as foo.dot-slash.net.
- mSLP DAs are needed to accept service registrations
and to answer service queries.
- To use DotSlash, you need to join a mutual-aid community. In other
words, you need to have a list of other web sites that can serve as your
rescue servers. At the same time, you should be prepared to serve
as a rescue server for other web sites.
- DotSlash uses
mSLP DAs
to provide directory services to form a mutual-aid community.
- For security reason, you should set up your own mSLP DA, and share
this DA with other web sites in your mutual-aid community.
- For testing purpose, you may use the mSLP DA in our dot-slash.net
domain, which can be discovered automatically via DNS SRV by this
implementation.
You can configure DotSlash via the Apache configuration file
httpd.conf using the following directives. Only configurations
marked as "required" must be specified. Most configurations are optional,
and they will take their default values if unspecified,
- LoadModule dots_module modules/mod_dots.so
[required]: load the DotSlash module dynamically, which is named as
dots_module and is located at modules/mod_dots.so.
This configuration is added automatically to the default
Apache configuration file ${apache_install}/conf/httpd.conf
by this implementation. If you use a non-default Apache configuration
file, please use this configuration to active the DotSlash module.
- <Location /dotslash-status>
SetHandler dotslash-status
<Location>
[optional]: obtain the DotSlash status via /dotslash-status
- DotsDotsdPidFile ${apache_install}/logs/dotsd.pid
[optional]: set the process ID file (absolute path) for DotSlash
daemon, default to ${apache_install}/logs/dotsd.pid.
- DotsControlInterval 1
[optional]: set the interval for performing control adjustments
(in second), default to 1 second
- DotsMaxDataRate 1000
[optional]: set the maximum data rate (in kB/second) for outbound HTTP
traffic, default to 1000 kB/second.
- DotsLinkUpperThreshold 0.75
[optional]: set the upper threshold for network bandwidth utilization,
between 0 and 1, default to 0.75
- DotsLinkLowerThreshold 0.50
[optional]: set the lower threshold for network bandwidth utilization,
between 0 and 1, default to 0.50
- DotsCpuMonitor On/Off
[optional]: set whether to monitor CPU workload, either "On" or "Off",
default to "Off"
- DotsCpuUpperThreshold 0.70
[optional]: set the upper threshold for CPU utilization, between 0 and 1,
default to 0.70, take effect only when DotsCpuMonitor is "On"
- DotsCpuLowerThreshold 0.45
[optional]: set the lower threshold for CPU utilization, between 0 and 1,
default to 0.45, take effect only when DotsCpuMonitor is "On"
- DotsRescueIdleTTL 300
[optional]: set the number of idle intervals for releasing rescue servers
(in the unit of DotsControlInterval), default to 300
- DotsOriginIdleTTL 300
[optional]: set the number of idle intervals for shutting down origin
servers (in the unit of DotsControlInterval), default to 300
- DotsPastOriginTTL 3600
[optional]: set the number of intervals for keeping old origin server
entries (in the unit of DotsControlInterval), default to 3600
- DotsSrvloc On/Off
[optional]: set whether to enable service discovery, either "On" or
"Off", default to "Off". If DotsSrvloc is "Off", then
DotSlash rescue service is off, but workload monitoring is still on.
- DotsSrvlocServer muni.cs.columbia.edu
[optional]: set the domain name or IP address for the service registry,
the default mSLP DA is discovered automatically via DNS SRV
in our dot-slash.net domain.
- DotsSrvlocPort 427
[optional]: set the port number for the service registry, default
to 427.
- DotsDiscoverTTL 120
[optional]: set the number of intervals for doing service queries
(in the unit of DotsControlInterval), default to 120
- DotsRegisterTTL 120
[optional]: set the number of intervals for doing service registrations
(in the unit of DotsControlInterval), default to 120
- DotsDnsUpdateServer muni.dot-slash.net
[optional]: set the domain name or IP address for the DNS server to
which dynamic DNS updates are sent. For the sake of efficiency, you
should set this server.
- DotsAliasBase foo.dot-slash.net
[optional]: set the alias base name, default to the server name
specified via the ServerName directive without the port number part.
Aliases are constructed sequentially in the form of
vh<seqno>.<alias_base_name>, where <seqno>
is a sequence number.
- DotsAliasPoolSize 0
[optional]: set the pool size of registered but unused aliases,
default to 0.
- DotsAliasRegNum 0
[optional]: set the number of registered aliases, default to 0.
- DotsAliasAllocNum 0
[optional]: set the number of used aliases, default to 0.
- Notes on DotsAliasRegNum, DotsAliasAllocNum, and
DotsAliasPoolSize
- DotSlash will register new aliases if
DotsAliasRegNum < DotsAliasAllocNum + DotsAliasPoolSize
- One way to avoid using dynamic DNS in DotSlash for alias registrations
is to register a large number (such as 100) of aliases off-line,
and set DotsAliasRegNum to that large number (i.e., 100).
- DotsDnsRoundRobin On/Off
[optional]: set whether to enable DNS round robin, either "On" or
"Off", default to "Off"
- DotsDnsStaticName staticname.foo.dot-slash.net
[optional]: set the static domain name which will be mapped to an
IP address that is not affected by DNS round robin. You can give
an IP address directly here. When DotsDnsRoundRobin is "On",
you should set DotsDnsStaticName. Default to the server name
specified via the ServerName directive without the port number part.
- DotsTsig On/Off
[optional]: set whether to use TSIG-signed DNS updates, either "On" or
"Off", default to "Off"
- DotsTsigKey foo.dot-slash.net
[optional]: set the key for TSIG. You need to get the key from your
DNS server. For our dot-slash.net domain, the key is the same as
your alias base name specified via DotsAliasBase. You must set
up this key if DotsTsig is "On".
- DotsTsigSecret xxxxxxxxxxxxxxxxxxxxxxxx
[optional]: set the secret for TSIG. You need to get the secret from
your DNS server. You must set up this secret if DotsTsig is "On".
- DotsTsigUpdateScript ${apache_install}/bin/dots_ddns
[optional]: set the update script (absolute path) for TSIG, default to
${apache_install}/bin/dots_ddns.
- ErrorDocument 302 " "
[optional]: set the notice in HTTP redirects. Apache has a default
but pretty long notice for HTTP redirects. You may want to reduce
the length of this notice in order to save network bandwidth since
DotSlash may trigger a lot of HTTP redirects. You can reduce this
notice to a single byte, but you cannot remove it completely.
- DotsRedirectOverhead 358
[optional]: set the overhead of TCP, IP, and Ethernet headers for
each HTTP redirect (in bytes), default to 358 bytes
- DotsAccountingScale 0.8
[optional]: set the accounting scale for HTTP traffic, i.e., the
portion of bandwidth that is usable for HTTP traffic, default to 0.8
In addition to the above configurations, you must configure memory
cache and disk cache for Apache because DotSlash depends on Apache caching
to cache static content. To save space, we will not cover configurations
for Apache memory cache and disk cache here. Please refer to Apache
documentation mod_cache for details. Next, we give two simple examples for DotSlash
configurations. For a complete example of DotSlash configurations, please
refer to the file
httpd.conf.dots.
- Example 1
To use DotSlash for rescue, at least add
the following two configurations to httpd.conf.
- LoadModule dots_module modules/mod_dots.so
- DotsSrvloc On
By doing so, you will use all default configurations as follows.
- You use the default mSLP DA in the dot-slash.net domain, which is
discovered via DNS SRV.
- You use your own DNS server for dynamic DNS updates, which is
determined automatically.
- All dynamic DNS updates are non-signed.
- Your maximum data rate for outbound HTTP traffic is set to 1000 kB/second.
- CPU workload monitoring is off.
- DNS round robin is off.
- Example 2
To use DotSlash for monitoring network workload,
add the following two configurations to httpd.conf.
- LoadModule dots_module modules/mod_dots.so
- <Location /dotslash-status>
SetHandler dotslash-status
<Location>
Thus, you can obtain the DotSlash status via /dotslash-status.
Running Apache with DotSlash is straight forward.
You do not need to start the DotSlash daemon dotsd manually
since it is started automatically by the Apache DotSlash module
mod_dots. However, you need to shutdown dotsd after
you shutdown your Apache server. Also, you need to shutdown dotsd
before you restart Apache with DotSlash. Assuming that the process
ID file for dotsd is ${apache_install}/logs/dotsd.pid, then
you can start, shutdown, and restart your Apache with DotSlash as follows.
- Start Apache with DotSlash
httpd -k start
- Shutdown Apache with DotSlash
httpd -k stop
kill -9 `cat ${apache_install}/logs/dotsd.pid`
- Restart Apache with DotSlash
kill -9 `cat ${apache_install}/logs/dotsd.pid`
httpd -k restart
Dots_PHP is a DotSlash extension for the PHP module of Apache
that supports replicating PHP scripts dynamically.
- Apache 2.0.49
- PHP 4.3.6
- MySQL 4.0.18
- Creating DotSlash extension for PHP
- Assume that the root directory of your PHP source code is
${php_source}.
- Create a skeleton for the "dotslash" extension:
cd ${php_source}/ext
./ext_skel --extname=dotslash --skel=./skeleton
which will create directory ${php_source}/ext/dotslash, and create
basic files under it, including dotslash.c, php_dotslash.h,
and config.m4.
- Copy the configuration file
config.m4
in this software release to ${php_source}/ext/dotslash/config.m4
- Re-build PHP configuration to include the "dotslash" extension:
cd ${php_source}
rm *.cache
./buildconf --force
- Compiling and installing Dots_PHP (go to directory
${php_source})
- Configure PHP as follows:
./configure --enable-dotslash --with-apxs2=${apache_install}/bin/apxs
--with-mysql=${mysql_source} --enable-experimental-zts
EXTRA_INCLUDES="-I${dots_apache} -I${apache_install}/include -I${apache_source}/modules/experimental"
where ${apache_install} is the root directory of your Apache installation,
${apache_source} is the root directory of your Apache source code,
${mysql_source} is the root directory of your MySQL source code,
and ${dots_apache} is the root directory of your Dots_Apache source code.
- Copy the following source files
dotslash.c and
php_dotslash.h
in this software release to ${php_source}/ext/dotslash
- Use make to compile
- Use make install to install,
which will install libphp4.so (integrated Dots_PHP)
under ${apache_install}/modules
- Configuring Dots_PHP using the Apache configuration file
httpd.conf.
- LoadModule php4_module modules/libphp4.so
[required]: load the PHP module dynamically
- AddType application/x-httpd-php .php .phtml
[required]: specify that files with extension php or phtml
are regarded as PHP scripts
- ErrorDocument 404 /_dots_404.php
[required]: set "File not found" handler (relative to
${DocumentRoot}) for PHP scripts
- DotsScriptRoot ${apache_install}/htdocs/_dots_script
[optional]: set the root directory (absolute path) for caching PHP
scripts, default to ${apache_install}/htdocs/_dots_script
- Running Apache with Dots_Apache + Dots_PHP
- Same as running Apache with Dots_Apache.
Dots_MySQL is a cache-enhanced PHP data driver
for MySQL databases that supports caching database query results on demand.
- php_mysql.c: cache-enhanced
PHP data driver for MySQL databases, which uses
memcached as the
cache engine, and uses
libmemcache
C library to access memcached.
- Compiling and installing Dots_MySQL (go to directory
${php_source})
- Configure PHP as follows:
./configure --enable-dotslash --with-apxs2=${apache_install}/bin/apxs
--with-mysql=${mysql_source} --enable-experimental-zts
EXTRA_INCLUDES="-I${dots_apache} -I${apache_install}/include -I${apache_source}/modules/experimental -I{libmemcache_install}/include"
EXTRA_LDFLAGS=-L${libmemcache_install}/lib
EXTRA_LDFLAGS_PROGRAM=-L${libmemcache_install}/lib
EXTRA_LIBS=-lmemcache
where ${apache_install} is the root directory of your Apache installation,
${apache_source} is the root directory of your Apache source code,
${mysql_source} is the root directory of your MySQL source code,
${dots_apache} is the root directory of your Dots_Apache source code,
and ${libmemcache_install} is the root directory of your
libmemcache installation.
- Copy the source file
php_mysql.c
in this software release to ${php_source}/ext/mysql.
- Use make to compile
- Use make install to install,
which will install libphp4.so (integrated Dots_MySQL)
under ${apache_install}/modules
- Configuring Dots_MySQL using the Apache configuration file
httpd.conf.
- DotsQCacheServer 127.0.0.1:11211
[optional]: set the query result cache server in the form of
"host_name:port_number" or "ip_address:port_number". For local
query result cache server, please use IP address 127.0.0.1.
Dots_MySQL is enabled only when DotsQCacheServer
has been set. By default, Dots_MySQL is disabled.
- DotsQCacheTTL 60
[optional]: set the number of intervals for keeping database query
results (in the unit of DotsControlInterval), default to 60
- Running Apache with Dots_Apache + Dots_PHP + Dots_MySQL
- You need to start memcached before starting Apache as follows.
memcached -d -m 500 -p 11211
httpd -k start
- This software release is developed and tested using the following
platforms and software components.
- Red Hat 9.0 and Red Hat Enterprise Linux AS v.3
- Apache 2.0.49
- BIND 9.2.2
- mSLP DA
- PHP 4.3.6
- MySQL 4.0.18
- memcached
- libmemcache
- We use the following tools to generate various workloads to
evaluate DotSlash.
Last updated
by Weibin Zhao