Adding Digest Access Authentication to a WebDAV Server

A web server that implements the extensions to HTTP to provide support for the Distributed Authoring and Versioning protocol (WebDAV) [1] must employ Digest Access Authentication [2] to authenticate users to the server. Digest Authentication is required so that a WebDAV server can prevent an unauthorized user from modifying content on the server as well as providing a mechanism for authenticating users who make requests for resource locks (e.g., exclusive write locks). A prototype WebDAV server was modified to use Digest Authentication since it originally had no authentication mechanism.

Introduction

The starting place for this project was a prototype WebDAV server [3] (or simply "WebDAV server") with no support for authentication. In an effort to bring the WebDAV server into compliance with the latest version of the WebDAV protocol specification, the Digest Access Authentication mechanism had to be supported by the server.

Digest Authentication relies on a simple challenge-response authentication mechanism. When a client requests access to a resource which requires authentication, the server replies with the "401 Unauthorized" HTTP status code and sends a "WWW-Authenticate" HTTP header in the response that contains a nonce value and specifies that the Digest authentication scheme is requested. The client is then expected to resubmit the original request but also add an "Authorization" HTTP header to the request. The Authorization header sent by the client must contain a checksum of the username, the password, the original nonce value, the requested resource and the HTTP method. Digest Authentication does not require a clear-text password to be transferred over the network unlike Basic Access Authentication [4].

In the section on WebDAV I will provide some background and motivation for the WebDAV protocol specification as well as provide reasons why Digest Authentication is necessary for ensuring security and consistency. In the Digest Access Authentication section I will briefly describe the overall operation of the Digest mechanism. I will explain the algorithms employed by Digest authentication and also consider security issues. I will describe the prototype WebDAV architecture in the Architecture section and explain how the architecture had to be modified to accommodate the authentication mechanism. Links to program documentation including a user and installation guide will be provided.

Related Work

The WebDAV protocol specification was submitted to the Internet Engineering Steering Committee (IESG) for approval as a Proposed Standard on April 9, 1998. To qualify as a Proposed Standard, the protocol must have a certain number of prototype implementations as specified in RFC 2200 [5]. Independent implementations of the WebDAV protocol and interoperability among WebDAV servers are key factors to allowing the WebDAV protocol specification to advance along the Internet standards track. Adding Digest Authentication to the prototype WebDAV server satisfies a compliance requirement of the current WebDAV protocol specification. If the server can be brought into full compliance with the WebDAV specification, it will allow the WebDAV Working Group to reference it as a compatible WebDAV server implementation and show that it is interoperable with other WebDAV servers.

Some examples of proprietary distributed authoring systems that use Basic Authentication are Basic Support for Collaborative Work (BSCW) [7] and Microsoft FrontPage [8].

Distributed Authoring for the World Wide Web

Motiviation

Traditional methods of maintaining content on a web server, for example, transferring files to the web server using FTP or authoring content directly on the web server, suffer from scalability and document management issues. Requiring authors to have login access to the web server, preventing authors from overwriting each other's changes (the "lost update" problem), and keeping track of changes made, becomes increasingly difficult if the number of authors is large. The Hypertext Transfer Protocol (HTTP/1.1) [6] does not provide methods for associating "meta" information or arbitrary properties with resources (for example, author information, version history, and external links).

There are several proprietary distributed authoring systems that provide solutions to remotely authoring content. Microsoft FrontPage [8], AOLpress [9, 10], BSCW [7], and LotusNotes [11] are examples of these authoring systems. The problem with these proprietary solutions is that they are not interoperable because they all solve the same problem in a non-standard way. FrontPage employs RPC over HTTP, AOLpress uses non-standard HTTP method extensions (e.g., MKDIR, BROWSE) that are similar to WebDAV methods, and BSCW uses the common gateway interface (CGI) to provide remote authoring and document management functionality.

WebDAV

The WebDAV protocol specifies a set of methods, headers and content-types that extend HTTP/1.1 to provide for the management of resource properties, creation and management of collections (directories), namespace manipulation, and resource locking. The WebDAV specification is currently an Internet Draft that is awaiting approval as a Proposed Standard by the Internet Engineering Steering Committee.

WebDAV allows arbitrary resource properties to be associated with a resource and allows properties to be queried for a resource. Property values must be well formed XML. On a WebDAV server, many of the standard HTTP/1.1 headers that are returned in a GET request for a resource are represented as actual WebDAV properties for that resource (e.g., "Content-Length", "Content-Type"). WebDAV provides the PROPPATCH method to allow properties to be set on a resource or removed from a resource. The PROPFIND method is provided to allow properties to be queried for a resource.

The WebDAV specification states that a collection is "a resource whose state consists of at least a list of internal members and a set of properties" [1]. On a file system, a directory can be considered a collection whose list of internal members is the list of files the directory contains. In fact, this is how the prototype WebDAV server implements collections. The WebDAV MKCOL method is used to create a collection on the WebDAV server and the DELETE method is used to delete a collection.

Several methods are defined to allow the namespace of the file system to be manipulated. The PUT, DELETE and GET methods have the same semantics as the HTTP/1.1 version of those methods. The COPY and MOVE methods are introduced by WebDAV to allow a resource to be copied or moved, respectively, on the WebDAV server.

The WebDAV LOCK and UNLOCK methods provides a mechanism for avoiding the scenario where one author unknowingly overwrites the changes of another. When a resource is locked with the LOCK method, only the author who originally locked the resource and therefore holds the "lock token", will be allowed to modify the resource. A typical workflow for the author would be to LOCK the resource on the server, create a local copy of the resource by using GET, modify the local copy, PUT the modified resource on the server, and then call UNLOCK to release the resource.

WebDAV relies heavily on Digest Access Authentication for maintaining security and consistency of the server resources. All the WebDAV methods that can modify a resource -- PUT, POST, DELETE, COPY, MOVE, PROPPATCH, LOCK, UNLOCK, and MKCOL -- require authentication of the user in the event that the resource being manipulated has a write lock. In addition, current proposals for WebDAV access control [12] will require authentication.

The prototype WebDAV server has additional authentication requirements because it uses internal write locks to ensure consistency and enforce method atomicity. For example, if a user wants to copy a resource and the resource at the destination location is not write locked, the WebDAV server will take out a temporary write lock on the destination resource until the method completes. This ensures that race conditions do not allow another user to overwrite the destination resource while the method is in mid-execution. Therefore the prototype WebDAV server requires authentication for methods that can modify a resource even if there are no write locks on the resources involved.

Digest Access Authentication

Overview

The Digest Access Authentication mechanism was proposed as a better form of authentication than Basic Authentication because Digest does not require the transmission of a clear-text password over the network. Digest Authentication uses the same challenge-response paradigm as Basic, where the server challenges the client to authenticate itself, and the client responds with its authentication.

The challenge consists of an HTTP response with the "401 Unauthorized" status code and a WWW-Authenticate HTTP header. The WWW-Authenticate header contains the authentication scheme (Digest), the realm which indicates to the user which username and password to use, and a nonce value that is an arbitrary value generated by the server. When the client application receives the 401 status code and WWW-Authenticate header it would resubmit the original request but also include an Authorization header. The Authorization header must contain a checksum of the username, password, the nonce value from the original WWW-Authenticate header, the requested uri, and the method. The client application may have to prompt the user to enter the username and password. When the server receives the Authorization header, it would perform the same algorithm that the client performed to generate the checksum and then compare it to the client's checksum to determine if the authentication succeeded.

Security Issues

Some possible security considerations are the "replay attack", "man in the middle", spoofing by counterfeit servers, and storing passwords on the server. The replay attack occurs when a network eavesdropper captures HTTP requests and resubmits them. Although replaying a GET request is rather harmless a replay attack can have dire consequences when applied to WebDAV methods which can modify a resource. To guard against this type of attack, the WebDAV server should be choose its nonce values carefully and require the client to submit a digest of the resource contents.

The "man in the middle" attack can occur when a hostile or compromised proxy sits between the client and the server and is able to intercept and alter messages between the two. A simple attack would be for the proxy to change the Digest authentication request to a Basic authentication request that could fool the client into sending a password in the clear. A WebDAV client should require Digest authentication from a WebDAV server and warn the user if a non-Digest authentication scheme is being requested. Requiring Digest authentication will also prevent the problem where a counterfeit WebDAV server attempts to learn a user's password by sending a Basic authentication request.

Digest authentication requires that the server store a digested value of username, password and realm. If this file is compromised, it will be possible for the attacker to have immediate access to documents on the server for that realm. This is a more serious problem then compromising a standard Unix password file because the Unix password file would have to be decrypted to gain access to documents. By including the realm in the digest, the consequences of a compromised Digest password file are limited to applying to files within that realm only. Still, the Digest password file should be protected as if it had unencrypted password in it and the realm should be different for each host doing the authentication.

Architecture

Prototype WebDAV Server

The original prototype WebDAV server was written in Java as an extension to a Java Web Server [13] using the Java Servlet API [14]. The WebDAV server has a layered system architecture including a Protocol Layer, an Executive Layer, a Subsystem Registry, and the Functional Subsystems. A brief overview of the system architecture and a description of how Digest Authentication was added to the architecture will be given.

The following diagram shows WebDAV system architecture with the new Authentication Subsystem. The description of each of the subsystems is given below.

The Protocol Layer handles hooking into the Java Web Server, capturing all incoming HTTP and WebDAV requests and forwarding them to the Executive Layer. This layer was implemented as a Java Servlet called the WebDavServlet.

The Executive Layer takes care of transforming raw HTTP and WebDAV requests into an internal representation Method object that has a CommandQueue object. A CommandQueue contains a sequence of Commands that, when executed, perform the request specified. The CommandQueue provides support for WebDAV method semantics detailed in the WebDAV specification. These semantics include precondition checking (e.g., does the requested file exist?), transaction support in the form of "rollback" to emulate atomic methods, and system locks to maintain internal consistency of server resources.

The Subsystem Registry allows Functional Subsystems to register themselves so that they are accessible to the Executive Layer and to other Functional Subsystems. This allows multiple subsystems of the same type to register to handle different namespaces on the server. An example of this would be one Namespace Manager registering itself to handle files on the server's file system while a different Namespace could handle resources that reside in a database on the server. The Subsystem Registry can also be used to allow one subsystem to use the services of another. In fact, the prototype WebDAV server does just this by implementing the Lock Manager to use the Property Manager to store locks as properties on the resource.

The Functional Subsystems provide the services as specified by the WebDAV protocol specification. The Namespace Manager handles manipulation of the namespace by supporting the GET, HEAD, POST, PUT, COPY, MOVE, MKCOL and DELETE methods. The Property Manager handles storing and retrieving the properties of a resource by supporting the PROPFIND and PROPPATCH methods. The Lock Manager handles locking and unlocking resources by supporting the LOCK and UNLOCK methods.

Authentication Manager Subsystem

In original prototype did not have any form of authentication. Instead it re-purposed the WebDAV "Owner-Info" header to have it identify the client. The client was trusted to provide reliable information in the "Owner-Info" header and to ensure that it uniquely identified the client. The first step in adding Digest Authentication to the WebDAV server was identifying the appropriate system layer that it belonged in.

I felt that it was appropriate to separate the authentication mechanism and policy from the Lock Manager subsystem. This would allow the Lock Manager to assume that it was only dealing with authenticated users which simplifies the Lock Manager implementation.

A new Authentication Manager subsystem was created to handle authentication. Adding an authentication subsystem allows the implementation of the subsystem to change as long as the interface stays the same. The Authentication subsystem interface assumes that the Digest authentication scheme is being used. The implementation of how the server retrieves the user passwords (or the shared secret), which checksum algorithms to use, and what security precautions are taken, can easily be changed which allows the system architecture to remain flexible. The implementation of the Authentication Manager for the WebDAV server is very simple: encrypted passwords are stored in a file on the server, the default MD5 [15] message digest algorithm is used to compute the checksum, and minimal security precautions were used.

The Java Server API provides the sun.security.realm package that allows the WebDAV server to obtain the password for a user. The password file is stored on the server and can be manipulated using the Java Web Server Administrator applet. This allows the WebDAV server to take advantage of the underlying security mechanisms employed by the Java Server to prevent the password file from being compromised. However, it would be easy to obtain the plain text passwords from the encrypted password file by using the sun.security.realm API in the same way that the Authentication Manager uses it.

To prevent certain security attacks mentioned in the Security Issues section, it is recommended that a digest of the resource contents, the request and the response bodies be required. In addition, careful creation and monitoring of nonce values by the server can prevent some attacks. This prototype implementation does not employ these security enhancements.

Authentication Command

The Authentication Manager subsystem defines an interface and implementation for authenticating a user. The AuthenticationCmd class was created to interact with the Authentication Manager in the context of a WebDAV Method's CommandQueue object. Once the AuthenticationCmd was created, it was trivial to add it to the CommandQueue's of each WebDAV Method that required authentication. For example, the Put Method's CommandQueue was modified in the following way:

CommandQueue for the Put Method
Before Authentication	After Authentication
1. Validate URI	1. Validate URI
2. Lock Resource	2. Authenticate User
3. Put Resource	3. Lock Resource
4. Unlock Resource	4. Put Resource
	5. Unlock Resource

The Put Method CommandQueue originally consisted of a ValidateUriCmd to ensure that the resource URI was valid, followed by a LockResourceCmd to prevent race conditions, followed by a PutResourceCmd to put the resource contents on the server, and finally an UnlockResourceCmd to unlock the resource. Adding authentication to the PutMethod required adding the AuthenticationCmd between the ValidateUriCmd and the LockResourceCmd.

The prototype Client application had to be extended as well to handle the authentication request from the server. The client now allows the user to enter a username and a plain text password (which is not echoed back to the user). When a server challenges with the 401 Forbidden status code and the WWW-Authenticate header, the client uses the username, password, realm, method and request uri to generate an Authorization header that is passed along with the resubmitted request.

Program Documentation

The documentation for the Java packages and classes for the WebDAV server, including the new package and classes to support Digest Authentication, can be found at

http://www.cs.columbia.edu/~eaddy/packages.html

The prototype WebDAV server and client source code can be obtained by downloading the Java Archive (JAR) file at

http://www.cs.columbia.edu/~eaddy/webdav.jar

Installation instructions and system requirements can be found in Appendix A and B of

http://www.cs.columbia.edu/~eaddy/webdav.html

Here is a brief synopsis of the webdav.authentication package and the new classes.

Package webdav.authentication

AuthenticationInterface.java	Provides an interface for authenticating a client using Digest Authentication.
AuthenticationManager.java	Concrete implementation of AuthenticationInterface which uses MD5 as the message digest algorithm and the sun.security.realm API to authenticate user passwords.
WWWAuthenticateHeader.java	Represents a WWW-Authenticate header. Used by the server to send the header and by the client to parse the header.
AuthorizationHeader.java	Represents a Authorization header. Used by the client to send the header and by the server to parse the header.

Package webdav.common (New Classes)

HeaderWithParameters.java	Represents an HTTP header with key="value" parameters.
Assert.java	Helper class to enforce programming assumptions similar to the standard C library assert( ) function.
DigestHelper.java	Helper class for using the java.security.MessageDigest class.

Package webdav.executive (New Classes)

AuthenticationCmd.java

Command that encapsulates an authentication request.

Task List

The prototype WebDAV server and client was designed and developed by Jonathan Shapiro, Eric Rong and Marc Eaddy. All of the source code for the WebDAV server is our original work and has been released into the public domain. The WebDAV client contains helper HTTP client code developed by Ronald Tschalär.

The Java classes created to support Digest Authentication were designed and developed by Marc Eaddy. The message digest algorithm used, MD5, and the username to password mapping mechanism (password file lookup) was provided by the Java Security API.

The sun.security.realm package provides a Unix realm that allows the server to use the native password file on a Unix server. This would allow all users who maintain a Unix account on the server to immediately be allowed to authenticate to the server. The Unix realm was not used to keep the prototype simple but would be an obvious future enhancement to the WebDAV server. Another option would be to do away with using plain text passwords all together and simply store the checksum of username, password, and realm as specified by the Digest Authentication RFC. This would prevent the possibility of a devious user from obtaining an unencrypted password but would open the server up to other security attacks as outlined in the RFC.

One obvious security hazard that comes with using the sun.security.realm API to handle looking up password for users is that the unencrypted password is obtainable via the API. This is exactly how the AuthenticationManager was able to authenticate user passwords.