MarketNet: A Survivable, Market-Based Architecture for Large-Scale Information Systems

Email Contact: Apostolos Dailianas

The Challenge

The challenge addressed by this project is how to accomplish quantifiable, predictable survivability of large scale information systems under loss or rapid dynamic changes in availability of resources.  The term resource is used in its broadest form to mean a physical resource such as CPU, memory, storage, bandwidth or sensors, or a higher level service resource such as a file server, database server, name server, web server, or a particular data or application software.
Loosely speaking, survivability means that client applications requiring use of these resources can adapt to reduced resources or use alternate resources to execute, according to their intrinsic priority.  It means that resource managers can reallocate resources to best reflect the needs and priorities of clients.  It also means that once an attack has been identified, the system can rapidly deploy protection of its healthy parts against further loss and damages.
 

The Approach

The MarketNet project is developing novel technologies, based on economic mechanisms, to ensure the systematic, quantifiable and predictable survivability of large-scale information systems. These technologies include mechanisms, protocols, algorithms and analyses to support the protection of network resources within a domain, the "fire walling" of domains to limit the spread of faults and the adaptation of network clients and services to changing resource availability.

In MarketNet, access to resources in a distributed information and communication system is governed by a market economy, where network elements can purchase and sell services, optimizing their utility measures and maximizing their revenues.

The resources to be traded include physical resources such as CPU cycles, storage, bandwidth, I/O devices or sensors as well as higher-level services such as file storage, name service, database or web service. When failures or attacks lead to loss of resources, prices will rise and limit access to high-priority (high budget) clients or encourage clients to adapt their resource demands.

Like a traditional economy, a provider of services may also be a consumer of other services, both at higher and lower layers. Servers can use this revenue to replicate their services. Thus, the services most valued by their clients are provided with the highest redundancy.  It is anticipated that the network currencies are fungible, i.e., exchangeable with traditional currencies.

Since clients requesting resources have to pay for these, the amount of damage a single client can do is limited by their available budget. We are building on existing secure electronic payment systems issued by trusted, secured "central banks" to ensure that clients cannot forge money.

The proposed economic approach enables quantification, analysis, planning and optimization of survivability measures. We will design mechanisms that minimize the transient time to reach a new supply/demand equilibrium after a loss of resources. The availability of network services to priority tasks can be measured by the average cost of network services access. The value lost due to an attack can be measured by the decrease in net revenues generated.

To limit fraud in MarketNet, we develop technologies and an infrastructure to provide the means of detecting patterns of attacks or fraudulent network activities in much the same fashion as is commonplace today in typical transaction processing systems, building upon our research in intrusion and fraud detection in financial information systems. Here we model temporal behaviors of agents and patterns of resource access to classify activities into those that are legitimate and those that seem suspicious and hence warrant further inspection and authorization. Knowledge about attacks is to be rapidly and effectively communicated to limit the ability of attackers to exploit the same techniques in different domains.
 

Status

We are currently developping the following components

Guided Research Projects

Please check the guided research projects for undergraduates and masters students.
 

This project is a DARPA/ITO-funded research effort.



[Distributed Computing and Communication (DCC) Lab]
Columbia University
New York, NY 10027

 Comments to Apostolos Dailianas