22-05-2012, 04:39 PM
Dual Framework and Efficient monitoring for Targeted
Online Data Delivery
Dual Framework and Efficient monitoring.doc (Size: 72 KB / Downloads: 59)
Abstract:
A variety of emerging online data delivery applications challenge existing techniques for data delivery to human users, applications, or middleware that are accessing data from multiple autonomous servers. In this project, we develop a framework for formalizing and comparing pull-based solutions and present dual optimization approaches. The first approach, most commonly used nowadays, maximizes user utility under the strict setting of meeting a priori constraints on the usage of system resources. We present an alternative and more flexible approach that maximizes user utility by satisfying all users. It does this while minimizing the usage of system resources. We discuss the benefits of this latter approach and develop an adaptive monitoring solution Satisfy User Profiles (SUPs). Through formal analysis, we identify sufficient optimality conditions for SUP. Using real (RSS feeds) and synthetic traces, we empirically analyze the behavior of SUP under varying conditions. The proposed framework aims at providing a scalable online data delivery solution. We identify three types of entities, namely servers, clients, and brokers. We propose a dual formulation OptMon2, which reverses the roles of user utility and system constraints, setting the fulfillment of user needs as the hard constraint. OptMon2 assumes that the system resources that will be consumed to satisfy user profiles should be determined by the specific profiles and the environment, e.g., the model of updates, and does not assume an a priori limitation of system resources.
Introduction
The diversity of data sources and Web services are currently available on the Internet as the diversity of client application requirements. In this paper, we address the task of targeted data delivery. Users may have specific requirements for data delivery, e.g., how frequently or under what conditions they wish to be alerted about update events or update values, or their tolerance to delays or stale information. The challenge is to deliver relevant data to a client at the desired time, while conserving system resources. We consider a number of scenarios including RSS news feeds, stock prices and auctions on the commercial Internet, and scientific data sets and Grid computational resources. We consider architecture of a proxy server that is managing a set of user profiles that are specified with respect to a set of remote autonomous servers. Push, pull, and hybrid protocols have been used to solve a variety of data delivery problems. Push-based technologies include BlackBerry and JMS messaging, push-based policies for static Web content and push-based consistency in the context of caching dynamic Web content.
Push is typically not scalable, and reaching a large number of potentially transient clients is expensive. In some cases, pushing information may overwhelm the client with unsolicited information. Pull-based freshness policies have, therefore, been proposed in many contexts such as Web caching and synchronizing collections of objects, e.g., Web crawlers. Several hybrid push-pull solutions have also been presented (e.g., We focus on pull-based resource monitoring and satisfying user profiles. As an example, consider the setting of RSS feeds that are supported by a pull-based protocol. Currently, the burden of when to probe an RSS resource lies with the client.
Although RSS providers use a Time-To-Live (TTL) measure to suggest a probing schedule, a study on Web feeds shows that 55 percent of Web feeds are updated on a regular hourly rate. Further, due to heavy workloads that may be imposed by client probes (especially on popular Web feed providers such as CNN), about 80 percent of the feeds have an average size smaller than 10 KB, suggesting that items are promptly removed from the feeds. These statistics on refresh frequency and volatility illustrate the challenge faced by a proxy in satisfying user needs. As the number of users and servers grow, service personalization through targeted data delivery by a proxy can serve as a solution for better managing system resources. In addition, the use of profiles could lower the load on RSS servers by accessing them only to satisfy a user profile.
We present an extensive evaluation of the solutions to the two monitoring problems. For our experimental comparison of OptMon1, we consider the WIC algorithm which provides the best solution in the literature
Literature survey
Monitoring can be done using one of three methods, namely push-based, pull-based, or hybrid. With push-based monitoring the server pushes updates to clients, providing guarantees with respect to data freshness at a possibly considerable overhead at the server. With pull-based monitoring, content is delivered upon request, reducing overhead at servers, with limited effectiveness in estimating object freshness. The hybrid approach combined push and pull, either based on resource constraints [6] or role definition. For the latter, consider the user profile language we have presented. Here, it is possible that servers of trigger classes will push data to clients, while data regarding query classes will be monitored by pulling content from servers once a notification rule is satisfied.
PROBLEM DESCRIPTION
To address some of the limitations of OptMon1,we propose a framework where we consider the dual of the previous optimization problem as follows:
Given some set of user profiles, minimize the consumption of system resources.
The proposed framework aims at providing a scalable online data delivery solution