Large-Scale On-Demand Broadcast for Large and Dynamic Data

Demet Aksoy


Recent advances in telecommunications have enabled the deployment of broadcast-based wide-area information services that provide on-demand data access to very large client populations. Improvements in interconnectivity are fueling an exponential growth in the number of users that want to access data. Meanwhile the amount of data available to these users is also increasing as continuous additions and updates of data take place. In this environment it is necessary to have efficient on-demand scheduling algorithms that will scale as the number of users, the data set size and the broadcast rate increases.

In our work to date we have developed a parameterized on-demand scheduling algorithm for memory-resident data, that provides a good performance across the criteria of responsiveness, scalability and robustness. This algorithm is tunable to emphasize different requirements of the system within the performance criteria and can balance individual and overall responsiveness. Unlike previous work on low overhead scheduling, the proposed algorithm is not based on estimates of the access probabilities of items, but rather, it makes scheduling decisions based on the current queue state. This behavior allows the algorithm to easily adapt to changes in the intensity and distribution of the workload. We have implemented a prototype that uses this algorithm for making broadcast scheduling decisions.

We are going to extend the existing study in three dimensions: 1) We will develop efficient scheduling policies for disk-resident data broadcast. In particular we will design techniques that will increase cache hit rates and minimize the latencies caused by the retrieval of data from disk. 2) We will develop a publish/subscribe based method to disseminate updates on the data in a timely fashion. 3) Finally, we define a query-based on-demand broadcast system where the clients access the database by issuing queries instead of requesting specific pages.

Back to the Spring 1998 dbchat index