Master of Science (MS), Wright State University, 2016, Computer Science

With an ever increasing amount of data that is shared and posted on the Web, the desire and necessity to automatically glean this information has led to an increase in the sophistication and volume of software agents called web robots or crawlers. Recent measurements, including our own across the entire logs of Wright State University Web servers over the past two years, suggest that at least 60\% of all requests originate from robots rather than humans. Web robots display different statistical and behavioral patterns in their traffic compared to humans, yet present Web server optimizations presume that traffic exhibits predominantly human-like characteristics. Robots may thus be silently degrading the performance and scalability of our web systems. This thesis investigates a new take on a classic performance tool, namely web caches, to mitigate the impact of robot traffic on web server operations. It proposes a cache system architecture that:~(i) services robot and human traffic in separate physical memory stores, with separate polices;~(ii) uses an adaptable policy for admitting robot related resources;~(iii) combines a deep neural network with Bayesian models to improve request prediction. Experiments with real data demonstrate (i) significant reduction in bandwidth usage for prefetching and (ii) improvements in hit rate for human driven traffic compared to a number of baselines, especially in configurations where web caches have limited size.

Committee: Derek Doran Ph.D. (Committee Chair); Tanvi Banerjee Ph.D. (Committee Member); John Gallagher Ph.D. (Committee Member) Subjects: Computer Science

Master of Science, University of Akron, 2006, Computer Science

The growth of the World Wide Web has been tremendous over the last decade, but at the same time, it has exacerbated the response time as perceived by users in accessing web documents. Web Caching and web prefetching are techniques used to alleviate such problems. Caching improve access latency by locally storing previously accessed web documents whereas prefetching techniques rely on predictive approaches to speculatively retrieve and store web objects into the cache for future use. Predictions on what to prefetch are made based on different criteria such as access patterns, popularity and structure of documents accessed. Prefetching approaches differ in their implementation and complexity, but more importantly in the performance that can be achieved based upon the prediction accuracy. In this work, we introduce a simple and transparent popularity-based prefetching algorithm which combines both the top 10 and next-n prefetching approaches. In addition to using access-frequency as the criteria for prefetching, we also use the time of access of web documents to generate the top 10 list. This approach of using access-frequency and time of access is known as the GDSP approach, which has been used in cache management. Instead of generating next-n list for all the documents accessed by the users, we log the next-n documents for the top 10 documents only, thus reducing complexity and overhead. The results obtained from simulations, in terms of hit rate and prefetching effectiveness show the efficacy of our proposed algorithm as compared to other approaches. Future work includes making the proposed approach dynamic by refreshing the top 10 list with the latest GDSP values and prefetching dynamic web documents.

Committee: Xuan-Hien Dang (Advisor) Subjects: Computer Science