Doctor of Philosophy, The Ohio State University, 2021, Industrial and Systems Engineering

This research explores the reinforcement learning methods, machine learning methods, and discrete event simulation models with applications in the field of cybersecurity. In cybersecurity, virtually all types of devices that contain computers have so-called “cyber vulnerabilities” which offer ways for attackers to gain access or at least limit performance. A race then follows between hackers' finding and applying “exploits”, and vendors offering patches that are discovered to be needed by scans and implemented by end users. If the hackers win, they cause losses. In this dissertation, we propose a discrete event simulation model in which the mechanism of vulnerabilities and hosts has been studied. A concept of a nested “birth and death” process is introduced in the context of vulnerability lifetime and its interaction with a host. Also, we investigate the benefits and drawbacks of the current scanning policy and maintenance policy with a case study of a major university. We also propose cost-effective alternatives and investigate the significance of celebrity vulnerabilities. Next, we explore the optimal control policies to schedule cyber maintenance actions in a partially observable environment caused by incomplete inspections. Incomplete inspection, resulting mainly from computers being turned off during the scan, leads to a challenge for scheduling maintenance actions. We propose the application of Partially Observable Markov Decision Processes (POMDPs) to derive cost-effective cyber-maintenance actions that minimize total costs. To assess the benefits of optimal policies obtained from POMDPs, we use real-world data from a major university. Compared with alternative policies using simulations, the optimal control policies can significantly (2x ~ 10x) reduce expected maintenance expenditures per host and relatively quickly mitigate the most important vulnerabilities. Further, we investigate the main disadvantages of the widely used Common Vulnerability Scoring S (open full item for complete abstract)

Committee: Theodore Allen (Advisor); Cathy Xia (Committee Member); Guzin Bayraksan (Committee Member) Subjects: Industrial Engineering

Doctor of Philosophy, The Ohio State University, 2010, Electrical and Computer Engineering

The presence of multiple users in a network provides us with a valuable resource known as multiuser diversity. With information on the instantaneous states of the channels, multiuser diversity can be tapped by opportunistic multiuser scheduling. It is important that the channel state information is acquired in a cost-effective way so that the losses involved in this operation do not offset the gains promised by opportunistic scheduling. For various network environments of practical interest, this dissertation models the radio frequency links with memory, and studies the modalities to exploit the channel memory to simultaneously estimate channel state information, while performing opportunistic multiuser scheduling. The data transmission at any point of time is shown to be associated with two potentially contradicting objectives: opportunistic scheduling for immediate gains and channel exploration for future gains. Thus the joint scheduling problem is a dynamic program, specifically a partially observable Markov decision process that is traditionally known to be intractable or computationally expensive to implement. For various networks, we study these processes in an optimality framework and whenever possible, derive the optimal scheduling policy in closed form. In other cases, strongly founded on the optimality framework, we derive computationally inexpensive scheduling policies with near-optimal numerical performances. By appropriately exploiting the memory in the fading channels, significant system level gains can be achieved using opportunistic scheduling, even with minimal feedback, and a considerable portion of these gains can be realized even with sub-optimal policies that are computationally inexpensive to implement -- This is the central message of this dissertation.

Committee: Philip Schniter PhD (Advisor); Ness Shroff PhD (Advisor); Emre Koksal PhD (Committee Member) Subjects: Computer Science; Electrical Engineering; Engineering