Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


2021_PhD_Dissertation_EnhaoLiu.pdf (5.56 MB)

ETD Abstract Container

Abstract Header

Innovative Simulation and Tree Models and Reinforcement Learning Methods with Applications in Cybersecurity

Liu, Enhao
http://rave.ohiolink.edu/etdc/view?acc_num=osu1618411811368024

Abstract Details

2021, Doctor of Philosophy, Ohio State University, Industrial and Systems Engineering.
This research explores the reinforcement learning methods, machine learning methods, and discrete event simulation models with applications in the field of cybersecurity. In cybersecurity, virtually all types of devices that contain computers have so-called “cyber vulnerabilities” which offer ways for attackers to gain access or at least limit performance. A race then follows between hackers' finding and applying “exploits”, and vendors offering patches that are discovered to be needed by scans and implemented by end users. If the hackers win, they cause losses. In this dissertation, we propose a discrete event simulation model in which the mechanism of vulnerabilities and hosts has been studied. A concept of a nested “birth and death” process is introduced in the context of vulnerability lifetime and its interaction with a host. Also, we investigate the benefits and drawbacks of the current scanning policy and maintenance policy with a case study of a major university. We also propose cost-effective alternatives and investigate the significance of celebrity vulnerabilities. Next, we explore the optimal control policies to schedule cyber maintenance actions in a partially observable environment caused by incomplete inspections. Incomplete inspection, resulting mainly from computers being turned off during the scan, leads to a challenge for scheduling maintenance actions. We propose the application of Partially Observable Markov Decision Processes (POMDPs) to derive cost-effective cyber-maintenance actions that minimize total costs. To assess the benefits of optimal policies obtained from POMDPs, we use real-world data from a major university. Compared with alternative policies using simulations, the optimal control policies can significantly (2x ~ 10x) reduce expected maintenance expenditures per host and relatively quickly mitigate the most important vulnerabilities. Further, we investigate the main disadvantages of the widely used Common Vulnerability Scoring System (CVSS) including (i) lack of sufficient granularity to differentiate the most severe vulnerabilities; (ii) static threat scores to assess the risks of vulnerabilities; and (iii) time-delay response to remediation due to longer evaluation period. To address these issues, we proposed a new Prioritization Scoring System (PSS) that leverages ensemble machine learning tree methods (XGBoost) to accurately predict and classify vulnerabilities that fuse data from local companies and several online sources including National Vulnerability Database, Exploit Database, Zero-day Database, Symantec, and Twitter. We provide evidence that the proposed PSS can achieve about 95% precision and 91% recall to predict exploits. To facilitate such predictions and others, we propose the first complete and correct optimal decision tree formulation that includes node complexity penalties. Apparently, the incompleteness of previous formulations was not noticed because they were solved with warm starts and heuristics. Specifically, we propose a new M-OCT formulation with novel leaf-branch-interaction constraints that guarantee valid tree structures when generating optimal trees. By incorporating the idea of binary encoding of thresholds from a previous article, we reduce the total number of variables. We then extend M-OCT to construct a novel formulation called BNP-OCT with binary splits and node complexity constraints. We show experimentally on 16 standard data sets that M-OCT & BNP-OCT outperform Classification and Regression Tree (CART) in standard training and testing data sets in terms of model accuracy and generalizations. Also, the proposed BNP-OCT is approximately 10 times faster than M-OCT on average.
Theodore Allen (Advisor)
Cathy Xia (Committee Member)
Guzin Bayraksan (Committee Member)
176 p.
Industrial Engineering
Partially Observable Markov Decision Processes; Optimal Trees; Discrete Event Simulation; Cybersecurity

Recommended Citations

Citations

  • Liu, E. (2021). Innovative Simulation and Tree Models and Reinforcement Learning Methods with Applications in Cybersecurity [Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1618411811368024

    APA Style (7th edition)

  • Liu, Enhao. Innovative Simulation and Tree Models and Reinforcement Learning Methods with Applications in Cybersecurity. 2021. Ohio State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=osu1618411811368024.

    MLA Style (8th edition)

  • Liu, Enhao. "Innovative Simulation and Tree Models and Reinforcement Learning Methods with Applications in Cybersecurity." Doctoral dissertation, Ohio State University, 2021. http://rave.ohiolink.edu/etdc/view?acc_num=osu1618411811368024

    Chicago Manual of Style (17th edition)

osu1618411811368024
76
© 2021, all rights reserved.
This open access ETD is published by The Ohio State University and OhioLINK.