In certain data analyses, the uncertainties associated with confounding or multicollinearity should result in a recommendation for additional data collection. In others cases, all plausible models lead to similar recommendations for action. This dissertation proposes a plotting technique to identify whether data limitations should preclude immediate recommendations. Specifically, the proposed “response-probability model analysis plots” (RPMAPs) show the probabilities of models being accurate versus box and whisker plots of the system responses of applying each model in related optimizations. The associated optimization formulations divide into three types. First, in the context of fractional factorial experiments, the decision-maker faces complete confounding of interactions but also the freedom to adjust all factor settings for response optimization which might involve engineering specification limits. The motivating applications here include real world data sets in injection molding and arc welding. The resulting recommendations range from collecting additional runs and exploring new factors. Second, the decision-maker is challenged by so-called “noise factors” that can only be controlled during experimentation but not during the normal system operations. Here, RPMAPs compare with Taguchi signal-to-noise ratio-based marginal plots. The motivating study here involves arc welding yield maximization and response-probability model analysis plots offer advantages in interpretability for multi-response optimization. The third type of optimization formulation relates to decision-making in the context of on-hand data. In these cases, the least squares estimates are generally not trustworthy because of multicollinearity and biasing interactions. As a result, the proposed response-probability model analysis plots are based on Bayesian shrinkage estimates. Also, the associated formulations are control policies because decision-makers can observe noise factor settings and (open full item for complete abstract)