AbstractAbstractPromptly predicting defects during an additive manufacturing process using only copious log data provides many advantages, albeit with computational limitations. We focus on predicting defects during electron beam melting with the black box nature of the manufacturing machine. For an accurate prediction of defects, which are rare (<2%), we extract temporal information to track abnormalities and formulate a feature selection algorithm that maximizes the expected value of a cost-sensitive accuracy. Correct identification of features responsible for the defects increases predictive power and informs manufacturers of potential corrective/preventive actions for process improvement. We solve the feature selection through resampling strategies integrated with ensemble procedures to handle data uncertainty and imbalance. Exploiting data uncertainty in our search leads to finding robust features with consistent predictive power. Our proposed methodology shows a 43% improvement in predicting defects (recall) without losing precision. Beyond additive manufacturing, this methodology has general application for rare-event prediction and imbalanced datasets.Keywords: Smart manufacturingfeature extractionsimulation optimizationrobust predictiondata uncertainty AcknowledgementsThe authors would also like to thank guiding notes from Dr.'s Chris Rock, Carter Keough, and Xiaolei Fang, and help of Santiago Volonte Morales, Hisham Abu Nimeh, and Carolyn Drahuse.Data availability statementThe data that support the findings of this study are available from the corresponding author, [EH], upon reasonable request.Additional informationFundingThis work was partially supported by National Science Foundation10.13039/100000001 Grant CMMI-2226347. This work was supported by Research Experience for Undergraduates (REU) Grant from North Carolina State University.Notes on contributorsEthan HouserEthan Houser is a master's student in the Edward P. Fitts Department of Industrial and Systems Engineering at North Carolina State University (NCSU). He also earned his BS in Industrial and Systems Engineering with a Statistics minor at NCSU. His research interests focus on applying simulation optimization and machine learning techniques to develop feature selection methodologies with an emphasis on high-dimensional binary classification problems.Sara ShashaaniSara Shashaani is an assistant professor in the Edward P. Fitts Department of Industrial and Systems Engineering at North Carolina State University. Her research interests lie in the intersection of simulation, analytics, and stochastic optimization with a focus on continuous zeroth-order problems as well as high-dimensional binary decision spaces. Quantifying bias in the model outputs with nonparametric approaches is another direction of her research, seeking to broaden the advantage of Monte Carlo methodology for machine learning. She is an active member and elected treasurer of the INFORMS Simulation Society and the co-creator of SimOpt: the open-source library and benchmarking platform for simulation optimization. Her research has contributed to renewable energy, climate adaptation, advanced manufacturing, and public health application areas.Ola HarryssonDr. Ola Harrysson is the Edward P. Fitts Distinguished Professor in the Department of Industrial and Systems Engineering at North Carolina State University. Dr. Harrysson is the Director of the Center for Additive Manufacturing and Logistics (CAMAL) housed by the ISE department, which houses the first Electron Beam Melting (EBM) machine in the world. CAMAL is currently involved in research with numerous industries as well as providing faculty and students with prototyping services. Most of the aerospace-related research involves process and material development for the EBM systems while the medical research is concentrated on custom design and direct metal fabrication of novel implants and development of new treatment methods. Dr. Harrysson holds faculty positions in the Biomedical Engineering Department and the Material Science and Engineering Department at NCSU.Yongseok JeonYongseok Jeon is a PhD student in the Edward P. Fitts Department of Industrial and Systems Engineering at North Carolina State University. He earned his BS and MS degree in Industrial Engineering from Sungkyunkwan University (SKKU). His research focuses on developing data-driven optimization and machine learning methods with application in real-world problems to improve decision-making and efficiency.