Machine learning (ML) and artificial intelligence (AI) applications have become crucial for current and future information systems. Meanwhile, hardware security threats are emerging for AI/ML applications, such as the possibility of private input/model leakage as a result of hardware side-channel leakage. Yet such vulnerabilities are only evaluated after deployment and as ad-hoc instances, which is too late and too costly. The development of a framework is necessary in order to evaluate attacks and defenses comprehensively, quickly, and accurately prior to their deployment.