Recently, there has been increased interest in ma-chine learning explainability. Understanding the complex relationship between input features of a model and their respective outputs is of increased relevance, especially in biological science. In this paper, we introduce Optimal Brain Dissection (OBD), an innovative methodology designed to examine the importance of first-layer connections in a biology-inspired autoencoder. We incorporated regulator-target interactions within the first autoencoder layer, representing biological regulatory networks, and identified their importance to the reconstruction error, a critical aspect in navigating the complexity of high-dimensional omics data. Through a combination of pruning techniques and counterfactual reasoning, OBD offers a method to quantify feature importance, factoring in both weight magnitude and time-to-laziness. To implement this method, we propose a Dense Autoencoder (DAE) architecture, aiming for increased efficiency and reduced computation. Tailored for omics data, the DAE employs skip concatenations and circumvents non-existent target-target interactions. Our approach aims to understand the relative importance of connections for autoencoder performance, a critical step towards better counter-factual reasoning for neural networks.