A pandemic caused by airborne pathogens raises a great need for N95 respirators and surgical masks. Subsequently, the risk of undersupply becomes a primary challenge requiring the prioritization of those masks for healthcare workers. Health agencies recommend wearing cloth masks in low-risk groups to reduce the demand. Unlike N95 respirators and surgical masks, cloth masks can be made from various fabrics, and their filtration performance becomes material-dependent. However, the existing literature presents limited and contradictory results on the property-performance relationship of fabrics used for cloth masks. Thus, the fundamental parameters determining the effectiveness of the fabrics remain unknown. Herein, we investigated the effects of yarn properties and multilayering on the filtration performance of single jersey fabrics. The fabrics performed up to 45% particle filtration efficiency, with the range of air permeability from 110–330 ft3/min/ft2. The results revealed that while the structural differences associated with the yarn choice had a smaller impact on the particle filtration efficiency of the fabrics compared to air permeability, their effects were great enough to yield statistically significant differences between the fabrics. In addition, our findings demonstrated that multilayering effectively improved the filtration performance of fabrics but resulted in a greater increase in airflow resistance than particle filtration efficiency. To limit the tradeoff between air permeability and particle filtration efficiency, yarn properties should be considered in the material selection of multilayer masks. We anticipate that our work will be a starting point for a guide on cloth masks with minimal filtration and breathability requirements.