Earthquake ground motions (GMs) may display distinct characteristics in all directions within the horizontal plane. However, the causes of GM polarization are still not fully understood. Structural designs use maximum rotated intensity measures (IMRotD100) to accommodate variations of the GM with orientation, but the orientation associated with IMRotD100, β, is not easily predictable. This study investigates the influence of linear site response to observed GM variability with direction. We analyze GMs recorded at the surface and at depth from four stations in the Japanese database, KiK-net. Selected events have moment magnitudes ranging 3–5, and rupture distances within 100 km. Findings provide evidence that site effects contribute to GM directionality, and that directional resonance can be observed at sites lacking significant topographic features. Additionally, values of β at depth are not correlated to the orientation corresponding to their expected polarization (from S-wave radiation patterns), which provides evidence of non-negligible path contributions to GM directionality observed at our study sites.