The transmission of light through sub-wavelength apertures (zero-mode waveguides, ZMW) in metal films is well-explored. It introduces both an amplitude modulation as well as a phase shift to the oscillating electromagnetic field. We propose a nanophotonic interferometer by bringing two ZMW (∼100 nm diameter) in proximity and monitoring the distribution of transmitted light in the back-focal plane of collecting microscope objective (1.3 N.A.). We demonstrate that both an asymmetry induced by the binding of a quantum dot in one of the two ZMW, as well as an asymmetry in ZMW diameter yield qualitatively similar transmission patterns. We find that the complex pattern can be quantified through a scalar measure of asymmetry along the symmetry axis of the aperture pair. In a combined experimental and computational exploration of detectors with differing ZMW diameters, we find that the scalar asymmetry is a monotonous function of the diameter difference of the two apertures, and that the scalar asymmetry measure is higher if the sample is slightly displaced from the focal plane of the collecting microscope objective. An optimization of the detector geometry determined that the maximum response is achieved at an aperture separation that is comparable to the wavelength on the exit side of the sensor. For small separations of apertures, on the order of a quarter of the wavelength and less, the signal is strongly polarization dependent, while for larger separations, on the order of the wavelength or larger, the signal becomes essentially polarization-independent.