Colloidal particles with anisotropic geometries and interactions display rich phase behavior and hence have the potential to serve as the basis of functional materials, which can tunably and reversibly self-assemble into different configurations. External fields are one design parameter that can be used to manipulate how systems of colloidal particles assemble with one another. One challenge in designing new materials using anisotropic colloidal particles is understanding how an individual particle's various anisotropic features, like geometry, affect their overall self-assembly. Here, we present the results of simulation studies that explore the self-assembly of 2D colloidal squares with offset magnetic dipoles in the presence of an external field. Annealing simulations are used to measure the equilibrium-phase behavior of systems of these particles in the ground state, when the magnetic interactions dominate over the thermal forces of the system. We find that the magnetic properties of these systems are strongly influenced by the relative number of squares with opposite "handedness", or chirality, that are present within the system. Systems of squares that contain equal numbers of either chirality are extremely responsive to the external field; a relatively weak external field is required to magnetize them. In contrast, systems that contain only one chirality of squares are significantly less responsive to the external field; a significantly stronger external field is required to elicit the same magnetic response. Ultimately, the differing macroscopic magnetic properties of these systems are related to their microscopic self-assembly in an external field. Simulation snapshots and ground state phase diagrams illustrate how the absence of opposite chirality squares prevents systems of these particles from leaving an energetically favorable antiparallel configuration in the presence of an external field. When opposite chirality squares are present, these magnetic particles assemble into a head-to-tail configuration, therefore inducing a magnetic state.