Faced with a rapidly growing electric vehicle (EV) load, distribution planners need a methodology to forecast when and where these loads will likely appear on specific distribution feeders. This paper proposes a new diffusion forecasting approach for residential EV and charging stations using agent-based modeling. Residential EV adoption is treated as a multi-criteria decision making problem modeled via analytic hierarchy process (AHP). The customer adoption model CANE is developed considering customer characteristics including car age, EV attractiveness valued using logistic regression, neighbor influences and customer economics. Distribution feeder topology is combined with property geographic information system (GIS) parcels and household travel survey data to determine geographic and electric locations for EV and charging stations. A Monte Carlo method is utilized to obtain the most likely outcome of the stochastic vehicle decision-making process and perform model calibration. The proposed diffusion forecasting approach is demonstrated using actual distribution feeder data. Using the diffusion results, EV impact on system annual peak, energy, losses, transformer aging and feeder upgrades is evaluated using quasi-static time-series power flow analysis. Case analyses are presented that examine the effect of EV price and charging station placement on EV diffusion and distribution feeder impact.