The identification of fatigue failure can significantly affect the determination and prediction of fatigue life for a given asphalt binder. In this work, different approaches for identifying the fatigue failure on linear amplitude sweep (LAS) and time sweep (TS) are compared, especially for highly-aged asphalt binders. The candidate methods include traditional 50% or 80% reduction in pseudo-stiffness parameter (denoted as C0.5 and C0.2), peak stored pseudo-strain energy (PSE), peak C × N and peak phase angle. For asphalt binder at multiple aging levels, peak C × N is found to be effective in defining fatigue failure of TS testing. Meanwhile, peak stored PSE is suitable as the failure definition of LAS because it can capture the benefit of polymer modification and has a high accuracy for fatigue life prediction. Nf in strain-control TS test determined by different failure definitions shows that aging can increase the fatigue resistance at early-aged levels and then decrease the fatigue resistance at highly-aged levels. In general, the selected failure definitions have little influence on the trend of Nf with age levels. The purpose of this work is to select the logical failure definitions of asphalt binder under multiple aging levels.