The present study introduces a covert eye-tracking procedure as an innovative approach to investigate the adequacy of research paradigms used in psychology. In light of the ongoing debate regarding ego depletion, the frequently used “attention-control video task” was chosen to illustrate the method. Most participants did not guess that their eyes had been monitored, but some participants had to be excluded due to poor tracking ratio. The eye-tracking data revealed that the attention-control instructions had a significant impact on the number of fixations, revisits, fixation durations, and proportion of long fixation durations on the AOIs (all BF10 > 18.2). However, number of fixations and proportions of long fixation durations did not mediate cognitive performance. The results illustrate the promise of covert eye-tracking methodology to assess task compliance, as well as adding to the current discussion regarding whether the difficulties of replicating “ego depletion” may be in part due to poor task compliance in the video task.