This study investigates a specific aspect of joint attention, that is, the emergence of the sense that one is leading the attentional focus of others. Thirty participants were placed in front of two avatars and had to pay attention to objects that were also attended to by the avatars. Unbeknownst to the participant, the avatars’ gaze orientations were alternately controlled by the participant’s eyes. Eye-tracking data were collected and participants were enquired about their experience to account for their sense of agency. Only five participants noticed that the avatars were following their gaze. The 25 participants who remained unaware that they controlled the avatars nevertheless modified their gazing behavior with an increase in direct gaze toward one of the avatars. This outcome highlights a dissociation between an implicit feeling that a change occurred and the explicit awareness of being the cause of this change.