We study how the human visual system processes visual information to allow successful interactions with the environment. Our approach is to combine computational methods and behavioral studies to understand what are the visual features that establish the mapping between vision and action. Contrary to the commonly held assumption that perception and action stem from separate visual mechanisms, we take a fundamentally different view, proposing that perception and action form a coordinated system: Perception informs action about the state of the world and, in turn, action shapes perception by signaling when it is faulty.