The spatial distribution of light intensity received by the eyes changes continually when an animal moves around in its environment. These retinal activity patterns contain a wealth of information on the structure of the environment, the direction and speed of self-motion, and on the independent motion of objects1,2. If evaluated properly by the nervous system this information can be used in visual orientation. In a combination of both behavioural and electrophysiological analysis and modelling, this article establishes the neural mechanisms by which the visual system of the fly evaluates two types of basic retinal motion patterns: coherent retinal large-field motion as induced by self-motion of the animal, and relative motion between objects and their background. Separate neuronal networks are specifically tuned to each of these motion patterns and make use of them in two different visual orientation tasks.