In this paper we describe communication with a responsive virtual environment with the main emphasis on the processing of spatial expressions in natural language instructions. This work is part of the VIENA project in which we chose interior design as an example domain. A multi-agent system acts as an intelligent mediator between the user and a graphics system. To make the communication about spatial relations more intuitive, we developed an anthropomorphic agent, which is graphically visualised in the scene. With reference to the human-like figure we explain the use of qualitative spatial expressions, like “right of” and “there”.