Perception and generation of verbal and nonverbal behavior is one of the main foundations of human social interaction. We model these abilities for embodied conversational agents (ECAs) on the basis of perception-action links as in humans. With a focus on gesture processing, we propose a computational model which enables ECAs to interact with humans in an embodied manner and supports many aspects of social interaction. The model performance is briefly illustrated on the basis of an interaction scene.