Humans make extensive use of specialized representations to remember people they interacted with. While current research on embodied conversational agents focuses on the relationship between agent and interlocutor, the representation of the latter is mostly neglected. But information about others are inevitable for an agent to adapt to its interlocutors and to establish long-term relationships with them. In this work, we present a model

of Person Memory for virtual agents. We discuss what kinds of information have to be stored about people. Furthermore, we stress the importance of social categories. In our scenario, we focus on first encounters between our agent and people. We show how the agent is able to exploit his Person Memory to acquire information about others during Small Talk and guide the conversation.