Conversational voice assistants that are available in multiple countries need to be able to generate utterances in the language that their users speak. In open domains, in which messages can be variable, pre-writing and translating all utterances in advance is unfeasible because it is costly, error-prone, and inflexible when changes need to be made. Approaches to automatically generate multilingual surface forms of utterances have been developed in the field of Natural Language Generation (NLG), however, these are rarely used when developing skills for conversational voice assistants. In this paper, we describe an evaluation study that analyses the feasibility of integrating NLG surface-realization frameworks (SimpleNLG and RosaeNLG) into the development process of an existing commercial and multilingual (English, French, German, Italian, Spanish) home-automation skill, and compare it to a more traditional localization approach. The study uses methods and measures from human–computer interaction and software engineering, and takes into account the perspective of various stakeholders in the development process (conversation designers, language experts and developers).