PURPOSE: The training of highly skilled officers in rescue forces is essential for success and performance of fire brigades in their daily work. The purpose of this paper is to develop a validated instrument assessing the quality of leadership trainings in firefighter education. DESIGN/METHODOLOGY/APPRACH: In Study 1, relevant factors of teaching quality in this specific context are established using semi-structured interviews (n=5 trainer, n=59 trainees), and a pool of corresponding survey items is tested in a pilot sample (n=7 trainer, n=26 trainees). In Study 2 (n=263 trainees), we select best-fitting items and explore the structure of latent variables via exploratory factor analyses. Study 3 (n=45 trainer, n=380 trainees) tests this structure by means of confirmatory analyses and validates the questionnaire using scales from other evaluation instruments for higher education. FINDINGS:Analyses resulted in a six-dimensional questionnaire reflecting relevant training processes and outcomes. Results suggest that the newly created Feedback Instrument for Rescue forces Education (FIRE) meets all relevant psychometric quality criteria. ORIGINALITY/VALUE: By examining critical factors of training quality, the authors enhance the understanding of critical processes in programs for rescue forces education. The developed questionnaire provides trainers and educational institutions with a validated tool to measure these relevant processes and the desired training outcomes. Therefore, the FIRE scales can contribute to an ongoing improvement of rescue forces trainings.