Since members of rescue forces such as firefighters have to deal with sometimes extreme and dangerous situations, high-quality basic trainings are indispensable for their professional success. There is therefore an obvious need for standardized tools assessing the training quality. This paper aims to develop and validate such an evaluation instrument. In Study 1, a qualitative analysis (N = 21) was used to identify core characteristics of good firefighter basic trainings and served as theoretical basis for the generation of corresponding items. In Study 2 (N = 257), the item set was piloted and reduced, its structure was assessed in exploratory factor analyses, and first validations were conducted. Study 3 (N = 451) tested the proposed factor structure via confirmatory analyses and validated the questionnaire comprehensively. Factor analyses showed a six-factor structure. The scales of the newly created Feedback Instrument for Rescue forces Education – Basic education (FIRE-B) are to be judged as reliable. Moreover, there are several clear indications of validity. Thus, the present research contributes to the understanding of critical factors and processes of basic trainings. Furthermore, the FIRE-B has a high practical relevance, both in the assessment of training quality and in the identification of opportunities for improvement.