The networked structure of the internet facilitates the dissemination of extremist messages and often makes removal impossible. Equipping media users with critical (preventive) skills appears a more promising strategy than trying to block any exposure to extremist messages. Although various prevention programs follow this approach, research on their impact and success is scarce. This contribution describes how prevention programs can benefit from systematic evaluation using the example of the CONTRA school program, which seeks to foster critical media literacy with regard to extremist online messages via three sub-dimensions: awareness, reflection, and empowerment. The feasibility, applicability and impact of methods and measures developed in the context of CONTRA were assessed using an innovative mixed method design. The contribution describes the scientific approach of the prevention program, outlines the results of the evaluation and extrapolates from this prevention program to the general question of best practices for prevention programs with a specific focus on the school context.