The manufacturing industry is increasingly being dominated by information and communication technology, leading to the development of cyber-physical systems. Most existing frameworks on the assessment of such technological advancements see the technology as a solitary system. However, research has shown that other environmental factors like organizational processes or human factors are also affected. Drawing on the sociotechnical systems approach, future technologies could be evaluated using scenarios of digitized work. These scenarios can help classify new technologies and uncover their advantages and constraints in order to provide guidance for the digital development of organizations. We developed an instrument for evaluating scenarios of digitized work on the relevant dimensions ‘technology’, ‘human’ and ‘organization’ and conducted a quantitative study applying this instrument on three different scenarios (N = 24 subject matter experts). Results show that our instrument is capable of measuring technological, human and organizational aspects of technology implementations and detecting differences in the scenarios under investigation. The instrument’s practical value is significant as it enables the user to compare and quantify scenarios and helps companies to decide which technology they should implement.