While the number of studies of the non-medical use of prescription drugs to augment cognitive functions is growing steadily, psychological factors that can potentially help explain variance in such pharmaceutical cognitive enhancement (CE) behavior are often neglected in research. This study investigates the association between the Big Five personality traits and a retrospective (prior CE-drug use) as well as a prospective (willingness to use CE drugs) measure of taking prescription drugs with the purpose of augmenting one's cognitive functions (e.g., concentration, memory, or vigilance) without medical necessity. We use data from a large representative survey of German employees (N = 6454, response rate = 29.8%). The Five Factor Model (FFM) of Personality was measured with a short version of the Big Five Personality Traits Inventory (BFI-S), which includes: openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism. Together with this, demographic variables such as gender, age, education, and income were used as potential confounders in multiple logistic regression models. Our results show a 2.96% lifetime prevalence of CE-drug use and a 10.45% willingness to (re)use such drugs in the future. We found that less conscientious and more neurotic respondents have a higher probability of prior CE-drug use and a greater willingness to use CE drugs in the future. No significant effects were found for openness, extraversion, or agreeableness. Prior CE-drug use was strongly associated with a greater willingness to take such drugs in the future. This study shows that specific personality traits are not only associated with prior enhancement behavior, but also affect the willingness to (re)use such drugs. It helps increase understanding of the risk factors of CE-drug use, which is a health-related behavior that can entail severe side-effects for consumers. The knowledge gathered can thus help improve interventions aimed at minimizing health problems.