It is easier to indicate the ink color of a color-neutral noun when it is presented in the color in which it has frequently been shown before, relative to print colors in which it has been shown less often. This phenomenon is known as color-word contingency learning. It remains unclear whether participants actually learn semantic (word-color) associations and/or response (word-button) associations. We present a novel variant of the paradigm that can disentangle semantic and response learning, because word-color and word-button associations are manipulated independently. In four experiments, each involving four daily sessions, pseudowords—such as enas, fatu or imot—were probabilistically associated with either a particular color, a particular response-button position, or both. Neutral trials without color-pseudoword association were also included, and participants’ awareness of the contingencies was manipulated. The data showed no influence of explicit contingency awareness, but clear evidence both for response learning and for semantic learning, with effects emerging swiftly. Deeper processing of color information, with color words presented in black instead of color patches to indicate response-button positions, resulted in stronger effects, both for semantic and response learning. Our data add a crucial piece of evidence lacking so far in color-word contingency learning studies: Semantic learning effectively takes place even when associations are learned in an incidental way.