We review distinct features arising from the unique nature of the carrier-phonon coupling in self-assembled semiconductor quantum dots. Because of the discrete electronic energy structure, the pure dephasing coupling usually dominates the phonon effects, of which two properties are of key importance: The resonant nature of the dot-phonon coupling, i.e. its non-monotonic behavior as a function of energy, and the fact that it is of super-Ohmic type. Phonons do not only act destructively in quantum dots by introducing dephasing, they also offer new opportunities, e.g. in state preparation protocols. Apart from being an interesting model systems for studying fundamental physical aspects, quantum dot and quantum dot-microcavity systems are a hotspot for many innovative applications. We discuss recent developments related to the decisive impact of phonons on key figures of merit of photonic devices like single or entangled photon sources under aspects like indistinguishability, purity and brightness. All in all it follows that understanding and controlling the carrier-phonon interaction in semiconductor quantum dots is vital for their usage in quantum information technology.