Parametric regression models that describe the dependence of the mean of some response variable on a set of covariates play a fundamental role in statistics. Allowing for simple interpretation and estimation these models, however, are often not flexible enough for describing the data at hand. In the last 15 to 20 years with the development of computer technology and statistical software, another approach - nonparametric regression - has received more attention and recognition. The mean of a response is thereby modelled as a smooth, but otherwise unspecified function of covariates.

The large domain of nonparametric regression models includes local techniques like kernel or locally-weighted smoothers and spline methods. The main focus of this thesis is on penalized splines (P-splines), which have become a very powerful and applicable smoothing technique over the last decade. This nonparametric method can be viewed as a generalization of smoothing splines with a more flexible choice of bases and penalties. The main attraction of P-spline smoothing is its ties with ridge regression, mixed models and Bayesian statistics. This allows the adoption of different techniques, like Markov chain Monte Carlo or likelihood ratio tests for penalized spline methodology. Smoothing, in particular, can be performed with any mixed model or Bayesian software.

This thesis addresses several problems of nonparametric techniques that can be successively handled with penalized spline smoothing, due to its link to mixed models. First, smoothing in the presence of correlated errors is shown to be more robust if performed in the mixed models framework. This property is used to estimate the term structure of interest rates. Next, the problem of smoothing of locally heterogeneous functions is treated by representing the adaptive penalized splines as a hierarchical mixed model. Application of Laplace approximation for parameter estimation of this model results in the fast and efficient method for adaptive smoothing, which is implemented in the R package AdaptFit. Investigation of the asymptotic rate at which the spline basis dimension is supposed to grow to minimize mean squared error concludes the thesis.