B Linear smoother (Buja, Hastie, and Tibshirani 1989)

One can also show that the smoothing spline is a linear smoother, and hence we can write down a smoother matrix. The following is taken from (Green and Yandell 1985).

Let \(h_i = x_{i+1} - x_i\), \(i = 1,2,\dots,n-1\), \(\Delta\) be a tridiagonal \((n-2) \times n\) matrix such as \[\Delta_{ii} = \frac{1}{h_i}, \hspace{.5cm} \Delta_{i,i+1} = -(\frac{1}{h_i}+\frac{1}{h_{i+1}}) , \hspace{.5cm} \Delta_{i,i+2} = \frac{1}{h_{i+1}},\] and let \(\mathbf{C}\) be a symnetric tridiagonal matrix of order \(n-2\) with :

\[C_{i-1,i} = C_{i,i-1} = \frac{h_i}{6}, \hspace{.5cm} C_{ii} = \frac{h_i+h_{i+1}}{3},\] Then is can be showned that solving \[\sum_{i=1}^n||y_i - s(x_i)||^2 + \lambda \int_a^b{s''(t)}^2dt,\] is equivalent to minimizing \[|| \mathbf{y} - s(\mathbf{x})||^2 + \lambda s(\mathbf{x}) \mathbf{K} s(\mathbf{x}) \hspace{.5cm} \text{where } \mathbf{K} = \Delta^T \mathbf{C}^{-1} \Delta\] with solution \(\hat{\mathbf{y}} = \hat{s}(\mathbf{x}) = \mathbf{S}\mathbf{y}\), where \(\mathbf{S} = (\mathbf{I}+\lambda \mathbf{K})^{-1}\).

References

Buja, Andreas, Trevor Hastie, and Robert Tibshirani. 1989. “Linear Smoothers and Additive Models.” The Annals of Statistics 17 (2): 453–510. http://www.jstor.org/stable/2241560.

Green, Peter J., and Brian S. Yandell. 1985. Semi-Parametric Generalized Linear Models. Lecture Notes in Statistics. Springer, New York, NY. https://link.springer.com/chapter/10.1007/978-1-4615-7070-7_6.