2.1 Notations

Let \(\mathcal{T}\) be a training set consisting of \(n\) pairs of examples labelled on a space \(\mathcal{Z} = \mathcal{X} \times \mathcal{Y}\): \(\mathcal{T} = \{(\boldsymbol{x}_{i}, y_{i})\}_{i=1}^{n}\), with \((\boldsymbol{x}_{i}, y_{i}) \in \mathcal{Z}\), \(\forall i\). Each couple \((\boldsymbol{x}_{i}, y_{i})\) is the realization of an \(i^{th}\) independent copy of a random vector couple \((\mathrm{X}, \mathrm{Y})\) distributed according to \(\mathcal{U}\), an unknown but fixed distribution on \(\mathcal{Z}\).

Generally, we will represent the vectors in bold lowercase letters and the matrices in bold capital letters. Thus, when \(\mathcal{X} \in \mathbb{R}^D\), the vector of the \(i^{th}\) component of \(\mathcal{T}\) represented by \(D\) variables, will be designated by the column vector \(\boldsymbol{x}_i = (x_{i1}, \dots, x_{id}, \dots, x_{iD})^T \in \mathbb{R}^D\) and its associated matrix by \(\mathbf{X} = (\boldsymbol{x}_1^T, \dots, \boldsymbol{x}_i^T, \dots, \boldsymbol{x}_n^T)^T \in \mathbb{R}^{n \times D}\).