# k Nearest Neighbours: k = 1

## Intro

Nearest Neighbour algorithms can be used for **classification** and **regression**.

Suppose we have the following training set:

We need to predict the *label* for a test object .

The algorithm will

- Search for the
*nearest*training object to the test object - Take the label of the nearest training object as the prediction for the label of the test object.

## Binary Classification Problem

When our label space contains only two distinct values, this is **binary classification**.

The label space could be something like or , for instance. It doesnâ€™t matter as long as we can distingish the two.

## Euclidean Distance

In this example our label space is . Our training set consists of the following training objects:

Positive Objects:

Negative Objects:

Test Object:

We will use **Euclidean Distance** as our distance measure. We need to
compute the Euclidean Distance between the test object and every training object.

To do this, we subtract vectors and compute the Euclidean norm (e.g. )

The Euclidean Distance between and the test object is computed like so:

Eventually, we have the Euclidean Distance for all training objects:

Now we make our prediction. We take the label of the object with the smallest Euclidean Distance. This is , which has the label .