K-Nearest Neighbor Classifier — Implement Homemade Class & Compare with Sklearn Import

What is K-Nearest Neighbors?

Courtesy of Datacamp

K-Nearest Neighbors is an algorithm used in classification and regression problems; however, this example application uses the algorithm as a classifier. K-nn is a form supervised learning where classification outputs are class membership and is considered lazy learning. This object in question is assigned a class based on the classes of its neighbors. This can be modified by alternating the number of neighbors provided (K). In regression the outputs are the property values for the specified objects. The value is the average of the values of k nearest neighbors.

~Objective~

The goal here was to create a model that takes in row values, computes the Euclidean distance between one another and output a prediction of the proper class. Then the results were compared to the sklearn model that can be imported with the Scikit-Learn library.

~Implementation~

The class was implemented to fit the data, predict on the data, and display the data. There is a class definition that calculates the Euclidean distance used for the math stuff. Next the input data is fit into an X_train and y_train (target). Then doing a bunch of fancy math, calculating distances between points, class predictions, analyzing neighbors, and more; the model is able to predict a class. Lastly, a quick function was created to be able to display the data similar to the Sklearn model such that comparison can be easy for the three datasets tested on. These datasets are Iris, Breast Cancer, and Wine datasets from Sklearn. The results and comparison between the imported model and my model are shown below for the listed datasets:

Iris Dataset Comparison

Breast Cancer Dataset Comparison

Wine Dataset Comparison

Results indicate my model performs as well if not better in most rudimentary basics. Please feel free to take this code to the next level and see what you can make of it! Code can be found for the model, testing, and blog are on Github. The class implementation can be seen below:

# Imports
import numpy as np
# Class
class k_nn:
# Calc euclidean distance to compare neighbors: Fit - predict - display
def __init__(self, num_neighbors=5):
"""Init definition"""
self.num_neighbors = num_neighbors
# Euclidean distance
def euclidean_distance(self, a, b):
"""Returns euclidean distance between rows"""
euclidean_distance_sum = 0.0  # initial value
for i in range(len(a)):
euclidean_distance_sum += (a[i] - b[i]) ** 2
"""Subtract - square - add to euclidean_distance_sum"""
euclidean_distance = np.sqrt(euclidean_distance_sum)
return euclidean_distance
# Fit k Nearest Neighbors
def fit_knn(self, X_train, y_train):
"""Fits the model using training data. X_train and y_train inputs for func"""
self.X_train = X_train
self.y_train = y_train
# Predict X for kNN
def predict_knn(self, X):
"""Return predictions for X based on the fit X_train and y_train data"""
# initialize prediction_knn as empty list
prediction_knn = []
for i in range(len(X)):
# initialize euclidean_distance as empty list
euclidean_distance = []
for row in self.X_train:
# find eucl_distance to X using
# euclidean_distance() function call and append to euclidean_distance list
euclidean_distance_sum = self.euclidean_distance(row, X[i])
euclidean_distance.append(euclidean_distance_sum)
neighbors = np.array(euclidean_distance).argsort()[: self.num_neighbors]
# initialize dict to count class occurrences in y_train
neighbor_count = {}
for num in neighbors:
if self.y_train[num] in neighbor_count:
neighbor_count[self.y_train[num]] += 1
else:
neighbor_count[self.y_train[num]] = 1
# max count labels to prediction_knn
prediction_knn.append(max(neighbor_count, key=neighbor_count.get))
return prediction_knn
# display list of nearest_neighbors & euclidian dist

def display_knn(self, x):
"""Inputs -- x // outputs a list w/ nearest neighbors and euclidean distance."""
# initialize euclidean_distance as empty list
euclidean_distance = []
for row in self.X_train:
euclidean_distance_sum = self.euclidean_distance(row, x)
euclidean_distance.append(euclidean_distance_sum)
neighbors = np.array(euclidean_distance).argsort()[: self.num_neighbors]
# empty display_knn_values list
display_knn_values = []
for i in range(len(neighbors)):
n_i = neighbors[i]
e_dist = euclidean_distance[i]
display_knn_values.append((n_i, e_dist))  # changed to list of tuples
return display_knn_values