Skip to main content
Practice

Evaluating Classification Models

Model evaluation is the process of measuring how well a trained model makes predictions.

The choice of evaluation metrics depends on the type of problem:

  • Classification: Accuracy, Precision, Recall, F1-score
  • Regression: R² (Coefficient of Determination), MSE, MAE

True Positive, True Negative, False Positive, False Negative​

When evaluating classification models, these terms are commonly used:

  • True Positive (TP): Correctly predicting positive cases (e.g., predicting a woman is pregnant when she actually is)
  • True Negative (TN): Correctly predicting negative cases (e.g., predicting a woman is not pregnant when she actually isn’t)
  • False Positive (FP): Incorrectly predicting positive cases (e.g., predicting a woman is pregnant when she isn’t)
  • False Negative (FN): Incorrectly predicting negative cases (e.g., predicting a woman is not pregnant when she actually is)

Classification Metrics​

Commonly used evaluation metrics for classification models include:

  • Accuracy: The ratio of correct predictions to total predictions Formula: (TP + TN) / (TP + TN + FP + FN)
  • Precision: The proportion of positive predictions that are actually correct Formula: TP / (TP + FP)
  • Recall: The proportion of actual positives that are correctly identified Formula: TP / (TP + FN)
  • F1-score: The harmonic mean of precision and recall Formula: 2 * (precision * recall) / (precision + recall)

Scikit-learn provides built-in functions to calculate these metrics easily.


Example: Calculating Accuracy Score​

The following example shows how to evaluate a classification model using accuracy:

Accuracy Example
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

# Load dataset (Iris dataset)
X, y = load_iris(return_X_y=True)

# Split data into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Train the classifier
knn = KNeighborsClassifier(n_neighbors=3)
knn.fit(X_train, y_train)

# Make predictions
y_pred = knn.predict(X_test)

# Evaluate accuracy
acc = accuracy_score(y_test, y_pred)
print(f"Accuracy: {acc:.2f}")

In this example, the accuracy_score() function is used to compute the accuracy.

Similarly, Scikit-learn provides APIs to calculate other commonly used classification metrics.

Want to learn more?

Join CodeFriends Plus membership or enroll in a course to start your journey.