Logistic Regression - Classifying with Probabilities

Logistic Regression differs from linear regression in that rather than predicting numerical values, it is a machine learning algorithm that classifies data into specific categories.

For example, Logistic Regression can be used to answer the question "Is this email a spam?" by classifying it as either 0 (not spam) or 1 (spam).

Logistic Regression not only outputs a 0 or 1 but rather calculates the probability that the email is spam, and if it exceeds a certain threshold, it is classified into that category.

Concept of Logistic Regression

Logistic Regression, like linear regression, learns weights ($W$) and a bias ($B$) for the input data, but transforms the result into a probability value between 0 and 1.

The function used for this transformation is the Sigmoid function.

What is the Sigmoid Function?

Logistic Regression takes the output of a linear regression and applies the following Sigmoid function to convert it into a probability value between 0 and 1.

It's okay if you don't fully understand the formula below. Understanding how logistic regression is structured like linear regression is the key takeaway.

$$\sigma(z) = \frac{1}{1 + e^{-z}}$$

Here, $z$ is calculated in the same manner as in linear regression.

$$z = W X + B$$

Consequently, the final prediction of Logistic Regression can be expressed as follows.

$$P(Y=1|X) = \frac{1}{1 + e^{-(WX + B)}}$$

This value is converted into a probability between 0 and 1, and classification is determined based on a specific threshold (commonly 0.5).

Example of Prediction

Input X = Email content
Prediction P(Y=1|X) = 0.85 → 85% probability of being spam

Logistic Regression is used in various real-world classification problems such as disease prediction and credit card fraud detection.

In the next lesson, we will explore Decision Trees.

Want to learn more?

Join CodeFriends Plus membership or enroll in a course to start your journey.

Learn moreSign In