Introduction to Multiple Linear Regression (MLR)

Multiple Linear Regression (MLR) is an extension of Simple Linear Regression (SLR) that involves predicting a dependent variable based on multiple independent variables. In MLR, the relationship between the dependent variable and multiple predictors is modeled linearly. The general form of MLR can be expressed as:

[ Y = B0 + B1X1 + B2X2 +……….. + BnXn +e ]

Where:

  • ( Y ) is the dependent variable.
  • ( X1, X2, …., Xn ) are the independent variables.
  • ( B0, B1, ……, Bn ) are the coefficients.
  • ( e ) is the error term.

The objective of MLR is to estimate the coefficients that minimize the difference between the predicted and actual values.


Code of MLR

Below is a simple Python implementation of Multiple Linear Regression using the scikit-learn library:

from sklearn.linear_model import LinearRegression
import numpy as np

# Sample data
X = np.array([[1, 2], [2, 4], [3, 6], [4, 8]])
y = np.array([3, 6, 9, 12])

# Create a linear regression model
model = LinearRegression()

# Fit the model to the data
model.fit(X, y)

# Print the coefficients and intercept
print("Coefficients:", model.coef_)
print("Intercept:", model.intercept_)

This code uses scikit-learn’s LinearRegression class to perform MLR on a small dataset with two independent variables.


Mathematical Formulation of MLR


Error Function of MLR


Minimizing Error

Minimizing the error in MLR involves finding the values of the coefficients that minimize the error function, such as the sum of squared errors. This process is often done using optimization techniques, and scikit-learn’s LinearRegression class uses the least squares method to find the optimal coefficients.


Error Function Continued

Continuing from the error function explanation, after obtaining the MLR model, one can evaluate its performance using various metrics. Common evaluation metrics include:

  • Mean Squared Error (MSE)
  • Root Mean Squared Error (RMSE)
  • R-squared (( R^2 ))

These metrics provide insights into how well the MLR model fits the data and how much variance in the dependent variable is explained by the independent variables.


Code from Scratch

Below is a simplified implementation of MLR from scratch using Python and NumPy:

import numpy as np

def multiple_linear_regression(X, y):
    # Add a column of ones to X for the intercept term
    X = np.c_[np.ones(X.shape[0]), X]

    # Calculate coefficients using the normal equation
    beta = np.linalg.inv(X.T @ X) @ X.T @ y

    return beta

# Sample data
X = np.array([[1, 2], [2, 4], [3, 6], [4, 8]])
y = np.array([3, 6, 9, 12])

# Add intercept term to X and perform MLR
X_with_intercept = np.c_[np.ones(X.shape[0]), X]
coefficients = multiple_linear_regression(X_with_intercept, y)

# Print the coefficients
print("Intercept:", coefficients[0])
print("Coefficients:", coefficients[1:])

This code implements the MLR calculation using the normal equation, including the addition of an intercept term to the independent variables.

Keep in mind that the scikit-learn library provides efficient and optimized implementations for MLR, and the purpose of this scratch implementation is for educational understanding of the underlying principles.

6 Replies to “Multiple Linear Regression (MLR)”

  1. Откройте двери в мир недвижимости с нашим ресурсом!
    Прочтите экспертные статьи от профессионалов в области недвижимости. Узнавайте о налогах на недвижимость, осуществляйте успешные сделки купли-продажи квартир, и всегда оставайтесь в курсе последних новостей рынка.
    http://ufficioporta.ru

Leave a Reply

Your email address will not be published. Required fields are marked *