Maharashtra Board Class 12 Maths Commerce Part II Chapter 3 Linear Regression PDF Download

Linear Regression

Let's Study

Meaning and Types of Regression

Fitting Simple Linear Regression

Least Square Method

Regression of Y on X

Regression of X on Y

Properties of Regression Coefficients

Let's Recall

Concept of Correlation

Coefficient of Correlation

Interpretation of Correlation

Introduction

We have already learned that correlation is used to measure the strength and direction of association between two variables. In statistics, correlation denotes association between two quantitative variables. It is assumed that this association is linear. That is, one variable increases or decreases by a fixed amount for every unit of increase or decrease in the other variable. Consider the relationship between the two variables in each of the following examples.

Advertising and sales of a product. Positive correlation.

Height and weight of a primary school student. Positive correlation.

The amount of fertilizer and the amount of crop yield. Positive correlation.

Duration of exercise and weight loss. Positive correlation.

Demand and price of a commodity. Positive correlation.

Income and consumption. Positive correlation.

Supply and price of a commodity. Negative correlation.

Number of days of absence in school and performance in examination. Negative correlation.

The more vitamins one consumes, the less likely one is to have a deficiency. Negative correlation.

Correlation coefficient measures association between two variables but cannot determine the value of one variable when the value of the other variable is known or given. The technique used for predicting the value of one variable for a given value of the other variable is called regression. Regression is a statistical tool for investigating the relationship between variables. It is frequently used to predict the value and to identify factors that cause an outcome. Karl Pearson defined the coefficient of correlation known as Pearson's Product Moment Correlation Coefficient. Carl Friedrich Gauss developed the method known as the Least Squares Method for finding the linear equation that best describes the relationship between two or more variables. R.A. Fisher combined the work of Gauss and Pearson to develop the complete theory of least squares estimation in linear regression. Due to Fisher's work, linear regression is used for prediction and understanding correlations.

Note: Some statistical methods attempt to determine the value of an unknown quantity, which may be a parameter or a random variable. The method used for this purpose is called estimation if the unknown quantity is a parameter, and prediction if the unknown quantity is a variable.

Teacher's Note

Regression helps us predict one thing from another. For example, we can predict a student's marks in Hindi based on their marks in English.

Exam Trick

Remember: Regression is about prediction. If correlation = 0, then regression cannot predict. Always check correlation first before using regression.

Points to Remember

Correlation only measures the strength between two variables.
Regression is used to predict one variable from another.
Two variables must have a linear relationship for regression to work well.
Karl Pearson and Carl Friedrich Gauss developed the methods we use today.
Prediction is for unknown random values, and estimation is for unknown parameters.

3.1 Meaning and Types of Regression

Meaning of Regression

Linear regression is a method of predicting the value of one variable when the values of all other variables are known or specified. The variable being predicted is called the response or dependent variable. The variables used for predicting the response or dependent variable are called predictors or independent variables. Linear regression proposes that the relationship between two or more variables is described by a linear equation. The linear equation used for this purpose is called a linear regression model. A linear regression model consists of a linear equation with unknown coefficients. The unknown coefficients in the linear regression model are called parameters of the linear regression model. Observed values of the variables are used to estimate the unknown parameters of the model. The process of developing a linear equation to represent the relationship between two or more variables using the available sample data is known as fitting the linear regression model to observed data. Correlation analysis is used for measuring the strength or degree of the relationship between the predictors or independent variables and the response or dependent variable. The sign of correlation coefficient indicates the direction (positive or negative) of the relationship between the variables, while the absolute value (that is, magnitude) of correlation coefficient is used as a measure of the strength of the relationship. Correlation analysis, however, does not go beyond measuring the direction and strength of the relationship between predictor or independent variables and the response or dependent variable. The linear regression model goes beyond correlation analysis and develops a formula for predicting the value of the response or dependent variable when the values of the predictor or independent variables are known. Correlation analysis is therefore a part of regression analysis and is performed before performing regression analysis. The purpose of correlation analysis is to find whether there is a strong correlation between two variables. Linear regression will be useful for prediction only if there is strong correlation between the two variables.

Types of Linear Regression

The primary objective of a linear regression is to develop a linear equation to express or represent the relationship between two or more variables. Regression equation is the mathematical equation that provides prediction of values of the dependent variable based on the known or given values of the independent variables.

When the linear regression model represents the relationship between the dependent variable (Y) and only one independent variable (X), then the corresponding regression model is called a simple linear regression model.

When the linear regression model represents the relationship between the dependent variable and two or more independent variables, then the corresponding regression model is called a multiple linear regression model.

Following examples illustrate situations for simple linear regression.

A firm may be interested in knowing the relationship between advertising (X) and sales of its product (Y), so that it can predict the amount of sales for the allocated advertising budget.

A botanist wants to find the relationship between the ages (X) and heights (Y) of seedling in his experiment.

A physician wants to find the relationship between the time since a drug is administered (X) and the concentration of the drug in the blood-stream (Y).

Following examples illustrate situations for multiple linear regression.

The amount of sales of a product (dependent variable) is associated with several independent variables such as price of the product, amount of expenditure on its advertisement, quality of the product, and the number of competitors.

Teacher's Note

Simple regression uses one X to predict Y. Multiple regression uses many X values to predict Y, like predicting house price using location, size, and age.

Exam Trick

Simple = one independent variable. Multiple = many independent variables. Look at the question to count how many X's they give you.

Points to Remember

Linear regression predicts one variable from another.
Dependent variable is what we want to predict (Y).
Independent variable is what we use to predict (X).
Simple regression has one X; multiple regression has many X values.
Correlation must be strong before regression gives good predictions.

This is a preview of the first 3 pages. To get the complete book, click below.