Regression Analysis

3.1 Regression Foundations: From Problem to Model

Regression answers who affects the outcome, by how much, and whether the relationship can be used for prediction. This section applies the four steps to a toy retailer.

What Linear Regression Is

• Linear regression fits a linear relationship between a variable of interest and a set of potentially related variables.
• The variable to predict is the dependent variable, denoted Y; the variable used to explain it is the independent variable, denoted X.
• The fitting objective is to make the gap between predicted and actual values as small as possible.
• A coefficient should be read as “all else equal, the marginal effect of all else equal a one-unit change in one variable”—this phrase cannot be omitted.

Case: A Toy Retailer’s Forecasting Problem

A sales manager at a toy retailer must forecast monthly sales for a product over the next six months. She previously relied on experience and intuition; now she wants a more disciplined process. Monthly sales may depend on unit price,advertising spending and promotion spending. The question is: given values of these three variables, what will sales be?

Data file:Toy-Sales.xlsx, containing 24 months of sales and three independent variables.

The Four Steps of Regression

1. Model formulation write the relationship between the dependent and independent variables, and decide which variables belong in the model.
2. Estimation use software, such as Excel’s Regression tool, to estimate the coefficients.
3. Inference assess whether the estimates are credible: are the coefficients significant, and are their signs sensible?
4. Prediction produce forecasts for the scenarios of interest and state the applicable range.

Steps One and Two: Specify the Model, Then Estimate It

In Excel, use “Data ▸ Data Analysis ▸ Regression,” selecting Unit Sales as the dependent variable and all three independent variables. The software finds the β values that minimise the sum of squared residuals and reports coefficients, standard errors, t values, p-values and R².This step produces only numbers; whether they are credible and usable is addressed in the next two steps.

Estimation: Reading Excel Regression Output

The model explains R² = 0.8588 of the variation; adjusted R² = 0.8377, with 24 monthly observations.The p-values of all three independent variables are below 0.05, so all are significant; the intercept is not significant, but this usually does not prevent using the three slopes.

Prediction: Sales under Three Scenarios

The main difference among the three scenarios is promotion spending: Scenario 3 is about 1,955 units above Scenario 1. Note that all scenario inputs lie within the range of the historical data—extrapolating beyond that range is at your own risk.

3.2 Error and Significance: Is the Model Credible?

Error in Regression

• Regression always contains error; a forecast will not exactly equal the actual value:residual = actual value − predicted value.
• The vertical distance between a data point and the regression line is the residual; large residuals indicate a poor fit and small residuals a good fit.
• The key question is not whether error exists, but how large it is and whether it is random.

R-Squared: A Measure of Goodness of Fit

R-squared is the proportion of variation in the dependent variable explained by the model. It ranges from 0 to 1, with values closer to 1 indicating a better fit. It answers “how much does the model explain,” not “how accurate is the forecast,” and there is no universal threshold for “good enough.”

The table above is Price only the price-only model: it explains 61.9% of the variation in sales. Adding advertising and promotion increases explanatory power to 85.9%. A high R-squared does not by itself make a model useful; coefficients must also be significant and directionally sensible.

Where Error Comes From and the Four Assumptions for the Error Term

These four assumptions determine whether t tests and confidence intervals are reliable. Plotting residuals before interpreting a regression is the cheapest diagnostic step.

Coefficients Also Have Error: t Values and p-Values

Coefficients are estimated from a sample and have their own standard errors. A smaller standard error indicates a more precise estimate; degrees of freedom = observations − number of independent variables − 1. Testing whether a coefficient is zero uses the following t statistic:

Reading the p-value: If the true coefficient were zero, the p-value is the probability of observing a t value this large or larger. A small p-value makes the “no effect” hypothesis difficult to reconcile with the observed data, so retain the variable; a large p-value provides no evidence of an effect in this sample and should not be treated as a conclusion.

The null states that the variable’s effect is zero; reject it when p < 0.05.Not significant ≠ no effect; it only means this sample does not reveal an effect.

3.3 Categorical Variables and Collinearity

Question 1: Only Regions A and B

A parcel-delivery company operates in two regions, A and B. Trucks depart from the same central depot, and the data record parcels per trip, truck age and total delivery time. The question is:how much longer does delivery to Region A take than to Region B?

Region is a categorical variable whose values are labels and cannot simply be coded 1 and 2—“A is one more than B” has no meaning. Introduce a dummy variable REGA: code 1 for trips to Region A and 0 for Region B, so both regions can enter one equation.

Data file:deliveries1.xlsx, based on 30 two-region deliveries.

Question 2: A Third Region C Is Added

New data now include Region C. Three categories cannot be represented by one dummy variable; you need two dummy variables: REGA and REGB. A delivery to Region C is coded (0, 0) and serves as the reference category—every regional coefficient is read relative to it.

Data file:deliveries2.xlsx, with 47 deliveries. First estimate the model yourself using the same method, then compare with the results below.

Results: Estimates from 47 Deliveries

Sample: 47 deliveries; R² = 0.9513; reference category: Region C.

Application: Drivers of Refrigerator Price (37 Models)

R² = 0.8379. Note:a different model specification can produce a different conclusion:

Total capacity is not significant in Model 2 because its coefficient compares equal increases in freezer capacity with equal decreases in refrigeration capacity. That is a property of the specification, not evidence that capacity is unimportant.

Car Fuel-Economy Case: Neither Variable Is Significant

Displacement and cylinders are each highly significant on their own, yet neither is significant when both enter the model—their correlation is 0.9403. This is multicollinearity.

Collinearity: Identification and Treatment

• Occurs when two or more independent variables are highly correlated, so the variation in Y cannot be assigned reliably among them.
• The model’s fit and forecasts can remain acceptable, but interpretations of “which variable has a larger effect” are no longer reliable.
• Detection: inspect the correlation matrix among independent variables and flag high correlations; some software also reports VIF.
• Treatment: usually unnecessary for prediction; when coefficient interpretation matters, a common approach is to remove one of the highly correlated variables.

3.4 Variable Transformations and Elasticity

Why Transform Variables?

• The regression model is linear in its parameters, but the relationship between Y and X need not be linear.
• Transformations can turn a curved relationship into a linear one and improve the fit.
• Transformations can also give coefficients the interpretation of elasticity or growth rate, making them easier to explain to management.
• Common transformations: add a squared term(X², allowing the marginal effect to vary with X), add an interaction term(X₁X₂, representing a joint effect), use reciprocals, powers or roots, take natural logarithms, or transform Y and X separately.

Interpreting Coefficients in Log Models

Case: Elasticity and Trend in Cocoa Demand

The data contain annual global cocoa demand, cocoa price and income per capita over several years. Demand is proportional to price and income, and it also changes gradually over time, so the model logs price and income and represents trend with year:

Run this regression in Excel, record the three coefficients and their interpretations, then compare with the results below.

Sample: 49 years (1960–2008); R² = 0.9573.price elasticity −0.22 Interpretation: a 1% price increase reduces demand by about 0.22% on average; because its absolute value is below 1, cocoa demand is inelastic. The income elasticity is −0.572 with p = 0.1127,not significant and not suitable as a conclusion.

3.5 AI-Assisted Regression Interpretation

AI is good at translating regression output into managerial language and equally good at attaching a coefficient to the wrong variable. The point is not “using AI to run regression,” but “drafting with AI and verifying as the human.”

What AI Can and Cannot Do in Regression

3.6 Summary and Assignment

• Regression answers who affects the outcome and by how much; the four steps are model formulation, estimation, inference and prediction.
• R-squared assesses fit; p-values assess significance; neither replaces the other.
• Dummy variables bring categories into a model,the reference category is chosen by the analyst, and coefficients are interpreted relative to it.
• Collinearity can undermine coefficient interpretation but usually does not prevent prediction.
• Log transforms provide elasticity and growth-rate interpretations; coefficient interpretation must follow the model form.

The next class begins Unit 4: Time Series and Text Data.Unit 4 treats this unit’s regression as a causal forecasting tool, so review how regression coefficients are interpreted.

Appendix: Interactive Demonstrations and Classroom Exercises

Data sources: Toy-Sales.xlsx (24 months), Home_Prices.xlsx (90 cities), Cars.xlsx (38 cars), Cocoa.xlsx (49 years), Refrigerators.xlsx (37 models), and deliveries2.xlsx (47 deliveries). Every coefficient, standard error, t value, p-value and R-squared on the page is calculated from these original data and can be checked against Excel.

Unit Glossary (English–Chinese)