Linear Regression

Plain-language explanations of the statistical terms used in this tool. Open any term to read more.

Linear Regression: A technique that fits a straight line through your data to describe how one variable tends to change as another changes. The result is an equation you can use to summarise the relationship and predict new values.
Predictor (X) / Independent Variable: The variable you are using as the input — shown on the horizontal axis. It is called "independent" because it is not what you are trying to explain; it is what you are using to do the explaining.
Outcome (Y) / Dependent Variable: The variable you are trying to understand or predict — shown on the vertical axis. Also called the response variable. Its value is assumed to be associated with (or influenced by) X.
Regression Line (Line of Best Fit): A straight line through the data that minimises the total squared vertical distances between the data points and the line itself. It represents the average value of Y for any given value of X.
Slope (b₁): How much Y changes, on average, for every one-unit increase in X. A slope of 3 means Y goes up by 3 when X goes up by 1. A negative slope means Y decreases as X increases. This is usually the most important number for understanding the relationship.
Intercept (b₀): The predicted value of Y when X equals zero — where the regression line crosses the vertical axis. It does not always have a meaningful interpretation. If X is, say, someone's height, then X = 0 is outside the data range and the intercept is just a mathematical anchor for the line.
R² (R-squared / Coefficient of Determination): A number between 0 and 1 measuring how well the model fits the data. An R² of 0.75 means 75% of the variation in Y is explained by X; the remaining 25% is due to other factors. Higher is generally better, but what counts as "good" depends on the subject area.
Adjusted R²: A version of R² that is slightly penalised for the number of predictors in the model. For simple regression (one predictor) the difference is small. It is useful when comparing models with different numbers of variables, because adding more predictors always raises R² even if they add no real information.
Pearson's r (Correlation Coefficient): A number between −1 and +1 measuring the strength and direction of the linear relationship. A value of +1 is a perfect upward line; −1 is a perfect downward line; 0 means no linear relationship. R² is simply Pearson's r squared, so the two always tell the same story about fit.
Null Hypothesis: The starting assumption being tested: that there is no linear relationship between X and Y in the population — in other words, that the true slope is zero. The p-value tells you how consistent your data are with this assumption.
p-value: The probability of observing a slope as large as yours (or larger) purely by chance, if there were actually no relationship in the population. A small p-value (commonly < 0.05) is taken as evidence against the null hypothesis. Crucially, the p-value is not the probability that your result is correct or incorrect.
Statistical Significance: A result is called statistically significant when the p-value falls below a chosen threshold — usually 0.05 (i.e. a 5% chance). This is a convention, not a hard dividing line between "true" and "false." With very large samples, even tiny, practically meaningless effects can be statistically significant. Always consider effect size alongside the p-value.
t-value (t-statistic): The slope estimate divided by its standard error. It measures how many standard errors the estimated slope is away from zero. A t-value with a large absolute value (typically greater than about 2) suggests the slope is unlikely to be zero. The p-value is derived from the t-value.
Degrees of Freedom (df): The number of independent pieces of information available to estimate uncertainty. For simple linear regression, df = n − 2 (total observations minus 2, because we estimated the slope and intercept). The t-distribution used to calculate p-values depends on the degrees of freedom — smaller df leads to wider, flatter distributions and less certainty.
F-statistic: An overall test of whether the model explains significantly more variation than a flat horizontal line would. For simple regression (one predictor), the F-statistic equals the t-statistic squared, so the two tests always give the same p-value. The F-statistic becomes more useful when comparing models with multiple predictors.
Standard Error (SE): A measure of how uncertain an estimate is. The standard error of the slope tells you how much the estimated slope would vary if you repeated your study with different samples from the same population. A smaller SE means a more precise, trustworthy estimate.
Confidence Interval (CI): A range of values constructed so that, if you repeated the study many times, about 95% of those intervals would contain the true population value. A 95% confidence interval for the slope is not the probability that this particular interval contains the truth — it is a statement about the long-run reliability of the method.
Confidence Band: The confidence interval extended across the whole regression line, creating a curved band around it. It is narrowest at the mean of X (where the line is pinned most tightly by the data) and fans outward toward the edges. It reflects uncertainty about where the true average line lies in the population.
Prediction Interval (PI): Wider than the confidence band, the prediction interval shows the range within which a single new, individual observation is likely to fall. It is wider because it accounts for both uncertainty about the true line and the natural variability between individual data points. Confidence bands and prediction intervals both widen away from the centre of the data.
Residuals: The vertical gap between each actual data point and the regression line — in other words, what the model could not explain. A positive residual means the actual value is above the line; negative means it is below. Ideally, residuals should look like random noise with no pattern.
Residuals vs. Fitted Plot: A diagnostic chart showing residuals on the vertical axis against predicted (fitted) values on the horizontal axis. A random cloud of points around the zero line is ideal. Curves or arcs suggest the true relationship is not linear. A funnel shape — where spread increases or decreases — suggests the model's errors are not consistent across all predicted values.
Homoscedasticity: The assumption that residuals have roughly equal spread across all values of X. When this holds, the model's uncertainty is consistent everywhere. The opposite — heteroscedasticity — means errors vary in size depending on where you are in the data, which can distort standard errors and make p-values unreliable even when the slope estimate itself is unbiased.
Normality of Residuals: The assumption that residuals follow an approximately bell-shaped (normal) distribution. This matters mainly for small samples. For larger samples (roughly n ≥ 30), a mathematical theorem called the Central Limit Theorem means slope estimates behave normally even when residuals are not perfectly bell-shaped.
Independence of Observations: The assumption that knowing one observation tells you nothing about another. This is violated by time-series data (where today's value depends on yesterday's), grouped or clustered data (students within schools, patients within hospitals), or repeated measurements from the same individuals. Violated independence typically makes standard errors too small, so p-values appear more significant than they truly are.
Sum of Squares (SS): A way of measuring variation. SSTotal is the total spread in Y around its mean. SSRegression is the portion the model explains. SSResiduals is what is left over. R² is simply SSRegression divided by SSTotal, so the three quantities always sum consistently: SSTotal = SSRegression + SSResiduals.
Mean Squares (MS): A sum of squares divided by its corresponding degrees of freedom. Mean squares make sums of squares comparable regardless of sample size — like an average amount of variation per degree of freedom. The F-statistic is MSRegression divided by MSResiduals.
ANOVA Table: A structured breakdown of the total variation in Y into what the regression model explains and what it does not. For simple regression it provides the same F-test as the t-test for the slope, but the table format becomes essential when comparing models or working with multiple predictors.

Linear Regression

Load Data

Configure Regression

Coefficients

ANOVA Table

Additional Measures