Trace Definition

Deep Breakdown

What it tells me (Situation Awareness): The trace $\text{tr}(A) = \sum_i a_{ii}$ sums the main diagonal entries of a square matrix. It is a single scalar that summarizes certain global properties of the matrix. The trace is intimately connected to eigenvalues ($\text{tr}(A) = \sum_i \lambda_i$), making it a quick, cheap sanity check for spectral properties. In DA/DS the trace appears in: the total variance explained by a covariance matrix ($\text{tr}(\Sigma)$ = sum of all variances), the mean squared error written as a trace ($\text{MSE} = \frac{1}{n}\text{tr}((Y - \hat{Y})^T(Y - \hat{Y}))$), the nuclear norm regularization objective ($\text{tr}(\sqrt{A^T A})$), and the Frobenius inner product ($\langle A, B \rangle_F = \text{tr}(A^T B)$).

Flowchart

Real-world use

Where it is used

Used to measure total variance in PCA (fraction of variance explained by $k$ components is $\sum_{i=1}^k \lambda_i / \text{tr}(\Sigma)$), to compute matrix norms in regularization, to write the Frobenius inner product between two matrices, and as a fast check on eigenvalue sums without computing the full eigendecomposition.

Problem it solves

Computing all eigenvalues of a large matrix is $O(n^3)$. In many cases you only need their sum (total variance, trace of a product). The trace is $O(n)$ to compute and gives you the eigenvalue sum directly, which is sufficient for variance accounting in PCA or for checking gradient flow in neural networks (trace of the Hessian approximates total curvature).

How it solves the problem

The diagonal entries $a_{11}, a_{22}, \ldots, a_{nn}$ are exactly the projections of the matrix onto the "self" directions. For a covariance matrix, $a_{ii} = \text{Var}(X_i)$ (the variance of feature $i$). Summing these gives the total spread of the dataset. The algebraic fact $\text{tr}(A) = \sum_i \lambda_i$ (sum of eigenvalues) means the trace is a polynomial invariant of $A$ independent of the basis.

Example in a business

A data analyst runs PCA on a marketing dataset with 10 features. The covariance matrix $\Sigma$ has trace $= 24.5$ (total variance). The first 3 principal components have eigenvalues $12.1, 6.4, 3.2$, summing to $21.7$. The explained variance ratio is $21.7 / 24.5 = 88.6\%$, computed entirely from eigenvalue sums without inspecting the eigenvectors. The trace enables this accounting in one line: explained = eigenvalues[:3].sum() / np.trace(cov_matrix).

The formula, piece by piece

• $\text{tr}(\cdot)$: the trace operator; input must be a square matrix; output is a scalar
• $A$: a square $n \times n$ matrix; the trace is only defined for square matrices
• $\sum_i$: sum over the diagonal index $i$, running from 1 to $n$
• $a_{ii}$: the entry at row $i$, column $i$; the "diagonal" entries (where row index equals column index)
• The off-diagonal entries $a_{ij}$ for $i \neq j$ are ignored entirely Full formula: $\text{tr}(A) = \sum_{i=1}^{n} a_{ii} = a_{11} + a_{22} + \cdots + a_{nn}$

The formula in words

"The trace of a matrix is the sum of the numbers on the main diagonal (top-left to bottom-right). Ignore everything else. The result is a single number." Tiny numeric example: $A = \begin{bmatrix}3 & 7 & -1 \\ 2 & 5 & 4 \\ 0 & -3 & 2\end{bmatrix}$. Diagonal entries: $a_{11} = 3$, $a_{22} = 5$, $a_{33} = 2$. $\text{tr}(A) = 3 + 5 + 2 = 10$. Cross-check via eigenvalues: characteristic polynomial gives $\lambda_1 + \lambda_2 + \lambda_3 = 10$.

Critical Concepts

• Only defined for square matrices: you cannot take the trace of a $3 \times 5$ matrix; it has no main diagonal of length $n$.
• Trace = sum of eigenvalues: $\text{tr}(A) = \sum_{i=1}^n \lambda_i$; this holds for any diagonalizable or even just any square matrix over $\mathbb{C}$.
• Cyclic property: $\text{tr}(ABC) = \text{tr}(CAB) = \text{tr}(BCA)$; you can cycle the order under a trace. Critically: $\text{tr}(AB) = \text{tr}(BA)$ even when $AB \neq BA$.
• Linearity: $\text{tr}(A + B) = \text{tr}(A) + \text{tr}(B)$; $\text{tr}(kA) = k \cdot \text{tr}(A)$.
• Frobenius inner product: $\langle A, B \rangle_F = \text{tr}(A^T B) = \sum_{ij} a_{ij} b_{ij}$; the trace of a product is the sum of elementwise products.
• Total variance in PCA: for a covariance matrix $\Sigma$, $\text{tr}(\Sigma) = \sum_i \text{Var}(X_i)$; the fraction of variance explained by the top $k$ components is $\sum_{i=1}^k \lambda_i / \text{tr}(\Sigma)$.
• NumPy: np.trace(A) computes the trace; for the Frobenius inner product use np.sum(A * B) (faster than np.trace(A.T @ B) for large matrices).

Step-by-step calculation (tiny example)

Step 1: Confirm the matrix is square and extract the diagonal $A = \begin{bmatrix}4 & 1 & 0 \\ 2 & -3 & 5 \\ 7 & 0 & 2\end{bmatrix}$. Shape: $3 \times 3$ (square). Diagonal entries: $a_{11} = 4$, $a_{22} = -3$, $a_{33} = 2$.

Step 2: Sum the diagonal entries $\text{tr}(A) = 4 + (-3) + 2 = 3$.

Step 3: Verify the eigenvalue connection numerically Eigenvalues of $A$ (computed numerically): approximately $\{6.02, -3.94, 0.92\}$. Sum: $6.02 + (-3.94) + 0.92 \approx 3.00$ (matches trace).

Code examples

Beginner example (Python)

import numpy as np

# -- Trace: definition, properties, and PCA variance explained ---------
A = np.array([[4.0, 1.0, 0.0], [2.0, -3.0, 5.0], [7.0, 0.0, 2.0]])
trace_a = np.trace(A)
print(f"A:\n{A}")
print(f"\ntr(A) = {trace_a}")

# Verify: tr(A) == sum of eigenvalues
eigenvalues = np.linalg.eigvals(A)
print(f"\nEigenvalues of A: {eigenvalues.real.round(4)}")
print(f"Sum of eigenvalues: {eigenvalues.real.sum():.4f}")
print(f"tr(A) == sum(eigenvalues): {np.isclose(trace_a, eigenvalues.real.sum())}")

# Cyclic property: tr(AB) == tr(BA)
B = np.random.default_rng(42).normal(size=(3, 3))
print(f"\ntr(AB) = {np.trace(A @ B):.6f}")
print(f"tr(BA) = {np.trace(B @ A):.6f}")
print(f"tr(AB) == tr(BA): {np.isclose(np.trace(A @ B), np.trace(B @ A))}")

Beginner example (R)

# -- Trace: definition, properties, and eigenvalue connection ----------
A <- matrix(c(4, 1, 0, 2, -3, 5, 7, 0, 2), nrow = 3, byrow = TRUE)
trace_A <- sum(diag(A))  # R has no built-in trace(); use sum(diag())
cat(sprintf("tr(A) = %.4f\n", trace_A))

# Eigenvalue check
ev <- eigen(A)$values
cat(sprintf("Eigenvalues: %s\n", paste(round(Re(ev), 4), collapse = ", ")))
cat(sprintf("Sum of eigenvalues: %.4f\n", sum(Re(ev))))
cat(sprintf("tr(A) == sum(eigenvalues): %s\n", isTRUE(all.equal(trace_A, sum(Re(ev))))))

# Cyclic property: tr(AB) == tr(BA)
set.seed(42)
B <- matrix(rnorm(9), nrow = 3)
cat(sprintf("\ntr(AB) = %.6f\n", sum(diag(A %*% B))))
cat(sprintf("tr(BA) = %.6f\n", sum(diag(B %*% A))))
cat(sprintf("tr(AB) == tr(BA): %s\n", isTRUE(all.equal(sum(diag(A %*% B)), sum(diag(B %*% A))))))

Professional example (Python)

import numpy as np
import pandas as pd
np.random.seed(42)   # -- Business scenario: PCA variance accounting via trace of covariance matrix
# Marketing analytics dataset: N campaigns, p KPI features.
# Use trace to compute total variance and explained variance ratio per component.
N_CAMPAIGNS = 300
KPI_NAMES = ["ctr", "cvr", "cpc", "roas", "spend", "impressions_k", "clicks_k", "conversions", "revenue_eur", "brand_lift"]
N_KPIS = len(KPI_NAMES)   # Simulate correlated KPI data
L_sim = np.tril(np.random.uniform(0.1, 1.0, size=(N_KPIS, N_KPIS)))
X = np.random.normal(0, 1, size=(N_CAMPAIGNS, N_KPIS)) @ L_sim.T   # Compute covariance matrix and its trace
X_centered = X - X.mean(axis=0)
cov_matrix = (X_centered.T @ X_centered) / (N_CAMPAIGNS - 1)
total_var = float(np.trace(cov_matrix))   # = sum of all feature variances # Eigendecomposition for PCA
eigenvalues, _ = np.linalg.eigh(cov_matrix)
eigenvalues = eigenvalues[::-1]   # descending order # Explained variance ratio (uses trace as denominator)
explained_ratio = eigenvalues / total_var
cumulative_var = np.cumsum(explained_ratio)
pca_df = pd.DataFrame({ "component": [f"PC{i+1}" for i in range(N_KPIS)], "eigenvalue": eigenvalues.round(3), "explained_pct": (explained_ratio * 100).round(2), "cumulative_pct": (cumulative_var * 100).round(2),
})
print(f"Total variance (tr(Sigma)): {total_var:.4f}")
print(f"Sum of all feature variances: {np.diag(cov_matrix).sum():.4f} (same thing)")
print(f"\nPCA variance accounting:")
print(pca_df.to_string(index=False))
print(f"\nComponents needed for 80% variance: {int(np.searchsorted(cumulative_var, 0.80)) + 1}")
print("\nInterpretation: tr(cov_matrix) is the total spread of the data; each eigenvalue's share of the trace is the fraction of variance that principal component captures.")

Source: 1300 p. 121

#linear-algebra#matrices-and-determinants#basic-principles-and-operations