M-Step (Means Update) · Rafael Braga-Kribitz

Contents

01Deep Breakdown
02Flowchart

\mu_k^{\text{new}} = \dfrac{\sum_{i=1}^n r_{ik} x_i}{\sum_{i=1}^n r_{ik}}

(1)

We are working on the M-Step means update in the Expectation-Maximization algorithm for Gaussian Mixture Models, with canonical formula $\mu_k^{\text{new}} = \dfrac{\sum_{i=1}^n r_{ik} x_i}{\sum_{i=1}^n r_{ik}}$ , where $r_{ik}$ is the responsibility of component $k$ for observation $x_i$ . The domain is EM for GMMs with soft cluster assignments, finite data, and valid responsibilities satisfying $r_{ik} \ge 0$ and $\sum_{k=1}^K r_{ik} = 1$ for each observation. This update computes the new mean of component $k$ as a responsibility-weighted average of the data.

Deep Breakdown

In a Gaussian Mixture Model (GMM), we assume the data is generated by a mixture of $K$ Gaussian components.

Each component has its own parameters:

mean $\mu_k$ ,
covariance $\Sigma_k$ ,
mixing weight $\pi_k$ .

The EM algorithm estimates these parameters iteratively.

It alternates between:

E-Step: compute soft assignments, called responsibilities,
M-Step: update model parameters using those responsibilities.

The means update belongs to the M-Step. After the E-Step, each observation $x_i$ is no longer assigned to exactly one cluster. Instead, it contributes partially to multiple components through $r_{ik}$ , the posterior probability that observation $i$ belongs to component $k$ .

The new mean for component $k$ is:

\mu_k^{\text{new}} = \frac{\sum_{i=1}^n r_{ik} x_i}{\sum_{i=1}^n r_{ik}}

(2)

This formula is a weighted mean:

the numerator adds the data points, each scaled by how much component $k$ “owns” that point,
the denominator normalizes by the total responsibility mass assigned to component $k$ .

This is the soft-clustering analogue of the ordinary sample mean.

Why this matters in DA/DS:

Soft clustering: unlike K-means, each point can contribute to multiple cluster centers.
Probabilistic segmentation: useful when groups overlap.
Stable parameter updates: outliers or ambiguous points can be down-weighted by low responsibility.
Interpretability: each mean represents the center of a latent Gaussian segment.
Foundation for full EM: the means update works together with covariance and mixing-weight updates.

A useful intuition:

if a point has high responsibility for component $k$ , it pulls $\mu_k$ strongly,
if it has low responsibility, it barely affects $\mu_k$ ,
if responsibilities are exactly 0 or 1, this reduces to the ordinary cluster mean.

So the M-step means update is a “soft centroid” calculation.

Flowchart

#machine-learning #machine-learning-algorithms #expectation-maximisation-em-for-gmms