Subject to · Rafael Braga-Kribitz

Contents

01Deep Breakdown
02Hard-margin case
03Soft-margin case
04Flowchart

y_i(w^\top x_i + b) \ge 1 - \xi_i,\; \xi_i \ge 0 \quad \text{or} \quad y_i(w^\top x_i + b) \ge 1

(1)

We are working on the constraint set in Support Vector Machines, which defines the geometric margin conditions for either hard-margin or soft-margin classification. The canonical constraints are

y_i(w^\top x_i + b) \ge 1

(2)

for hard-margin SVM, and

y_i(w^\top x_i + b) \ge 1 - \xi_i,\quad \xi_i \ge 0

(3)

for soft-margin SVM, where $y_i \in \{-1,+1\}$ , $w$ is the separating hyperplane normal vector, $b$ is the intercept, and $\xi_i$ are slack variables that allow margin violations.

Deep Breakdown

In an SVM, the phrase subject to is not a minor detail. It is the geometric heart of the optimization problem. The objective may say “maximize margin” or “minimize complexity plus violations,” but the constraints define what it means for a classifier to be acceptable.

For a linear classifier, the decision rule is:

f(x) = w^\top x + b

(4)

and the predicted class is:

\hat{y} = \operatorname{sign}(w^\top x + b)

(5)

The SVM constraints require that training points lie on the correct side of the separating hyperplane, and not just barely, but with a margin.

Hard-margin case

If the data is perfectly linearly separable, the constraint is:

y_i(w^\top x_i + b) \ge 1

(6)

This means:

if $y_i = +1$ , then we require:

w^\top x_i + b \ge 1

(7)

if $y_i = -1$ , then we require:

w^\top x_i + b \le -1

(8)

So the points are forced outside the two margin boundaries:

w^\top x + b = 1 \quad \text{and} \quad w^\top x + b = -1

(9)

Soft-margin case

If the data is not perfectly separable, we introduce slack variables $\xi_i$ and use:

y_i(w^\top x_i + b) \ge 1 - \xi_i

(10)

with

\xi_i \ge 0

(11)

Now some points can violate the ideal margin. The slack variable tells us how badly. This constraint set matters in DA/DS because it explains:

which points become support vectors,
which points lie safely outside the margin,
which points lie inside the margin,
which points are misclassified,
why the hinge loss has the form it does.

Inputs:

labeled data $(x_i, y_i)$ ,
hyperplane parameters $w$ and $b$ ,
optionally slack variables $\xi_i$ .

Outputs:

a feasible set of classifiers that respect the margin logic.

Why it matters in practice:

it defines the geometry of classification,
it controls robustness to noise and overlap,
it determines which points actively constrain the model,
it provides the bridge between geometric margin and optimization.

This object links directly to:

Soft Margin Optimization (Primal), because these are its constraints,
Hard Margin SVM, because the hard-margin case is the zero-slack version,
Hinge Loss, because the minimum feasible slack is the hinge-loss value,
Primal and dual forms, because these constraints generate the KKT structure and support-vector behavior.

Flowchart

#machine-learning #machine-learning-algorithms #support-vector-machines-svm-and-classification