itslabtime.com

Information · Computing · Artificial Intelligence Fundamentals

k-means clustering and sum of squared errors

k-means repeatedly assigns points to the nearest center and moves each center to its assigned points’ mean. Clustering differs from classification trained on known labels.

Related experiment: K-means clustering →
K-means clustering
K-means clustering: simulation view →

Meaning and mechanism

Given k centers, each point is assigned to the nearest cluster. Subsequently, the average of the points within each cluster is used as the new center.

Allocating to nearby centers and renewing the average—these are oriented in a way that do not increase the same squared objective function. It is possible to arrive at different results depending on the initial center.

Relationships and conditions

SSE is the sum of the squares of the distances between each point and its assigned center. A halt in SSE reduction during standard iterations, and the finding of a global optimum among all possible clusters, are distinct concepts.

Euclidean distance and average are well-suited for spherical cluster analysis, and this also affects the results. The choice of k, and the method used to handle outliers and sparse clusters, are important.

What to observe in the simulation

Assignment and mean updates never increase squared error.

Euclidean distances and seeded points are used. Empty clusters keep their centers. A local minimum is not a ground-truth class.

Key relationships and symbols

μⱼ = mean(xᵢ ∈ Cⱼ) · SSE = Σ‖xᵢ − μⱼ‖²

Symbols and units used in the formulas
SymbolMeaningUnit
kSpecified number of clusters items
μⱼMean center of cluster jFeature coordinates
SSESum of squared distances within clustersSquared feature units

Worked example

How is each center updated?

Mean of assigned points · Assignment and mean updates never increase squared error.

Common misconceptions

  • The military code numbers do not inherently guarantee that they possess a pre-existing, correct name or meaning.

Explore this concept through experiments

References

OpenStax · Introduction to Computer Science (English)