You have fifty thousand support tickets, or chat logs, or survey responses. Nobody has read them. You need to know what is in there before you can decide what to do about any of it. Clustering is the standard answer, and the pipeline is more interesting than “run k-means on it” suggests — most of the difficulty is in the steps either side of the clustering itself.
Step one: embeddings
An embedding model maps each document to a vector, positioned so that texts with similar meaning land near each other. This is what makes the whole approach work on language: “my card was declined” and “payment failed at checkout” share almost no vocabulary, but a decent embedding model puts them close together. Keyword methods never could.
Two practical points. First, chunk length matters — embedding a whole document averages away the distinctions you care about, so split long texts into coherent pieces first. Second, embeddings inherit the biases of their training data. A model trained on general web text will cluster your specialised domain vocabulary less sharply than a domain-adapted one, and the failure is quiet rather than obvious.
Step two: the curse of dimensionality
Embedding vectors typically have several hundred to a couple of thousand dimensions, and distance metrics behave badly at that size. As dimensions grow, the distance between the nearest and furthest points in your dataset converges, so “nearest neighbour” progressively stops meaning anything and clustering algorithms that rely on distance degrade with it.
The fix is dimensionality reduction before clustering — typically down to something in the tens. UMAP is the common choice because it preserves local neighbourhood structure better than PCA while remaining tractable at scale. Be aware that the reduced space is for the algorithm, not for you: the two-dimensional version you plot is a further projection and the distances in that picture are not the distances the clusterer saw.
Step three: clustering, and why not k-means
K-means is the obvious tool and usually the wrong one here. It requires you to declare the number of clusters up front — which is the thing you are trying to discover — it assumes roughly spherical clusters of similar size, and it forces every point into a cluster whether or not it belongs in one.
Real text data does not behave that way. Topics come in wildly different sizes, they have irregular shapes, and a meaningful fraction of your documents are one-off noise that belongs nowhere. Density-based methods such as HDBSCAN handle all three: they infer the cluster count, tolerate arbitrary shapes, and explicitly label outliers as noise instead of forcing them into the nearest blob.
Expect a substantial noise bucket on the first run. That is usually correct behaviour rather than a bug, though a very large one suggests your embeddings or your reduction parameters need attention.
Step four: labelling, where it becomes useful
A cluster identified as “cluster 7” is worthless to a stakeholder. The output only becomes actionable once each group has a name a human recognises.
The old approach extracted distinctive terms statistically — the words that are common inside the cluster and rare outside it. It is fast and it gives you a bag of keywords, which is better than nothing and worse than a sentence.
The better approach now is to sample representative documents from each cluster — those nearest the centroid, plus a few from the edges — and ask an LLM to name and describe the group. The result reads like something a person wrote, because it is the kind of summarisation LLMs are genuinely good at. Include the edge cases in the sample: they are what tells you whether the cluster is one coherent topic or two that should be split.
Hierarchy
Flat clusters lose information. “Billing” is a useful label, but underneath it sit failed payments, refund requests, and confusion about invoices, and those need different responses. Hierarchical clustering gives you the whole tree, so you can cut it at whatever granularity the question demands — a handful of themes for an executive summary, forty for the team actually fixing things.
What to do with the result
The obvious use is exploratory — finding out what is in a corpus nobody has read. The less obvious uses are often worth more. Cluster sizes over time make a change-detection signal: a topic that triples in a week is an incident. Cluster assignments make a sampling frame, so you can build an evaluation set that covers your actual distribution rather than over-representing whatever is most common. And clusters that are large but absent from your documentation are a content gap, stated in the users’ own words.
