This MIT technology review explains how the iTunes Genius feature works, parsing millions of iTunes users’ libraries to generate suggestions and recommendations. One of the observations: we’re not unique and special snowflakes. No matter how individual you think you are, you’re part of a large online group.
Discovering the hidden or “latent” factors in your data set is a handy way to reduce the size of the problem that you have to compute, and it works because humans are predictable: people who like Emo music are sad, and sad people also like the soundtracks to movie versions of vampire novels that are about yearning, etc. You might think of it as the mathematical expression of a stereotype–only it works.
Recommendation engines are notoriously difficult to get right, as the Netflix Prize proved.