The real words
Why words are difficult data
A number needs a scale. A word needs far more than an equality test.
- High dimensionality, 10,000 distinct words and more
- Correlations: Hong Kong, San Francisco, Bay Area are pairs that belong together
- Order: April, February, January, June, March, May is alphabetical and wrong
- Membership: Tennis, Running, Swimming, Hiking, Piano, where one does not belong
- Hierarchy, antonyms and synonyms, entities
Sorting month names alphabetically is the classic text-chart failure. The words carry an order the computer cannot see.