Bag of Words (BoW)
Simplest text representation that represents text as an unordered set of words with frequencies.
Bag of Words represents text as word frequency vector without order – simplest baseline for text classification, now replaced by embeddings.
Explanation
BoW ignores grammar and word order: "The dog bites the man" and "The man bites the dog" have the same representation. Despite limitations, useful as a baseline.
Marketing Relevance
BoW is the foundation of many classical ML methods for text classification.
Common Pitfalls
Ignores semantics and word order. Sparse vectors with large vocabulary. Largely replaced by embeddings.
Origin & History
The BoW concept comes from linguistics by Zellig Harris (1954). It became the standard in information retrieval and spam filters. TF-IDF extended BoW with relevance weighting. Word2Vec (2013) and Transformer (2017) made BoW obsolete for many tasks.
Comparisons & Differences
Bag of Words (BoW) vs. Word Embedding
BoW creates sparse frequency vectors; word embeddings create dense meaning vectors that capture semantics.
Bag of Words (BoW) vs. TF-IDF
BoW only counts frequencies; TF-IDF additionally weights with the rarity of a word in the overall corpus.