2 Generating synonyms

Synonyms are a practical way to boost recall in search by allowing different phrasings of the same intent to match the same documents. The chapter shows how synonym expansion can be applied at indexing or query time, and explains the trade-offs: expanding at index time increases index size but speeds up queries, while expanding at query time keeps the index lean but adds runtime cost. It first walks through a vocabulary-based approach using Lucene analyzers and token filters, then improves maintainability by loading synonyms from files and large resources like WordNet. However, static dictionaries are limited: they may not fit domain data, lag behind slang, acronyms, or evolving usage, and ignore context.

To overcome these limits, the chapter introduces learning synonyms from data via word2vec, grounded in the distributional hypothesis that words in similar contexts tend to share meaning. After a primer on feed-forward neural networks and backpropagation, it details the two word2vec architectures—CBOW and Skip-gram—and how they produce word embeddings whose proximity (e.g., cosine similarity) reveals semantic relatedness. The chapter demonstrates that higher-dimensional embeddings trained on larger, domain-relevant corpora yield much better neighbors, and shows a practical setup with Deeplearning4j to train vectors from song lyrics and query nearest words to act as synonyms.

Finally, it integrates learned synonyms into a Lucene pipeline by building a token filter that augments tokens with top, high-confidence neighbors from word2vec, while controlling index growth and noise via thresholds, part-of-speech constraints, document/term informativeness, and limits on the number of expansions. It underscores evaluation through precision, recall, zero-result rates, and model selection via cross-validation and parameter tuning (dimensions, window size, model choice). For production, it recommends retraining embeddings directly from the indexed corpus—using a custom SentenceIterator over stored field values—so synonym generation stays current as data evolves, and revisits the operational choice between index-time and query-time expansion based on performance needs.

Figure 2.1. Synonym expansion at search time, with a neural network

Figure 2.2. Synonym expansion graph

Figure 2.3. Split portions of the text depending on the type of data

Figure 2.4. Predicting price with a feed forward neural network with 3 inputs, 5 hidden units and 1 output unit

Figure 2.5. Propagating signal through the network

Figure 2.6. Backpropagating signal from output to hidden layer

Figure 2.7. Geometric interpretation of backpropagation with gradient descent

Figure 2.8. Plotted word vectors for Aeoroplane

Figure 2.9. Feeding word2vec (skip gram model) with text fragments

Figure 2.10. Continuous Bag of Words model

Figure 2.11. Continuous Skip-gram Model

Figure 2.12. Highlights of Word2vec vectors over Hot 100 Billboard dataset

Figure 2.13. Token stream after word2vec synonym expansion

• Synonym expansion can be an handy technique to improve recall and make the users of our search engine happier
  
• Common synonym expansion techniques are based on static dictionaries / vocabularies which might require manual maintenance or, at least, are often very far from the data they are used for
  
• Word2vec is a neural network based algorithm for learning vector representations for words which can be used to find words with similar meanings—or at least that appear in similar contexts so that it sounds reasonable to exploit it for synonym expansion too
  
• Word2vec can provide good results, but we need to manage word senses or part of speech when using it for synonyms
  

FAQ