Search Tool with NLP

Text Search with spaCy and scikit-learn you own this product

This project is part of the liveProject series Build a Search Tool with NLP
prerequisites
intermediate Python • basic scikit-learn • natural language processing tokenization, lemmatization, and cleaning of text data
skills learned
apply Python’s spaCy library to perform essential natural language processing steps • compute TF-IDF tables and apply term frequency search to them • build an inverted index
Olesya Bondarenko
1 week · 9-15 hours per week · INTERMEDIATE
Included with a Manning Online subscription
try now

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases

lite $19.99 per month

  • access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

Look inside
In this liveProject, you’ll explore and assess essential methods for unstructured text search in order to identify which is the best for building a search engine. You’ll preprocess the data for this task using the spaCy library, and then experiment with implementing both a TF-IDF search and an inverted index search to find relevant information.
This project is designed for learning purposes and is not a complete, production-ready application or solution.

book resources

When you start your liveProject, you get full access to the following books for 90 days.

project author

Olesya Bondarenko
Olesya Bondarenko has a multidisciplinary background and experience in natural language processing (NLP), machine learning, deep learning, statistics, time-series analysis, process automation, engineering R&D and new product prototyping. Currently, she is a data scientist at Strong Analytics, a leading provider of customized AI solutions, where she specializes in developing NLP systems. Prior to joining Strong, she worked with several startups leading research and development efforts in the areas of conversational AI, data-leveraged scientific discovery solutions, and a variety of automated analytic and data collection tools. Olesya received her PhD in electrical engineering from the University of California San Diego where she designed and prototyped novel optical devices, as well as custom instrumentation for their analysis.

prerequisites

This liveProject is for intermediate Python programmers familiar with the basics of manipulations with strings, lists and dictionaries. To begin this liveProject, you will need to be familiar with:

TOOLS
  • Intermediate Python
  • Basic understanding of conda environments
  • Basic scikit-learn
  • Basic NumPy
TECHNIQUES
  • Reading data from and writing to JSON files
  • Manipulations with tuples, lists and dictionaries using loops and comprehensions
  • Natural language processing tokenization, lemmatization, and cleaning of text data
  • Basic NumPy array operations

you will learn

In this liveProject you will learn to implement the simple-but-effective term frequency - inverse document frequency (TF-IDF) search method. This method will encompass calculating the frequency of certain words in documents.

  • Use Python’s built in JSON library to store multi-level text data
  • Create, update and transform lists and dictionaries with text data
  • Apply Python’s spaCy library to perform essential natural language processing steps
  • Compute TF-IDF tables and apply term frequency search to them
  • Calculate cosine similarity with scikit-learn
  • Build an inverted index, an essential element of a search engine

features

Self-paced
You choose the schedule and decide how much time to invest as you build your project.
Project roadmap
Each project is divided into several achievable steps.
Get Help
While within the liveProject platform, get help from other participants and our expert mentors.
Compare with others
For each step, compare your deliverable to the solutions by the author and other participants.
book resources
Get full access to select books for 90 days. Permanent access to excerpts from Manning products are also included, as well as references to other resources.

choose your plan

pro

monthly
annual
$24.99
$249.99
only $20.83 per month
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • Text Search with spaCy and scikit-learn project for free

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • Text Search with spaCy and scikit-learn project for free