Ashley Paul Mundaden

In this phase I have created a search box that gets your relevant rows from the data-set by calculating the Tf-IDF values of the search input and compare them with the TF-IDF values of each of the rows and return the one with highest cosine similarity.

​

The term frequency of a word in a document. There are several ways of calculating this frequency, with the simplest being a raw count of instances a word appears in a document.

The inverse document frequency of the word across a set of documents. This means, how common or rare a word is in the entire document set. The closer it is to 0, the more common a word is.This metric can be calculated by taking the total number of documents, dividing it by the number of documents that contain a word, and calculating the logarithm.

​

Every time we input a string we treat it as a document and perform TF-IDF calculation for the input doc. We create it in a vector form with the vector size being the size of number of words that were in the input and each value would be the corresponding TD-IDF score of the word. We then compare it with the TF-IDF calculation of the rows in the data-set for those words. When we compare we create a cosine similarity of our input vector with each of the rows and then sort to get the most relevant pages on top. These rows that were fetched will be more useful to the user compared to when doing a plain text search.