Webcrawlers April 3 Lecture
tags:: #memex3 #idea projects::
Today in class, Graham talked about web crawlers and PageRank, something I have experience in making from one of my classes. There are many ways to cheat your way into a better ranking, such as filling your website with hidden text, something Graham went over with in class. It is actually a lot more impactful to the early internet than it may seem, you could have thousands and thousands of hidden words not even related to your website just to make your website recommended more often. Another big way to make recommendations more accurate was the use of using vector space projection and cosine similarity, which combines both term frequency and page rank. Vector space projection is doing calculations on your search and creating a vector with it, then creating a vector with the possible pages. Cosine similarity takes those vectors then whichever vector has an angle closest to the search vector is the best match.