Analyzing Search Engines
...searching and browsing easier for advanced users. These features make the search engines more useful and more of a 'one-stop destination' for finding information, including news, sport results, TV guides, weather reports and other information in a 'start page' design to allow the user to easily access the information he needs quickly. These services are particularly relevant to the advanced and regular web users because they use the web frequently, making such a page very convenient. In evaluating this criteria, the three test search engines were tested to see if they offered the following additional services/features: # Online shopping guide # Suggested searches. Searches based on what you have searched for # Other searches - whitepages, yellowpages, maps/directions etc # Hand-picked categories or channels for web browsing # Personalisation of search results, layout, information etc The number of additional services offered by each of the test search engines was calculated and then recorded out of a score of 5. Criteria 5 - Page Design and Layout This criteria was to evaluate the usefulness and usability of the page design and layouts of the search engines. This criteria is important for all users, especially advanced users because they visit search engines regularly and therefore want a page that looks nice, is easy to get around, and loads quickly, even on low-end machines. Because although advanced web users would probably be able to navigate around and use a poorly designed site, a well designed site will mean reduced time using the site and quicker access to the information they're after. Note: Page time is a factor of consideration for this criteria. This is referring specifically to the time the HTML page loads (based on page elements such as tables (and nested tables), images, CSS, JavaScripts etc In evaluating this criteria the three test search engines were tested against the following points: # Easy navigation # Good Appearance/Aesthetics # Quick Page load time Each of these points were graded out of 3 points with 1 for poor, 2 for average, 3 for excellent. Search Engine Selection The three test search engines that the previous criteria were tested against were selected as being the three search engines most likely to be used by an advanced web user. AltaVista was selected because of its powerful search syntax options (eg link:, url: etc) and the fact that it claims to have the largest page index of all search engines. It also provides other features and services useful to a corporate web user and therefore is likely to be one of the first choices for such a user. Google was selected because its 'no-noise' approach. This ideology would be appreciated by many advanced users who don't need fancy graphical interfaces, or comprehensive search help. The first page is simply a search screen, with other features (directory etc) tucked away, where users can access it if they need it; however the main emphasis of Google is on searching. Google's simple search results screen also ensures that the results are returned quickly and its search options (with the possibility of saving preferences to a cookie) means that advanced users can easily have Google configured to their specific needs. Google is very much an advanced user's search engine and it is for this reason it was included for evaluation. DirectHit was selected because of its new approach to ranking search results. Most search engines base the results on keywords, keyword location, keyword density etc, as does DirectHit, but this search engine also takes notice of the links that users click when they perform a search, and how much time they then spend on the sites. These results are then applied to future searches, effectively making real people the judge of the search order (http://www.directhit.com/help/score.html). This different approach is one that is likely to interest advanced web users who are often seeking information quickly and easily. 4. Report on results The following table represents the results of the three test search engines (Part 2) against the evaluation criteria (Part 1 and 3). Accuracy + Relevance Speed Search Options Additional Services Layout + Design AltaVista 60% Excellent 9/10 5/5 6/10 Google 55% Excellent 8/10 2/5 6/10 DirectHit 85% Good 5/10 5/5 7/10 Specific Details of the search engine scores for the Search Options, Additional Services and Layout and Design criteria are available in Appendix 1. 5. Evaluate your criteria and Methodology After evaluating three search engines according to the criteria established in Part 1, the usefulness of some of the criteria was questionable. The Accuracy and Relevance score was useful, and seemed to accurately represent how accurate I felt the search engine was. However, in evaluating the accuracy and relevance of pages, no 'scientific' method could be employed - rather, it was a purely personal decision, of whether I thought the page was relevant to someone using the search term used. For a more in-depth analysis of search engines, I would have to suggest a more tangible method of evaluating this criteria. However, I can't see relevance of search results being measured according to 'rules'; I think personal opinion is an important part of determining relevancy (what may be relevant to one person may be irrelevant to another). Therefore, I don't think a definitive statement of a search engine's accuracy and relevance for all people is possible. Due to the short time-span for this assignment and a lack of inside knowledge of the operation of search engines, the Speed criteria could be measured only by my own guess. This meant that no specific numbers about search speed could be returned and the results I came to are very much debateable. In retrospect, I think the Speed criteria should have been evaluated according to the following points: # CPU time taken to perform search (based on 100 searches). Of course, finding these details would probably require access to the search engine server. # Average ping time (ping search server from about 10 worldwide locations). Again, this point would have been very difficult to do, especially in the limited timeframe. I think a measurement of speed according to the above points would be more accurate and useful, and would provide more specific results. I think the Search Options and Additional Services Criteria were very useful and establishing specific points that the search engines could be easily evaluated against (either true or false) made establishing a score for each search engine easier. The only flaw with the criteria as set out in Part 2 was probably a lack of points, and the scope of these points may have been too narrow. For a more in-depth study, I think more criteria points would have to be established. I think the Layout and Design Criteria were too broad. I think for a more in-depth study, more (and more specific) criteria would have to be established. I would also suggest a true/false style points system (rather than three points ...