II. Directory Search:

Directories are composed of links to web sites arranged by topic (subject matter). They are man-made and have fewer links available than search engines. The hyperlink: Internet By Subject on the COS Library home page is a directory for the Web. Notice that directories have very broad subject headings and that by making more and more choices, the topics are eventually narrowed down to particular sites. You should also do a directory search when you want to see a list of sites on your topic with annotations. The Librarian's Index to the Internet is a very good annotated web directory at the URL: http://lii.org .Using a directory may help you to avoid sites where the content is very brief or superficial that are more likely to be found when using search engines.

The University of California at Berkeley Library sponsors an excellent tutorial and table of features for recommended subject directories.

WHEN YOU ARE LOOKING FOR NON-SPECIFIC INFORMATION, DO A DIRECTORY SEARCH.

III. Search Engine Search:

Search engines are software programs that search the web and log words into a database. When a word is typed into a search box the program scans the search engine's database for the word and returns sites that are mathematically determined to be relevant. Most search engines rank the sites by relevancy. Remember this relevancy is determined by a software program not by humans who have actually looked at the particular site. Search engines also compile databases based on popularity.

 Because the databases compiled by search engines can have words from millions of web sites, it is important to plan a search strategy when using search engines.  For help in preparing a search strategy, read: Recommended Search Strategy: Analyze your topic & Search with peripheral vision prepared by the UC Berkeley Library. A worksheet to help in the analysis is also provided: Analyze Your Topic at this site.
Taking a few extra minutes to think about a search strategy, will actually save time and provide better results.

Although many of the search engines have indexed millions of web sites, there is no one search engine or directory which has indexed everything on the web or the Internet. Also there is always a lapse in time from what is published on the Web to what is indexed in a search engine; there is no instant indexing. Because search engine software differs, each search engine will give a different view of the Web. It is usually a good idea to try the search strategy with more than one engine.

Because each search engine is unique, it is a good idea to always check the "Hints", "Help", "Tips" etc. links before performing a search. Also like everything else about the Web, search engines change.   

The search engine is by key word rather than by subject. You should use a search engine when you are looking for an obscure topic, a specific site, or need to retrieve many, many web sites, or want to search by words in a domain or URL, or to retrieve by popularity ranking.

The University of California at Berkeley Library sponsors an excellent tutorial and table of features for recommended search engines: http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/SearchEngines.html

If the search retrieves too few results, here are some troubleshooting ideas:

• Double check your search strategy to see if it is correct for the particular search engine used.

• Check your spelling.

• Try more synonyms or variants or truncation.

• Try the search with a different search engine.

If the search retrieves too many results, here are some troubleshooting ideas:

•  Be more specific in choice of words 

•  Add more concepts to the search.

•  If the search engine allows, limit the search by language or top-level domain.

•  Try reorganizing your words to do a phrase search.

WHEN YOU HAVE SOMETHING VERY SPECIFIC TO FIND, USE A SEARCH ENGINE AND PLAN A VERY PRECISE SEARCH STRATEGY.
 

____________________________________________________________________________________

¹ Robert Harris, A Guidebook to the Web. (Guilford, Connecticut: Dushkin/McGraw-Hill, 2000)51.

Glossary *

ARPANET   (Advanced Research Projects Administration Network) It was the precursor to the Internet. It was developed in the late 60's by the U. S. Department of Defense as an experiment in wide-area networking that could survive nuclear war.

Berners-Lee, Tim    inventor in 1989 of the combination of software programs and networking protocols known as the World Wide Web.

Blog    A web site that contains an online personal journal with reflections, comments and often hyperlinks.

Boolean    From the name of an English mathematician, George Boole (1815-1864),refers to a search strategy which relies on the concepts of AND, OR, and NOT. For a more detailed explanation of the use of Boolean logic in Internet searching, click on this link from the University at Albany Libraries.

Browser    A software program used for displaying the HTML language of the Web. The first browser was introduced in 1993. Internet Explorer is the leading browser currently.

Cache    is the memory storage of the computer. The computer uses data in the cache as a short-cut for retrieving information which speeds up the performance of the browser. The cache should be cleared on a regular basis or it will slow down the browser instead of speeding it up.

CERN    European Laboratory for particle Physics where the World Wide Web project originated in 1989.

Cyberspace    Term originated by the author, William Gibson, in his novel NEUOMANCER. The word is used to describe the whole range of information resources available through computer networks.

Domain Name    The unique name that identifies an Internet site. Domain names always have two or more parts separated by dots. The part on the left is the most specific and the part on the right is the most general.

E-mail    Electronic mail. Messages are sent from one person to another person electronically via computer. It is the most widely used service on the Internet. The Internet mail protocol is called SMTP (simple mail transport protocol).

Encryption    Any procedure used to convert text into cipher text to insure network security.

FAQ    Acronym for Frequently Asked Questions. FAQ's are documents that list and answer the most commonly asked questions on a particular subject.

FTP    Acronym for File Transfer Protocol allowing users to transfer text files, programs, software etc. from one Internet site to another. This was one of the original Internet services. Files open to the public are called "anonymous" FTP sites.

Gutenberg    Project Gutenberg's mission is to create a free library on the Internetof the world's greatest books. The Project is named for the 15th century printer, Johannes Gutenberg, who invented the printing press and made books available to the masses for the first time in history.

Gopher    An Internet protocol which is text and menu-based and organized by subject. Gopher gets its name from the University of Minnesota where the protocol was developed. Gopher is also a pun on "gofer"- one who fetches things.

Hit    A result of a search query.

Home Page    A document created to serve as the starting point for World Wide Web users; it is also the first page on a Web site.

Host Computer    A computer connected to a network that a user can access for information or for running a computer software application.

HTML    Acronym for hypertext markup language which is a coding language used to create documents on the World Wide Web.

Http    Acronym for hypertext transport protocol, the protocol of the World Wide Web.

Hyperlink    An embedded HTML code that appears on the computer screen as a highlighted word or image or icon.. It is the basis for the World Wide Web. It allows users to go with the click of a mouse to other documents.

Internet    An international, decentralized network of computer networks running on the TCP/IP protocol.

Meta Search Engines   Allow searches to be sent to several individual search engines all at one time. The results are shown all at once. Metacrawler at www.metacrawler.com  was one of the first. Dogpile at www.dogpile.com/ is a popular example.

Natural Language Search    Some search tools allow a question to be typed in the search box. Ask.Com  is a good example of this type of tool.

Netscape    A graphical browser for the Web; a software program for viewing hypertext documents and all other protocols of the Internet. It was released in October 1994.

Packet Switching    A method used to move data on the Internet where all the data coming out of the computer is broken into chunks, sorted and directed to different routes and reassembled at the destination.

Password    A code used to gain access to a locked system. Good passwords contain letters and non-letters.

Phishing    An Email is sent out and directs  the user who takes the bait to a web site, that although it may look authentic, is actually bogus. The user is then asked for personal information such as a social security number or bank account numbers or other information that is used for identity theft.

Protocol    A set of rules or standards defining how computers communicate and exchange information with each other.

Relevancy Ranking    A method search engines use to sort the results list . The relevancy is determined by using algorithms, but actually it is the searcher who ultimately determines which sites are relevant.

Results List    The web pages that are retrieved by the search engine that match what was typed in the search box. The most helpful results include some text from the page, the date modified, and a choice of "similar pages".

Search Directories    Are subject (topic) indexes of Web sites organized by categories. They are usually man-made according to selection policies and have fewer sites included than search engine databases. .An example is the Librarian's Index to the Internet at www.lii.org.

Search Engines    Are searchable databases compiled by automated programs, which are called "crawlers," "robots," or "spiders."  The databases are based on  key word rather than on topic. They usually have coverage of millions of sites and the relevancy is mathematically determined. Some examples are as follows: Google at www.google.com and  Hotbot at http://hotbot.com.

Search Strategy    Components are as follows: 1)State what you want to find 2) Identify key words 3)Select synonyms and variant word forms 4) Combine synonyms, keywords, and variant word forms 5)check your spelling.

Status Bar    At the bottom of the screen, the status bar displays the Web address of the document being transferred, the progress of the downloading, and the completed percentage of the document layout, as the page is loaded.

Stemming    Some search engines allow a word to be reduced or truncated to its root form and will retrieve all forms of the same word in the pages. For example, in many search engines, using teach* will retrieve teaches, teacher, teaching.

TCP/IP    Acronym for Transmission Control Protocol/Internet Protocol which is the suite of protocols that define the Internet; to be on the Internet the computer must run on TCP/IP.

Telnet    An Internet protocol for remote terminal connection. One of the most common uses of Telnet is to access online library catalogs at colleges and universities world-wide. Login, passwords and terminal emulation for each site will be different.

URL    Acronym for Uniform Resource Locator which is the standard way to give the address of any resource on the World Wide Web. Every page on the Web has a unique URL. It consists of the protocol  used, the domain, sub-domain and host and usually a directory and filename.
            Example:    http://www.cos.edu/library/     

Web 2.0    A phrase coined by O'Reilly Media in 2004 referring to a second-generation of web-based services such as social networking, wikis, blogging, podcasting, and folksonomies. The emphasis is on online sharing, networking, and collaboration.

Web Page    A document on the World Wide Web. Each page is an individual HTML file with its own URL.

Web Site    A group of related web pages collected around a main page.

Veronica    Software which searches keywords in gopher menus.

Wiki    A web site that lets visitors add, remove and edit content. The most famous example of a wiki is Wikipedia.

WWW    Acronym for the World Wide Web.

World Wide Web    An Internet service which contains text, graphics, video and sound. It is currently the most powerful, comprehensive and popular Internet protocol using hyperlinks to link one piece of information to another in a web pattern.

*This glossary was compiled from the following sources:

Courtney, Nancy, ed.. Library 2.0 and Beyond: Innovative Technologies and Tomorrow's User Westport, Connecticut: Libraries
        Unlimited, 2007.

Flanagan, Debbie. "Preparing Your Search". Web Search Strategies.1999,2000 
         (http://home.sprintmail.com/~debflanagan/prepare.html). 13 June 2000.          

Harris, Robert. A Guidebook to the Web. Guilford, Connecticut: Dushkin/
          McGraw-Hill, 2000.

Henderson, John. "ICYouSee Glossary". ICYouSee A Guide to the World Wide Web. June 1999.
           (http://www.ithaca.edu/library/Training/glossary/cache.html)    15 Oct. 2001.

Hillstom, Kevin. Defining Moments: The Internet Revolution. Detroit: Omnigraphics, 2005.

Morley, Deborah. Getting Started: Web Page Design with Microsoft 
           Frontpage 98. Fort Worth: Dryden Press,1999.

Rappoport, Avi. "Search Terms Glossary". Search Terms for Web Sites and Intranets. 2004.
            (http://www.searchtools.com/info/glossary.html ) 26 Aug. 2004.  

"What is phishing?"  Webopedia. 2004 (www.weobopedia.com/TERM/p/phishing.html) 27 Jan. 2005.

"Whatis?com". What Is Com Home Page. 1996-2000. 9 June 2000 
            (http://www.whatis.com/nfindex.htm).   
 

Proprietary Databases

Although they may appear to be normal web sites, Proprietary  Databases are not. Proprietary means exclusively owned, private. If you have access to searching one of these databases, it is because someone has paid for that privilege. These databases are not searched by web search engines and, therefore, their information does not show up in normal web search results. These databases make up part of what is referred to as The Invisible or Deep Web. The proprietary databases that are available to COS students are linked to the COS Library Web Page under the category: Academic Electronic Resources. The link is:  Academic Journals/ Magazines | Newspapers.

The Invisible or Deep Web

Proprietary Databases are one part of the invisible web. Usually these databases require a paid subscription. Sometimes the site may be password-protected for members only of an organization; sometimes the site may offer free searching but  registration is required; often newspaper sites require registration.
General search engines don't necessarily search file formats such as .PDF, audio, video, or images, so information in these formats also become part of the invisible web.
Database-driven  web sites are also part of the invisible web, where the information only becomes available after a search is done at that particular site. Thomas: Legislative Information on the Internet, a database accessing major legislation, the Congressional Record, committee information and historical documents,  and  The Internet Movie Database are examples of this type of database.
 

The above information on the Invisible or Deep Web was taken from the following:

Clyde, Anne. "The Invisible Web" Teacher Librarian. April 2002.
          http://www.teacherlibrarian.com/pages/infotech29_4.html (25 Sept. 2003).

For more information on the Invisible Web along with some specialized directories, search engines, and databases, link to Robert J. Lackie's web site: Those Dark Hiding Places: The Invisible Web Revealed .

The Library site at the University of California at Berkeley offers a more in depth discussion of the Invisible Web: http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/InvisibleWeb.html .

| First Class Session | Second Class Session | Third Class Session | Fourth Class Session |
 | Library 102 Syllabus |           

 Since September 25,  2003, your are visitor:
Date Last Updated:04/10/08
For questions and comments please E-mail ginah@cos.edu