cs132.webPage
Class ExtractHRefs

java.lang.Object
  cs132.webPage.ExtractHRefs

public class ExtractHRefs
extends java.lang.Object

Class provides static methods for extracting URLs from a downloaded web page.

Method Summary

static void	extractHRefs(java.net.URL webPage, Callback callback) Connect to a URL, download the contents of the contents and look for embedded HRefs.
static void	main(java.lang.String[] args)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

extractHRefs

public static void extractHRefs(java.net.URL webPage,
                                Callback callback)

Connect to a URL, download the contents of the contents and look for embedded HRefs. This method should always return normally, without throwing an exception, even if an exception or other error occurs in trying to connect to the URL. If the URL doesn't use an http or file protocol, the method shouldn't try to connnect, but just silently return. Similarly, if the ContentType of the contents of the URL doesn't start with text/html, the method should silently return without trying to download the contents of the URLConnection. Otherwise, we scan each line of the contents, looking for a string consisting of

• the string "href" (in any capitalization)
• optional whitespace
• an '=' character
• optional whitespace
• a '"' character
• a string that does not contain any '"' characters (this is the URL found)
• a '"' character

A URL found in an HRef may be an absolute URL (e.g., "http://www.cs.umd.edu") or a relative URL (e.g., "index.html"). Either way, it must be converted into an absolute URL. If relative URLs are found in the web page, they should be resolved in the context of the URL of the webpage, using the URL.URL(java.net.URL, java.lang.String) constructor for URLs. The first parameter should be the result of calling URLConnection.getURL() on the URLConnection being used to download the contents of the web page We only look for (and want to find) an HRef contained on a single line of text, and should ignore HRefs that span more than one line of text.

Parameters:

webPage - - URL of web page to download and extract URLs from

callback - - callback to invoke for each HRef found.

See Also:

URL.URL(java.net.URL, java.lang.String)

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception