Recent Changes - Search:

Network Programming

This website demonstrates using wikis as teaching and learning tool.

The course instructor is happy to share the teaching materials here with those who find it readable.

Lecture /

Network Programming in Java - Part Five: URL Framework

A Network Programming Lecture by Steven Choy

Lecture Overview: Introduction to Java URL Framework - Java URL-related classes and interfaces - Interactions in the URL framework - Using the URL class - Using the URLConnection class

Introduction

  • In the previous lectures, we learned how to program TCP/IP clients and servers using sockets.
  • There are other Java built-in APIs that perform networking.
  • In this lecture, we will learn writing Web client programs that interact with Web servers to access various resources on the Web.
  • A resource available on the Web can be located by a Uniform Resource Locator (URL)
  • Examples: HTML documents, images, audio clips, video clips and programs
  • URL format: protocols, ports, hostnames and filename parts.
(source: http://www.cs.uvm.edu/oracle9doc/java.901/a90213/architecture.htm)
  • In Java, the URL framework enables you to manipulate and retrieve access the URL-addressable objects on the Web.

Java URL-related classes and interfaces

  • The extensible URL framework in Java can be divided into two parts: handling protocols and handling contents

Handling protocols

  • It takes care of the communication between a client and a server in downloading the object in interest.
  • It involves the following classes and interfaces:
    • URL class
    • URLStreamHandlerFactory interface
    • URLStreamHandler abstract class
    • URLConnection abstract class.

Handling contents

  • It decodes the received data and then constructs a useful Java representation of the object.
  • It involves the following classes and interfaces:
    • URLConnection abstract class
    • ContentHandlerFactory interface
    • ContentHandler abstract class

Interactions in the URL framework

  • The URL class is meant to represent a URL. You need to create a URL object and specify the URL when you attempt to retrieve a resource on the Web.
  • The URL object will call on a URLStreamHandlerFactory to find an appropriate URLStreamHandler based on the protocol of the given URL, for example, HTTP.
  • That URLStreamHandler will then be asked to create a URLConnection appropriate for the URL.
  • That URLConnection is then used for retrieving the addressed resource.
  • A URLConnection is capable of interacting directly with the server to download the addressed resource.
  • It is able to establish an input stream to read raw data from the resource, with all traces of the protocol stripped.
  • The URLConnection is able to identify the content type of the web resource.
  • If URLConnection is requested to also decode the downloaded resource, it will delegate the task to a ContentHandlerFactory and a ContentHandler — asking a ContentHandlerFactory to return an appropriate ContentHandler for the given content type.
  • That ContentHandler is then responsible for decoding a Java representation of the downloaded URL resource and then return it to its caller.

Using the URL class

  • create a URL from the specified string 
      URL(String string)
  • create a URL from the specified protocol, host name, port number and file 
      URL(String protocol, String host, String file)
      URL(String protocol, String host, int port, String file)
  • create a URL from relativePath relative to the base URL. 
      URL(URL base, String relativePath)
  • open a connection to the URL and returns an input stream of the connection. 
      InputStream openStream()
  • opens a connection to the URL and returns it 
      URLConnection openConnection()
  • returns the protocol, host name, port number, path part, file part or query part of the URL, respectively. 
      String getProtocol()
      String getHost()
      int getPort()
      String getPath()
      String getFile()
      String getQuery()

Example code : uses the URL class to retrieve Web resources and stores them into local files.

  1. import java.io.BufferedInputStream;
  2. import java.io.File;
  3. import java.io.FileOutputStream;
  4. import java.io.IOException;
  5. import java.net.URL;
  6. import java.util.Scanner;
  7.  
  8. public class Downloader {
  9.  
  10.     public static void main(String[] args) {
  11.         String str;
  12.         if (args.length > 0) {
  13.             str = args[0];
  14.         } else {
  15.             Scanner scanner = new Scanner(System.in);
  16.             System.out.print("Enter URL: ");
  17.             str = scanner.nextLine();
  18.         }
  19.  
  20.         try {
  21.             System.out.println("Opening " + str);
  22.             URL url = new URL(str);
  23.             BufferedInputStream bis = new BufferedInputStream(url.openStream());
  24.             String filename = new File(url.getPath()).getName();
  25.             FileOutputStream fos = new FileOutputStream(filename);
  26.             byte[] buffer = new byte[1024];
  27.             int nBytes;
  28.             while ((nBytes = bis.read(buffer)) >= 0) {
  29.                 fos.write(buffer, 0, nBytes);
  30.             }
  31.             fos.close();
  32.             System.out.println("File written to " +
  33.                     new File(filename).getCanonicalPath());
  34.         } catch (IOException ex) {
  35.             ex.printStackTrace();
  36.         }
  37.     }
  38. }
[Get Code]

Using the URLConnection class

  • For the HTTP protocol, the URL class makes a request using the GET method by default.
  • Another common HTTP method is POST, which is often employed to submit form data to Web servers.
  • To use POST method with the URL class, you need the URLConnection class.
  • Among the six classes in the URL framework, URLConnection is the most complex.
  • However, it gives us more control over a connection than just using the URL object, as it contains a lot of methods for you to do things that are not possible with URL objects.
  • The URLConnection class is abstract and has no public constructor. Therefore, you cannot create a URLConnection using the new keyword. Instead, you create a URL object and invoke its openConnection() method to obtain a URLConnection object that is appropriate for that URL.

URLConnection methods

InputStream getInputStream(), OutputStream getOutputStream()
— returns an input stream or output stream, respectively, of the connection.
boolean getDoInput/getDoOutput(), void setDoInput/setDoOutput(boolean b)
— gets or sets the doInput (default true) or doOutput (default false) flags, respectively, of the connection. They specify whether data are read from or written to the connection.
String getContentEncoding(), String getContentType(), int getContentLength(), long getDate(), long getExpiration(), long getLastModified()
— returns the content encoding, content type, content length, date, expiration or last-modified header field, respectively.
void connect()
— establishes the connection to the associated URL.
Object getContent()
— returns the content of the URL.
URL getURL()
— returns the URL of this connection.
String getRequestProperty(String key), void setRequestProperty(String key, String value)
— gets or sets the value of the request property with the specified key.

Example code : make a HTTP request with the POST method

To make a HTTP request with the POST method, you obtain the URLConnection from a URL object, enable its output flag, and write the POST data to its output stream. It is illustrated in the following example.
  1. import java.io.BufferedReader;
  2. import java.io.IOException;
  3. import java.io.InputStreamReader;
  4. import java.io.OutputStream;
  5. import java.net.URL;
  6. import java.net.URLConnection;
  7. import java.util.Scanner;
  8.  
  9. public class PostExample {
  10.  
  11.     public static void main(String[] args) {
  12.         String postUrl = args.length > 0 ? args[0] :
  13.                 "http://www.findxxxx.hk/html/form2email.php";
  14.  
  15.         Scanner scanner = new Scanner(System.in);
  16.         System.out.print("Enter POST data: ");
  17.         String postData = scanner.nextLine();
  18.  
  19.         try {
  20.             URL url = new URL(postUrl);
  21.             URLConnection conn = url.openConnection();
  22.             conn.setDoOutput(true);
  23.             // conn.setRequestProperty("Referer", "http://www.findxxxx.hk/");
  24.  
  25.             // write POST data to server
  26.             OutputStream output = conn.getOutputStream();
  27.             output.write(postData.getBytes());
  28.             output.close();
  29.             // read response from server
  30.             BufferedReader reader = new BufferedReader(
  31.                     new InputStreamReader(conn.getInputStream()));
  32.             String line;
  33.             while ((line = reader.readLine()) != null)
  34.                 System.out.println(line);
  35.         } catch (IOException ex) {
  36.             ex.printStackTrace();
  37.         }
  38.     }
  39. }
[Get Code]

Example code : invoke the methods of the HttpURLConnection class

  1. import java.io.IOException;
  2. import java.net.HttpURLConnection;
  3. import java.net.URL;
  4. import java.net.URLConnection;
  5. import java.util.Date;
  6. import java.util.Scanner;
  7.  
  8. public class URLConnExample {
  9.  
  10.     public static void main(String[] args) {
  11.         Scanner scanner = new Scanner(System.in);
  12.         System.out.print("Enter URL: ");
  13.         String str = scanner.nextLine();
  14.  
  15.         try {
  16.             URL url = new URL(str);
  17.             URLConnection conn = url.openConnection();
  18.             System.out.println("Content type: " + conn.getContentType());
  19.             System.out.println("Content encoding: " +
  20.                     conn.getContentEncoding());
  21.             System.out.println("Content length: " + conn.getContentLength());
  22.             System.out.println("Date: " + new Date(conn.getDate()));
  23.             System.out.println("Expiration: " + conn.getExpiration());
  24.             System.out.println("Last modified: " +
  25.                     new Date(conn.getLastModified()));
  26.             if (conn instanceof HttpURLConnection) {
  27.                 HttpURLConnection http = (HttpURLConnection) conn;
  28.                 System.out.println("Request method: " +
  29.                         http.getRequestMethod());
  30.                 System.out.println("Response code: " + http.getResponseCode());
  31.                 System.out.println("Response message: " +
  32.                         http.getResponseMessage());
  33.             }
  34.         } catch (IOException ex) {
  35.             ex.printStackTrace();
  36.         }
  37.     }
  38. }
[Get Code]
  • In order to invoke the methods of the HttpURLConnection class, you need to cast the URLConnection to an HttpURLConnection as shown in the following code:
  • The program displays information about the URLConnection by using its methods — getContentType(), getContentEncoding(), getContentLength(), getDate(), getExpiration(), getLastModified().
  • To get the HTTP information, we first tests whether conn is an instance of HttpURLConnection; if so, it is cast to a HttpURLConnection and methods of HttpURLConnection — getRequestMethod(), getResponseCode() and getResponseMessage() — are invoked.

Questions

  • A URL consists of four fields. What are they?
  • What type of exception may be thrown by the constructors of URL?
  • What class of object is returned by calling the openConnection() method of URL?
  • Calling the openStream() method of URL is equivalent to calling a method of URLConnection. What is this method?
  • There are two subclasses of URLConnection in the Java API. What are they?
  • To specify the POST method instead of the GET method in an HTTP URL, which method of URLConnection shall be called?
  • Briefly describe how you can get the response code of a HTTP URL.

Reading

Go to the trail Custom Networking of the Java Tutorial, at http://java.sun.com/docs/books/tutorial/networking/index.html. In the lesson Working with URLs, read the four sections What Is a URL?, Creating a URL, Parsing a URL and Reading Directly from a URL.
Go to the trail Custom Networking of the Java Tutorial, at http://java.sun.com/docs/books/tutorial/networking/index.html. In the lesson Working with URLs, read the two sections Connecting to a URL and Reading from and Writing to a URLConnection.

Thanks for Reading

If you would rather like to have this lecture note in printed format, please click the print action link in the top right corner.

If you find any problem in this lecture note, please feel free to tell Steven via steven@findaway.hk.

Edit - History - Print - Recent Changes - Search
Page last modified on November 19, 2009, at 02:02 PM