Apache HTTP Conditional Policy Requests for checking whether resource is modified or not and reduce unnecessary processing

Hi,

Before writing this blog, I would first like to make you familiar with one quote “Be liberal in what you accept, and conservative in what you send”. Yes that’s what HTTP robustness protocol states.

Scenario :
We have a use case where our server need to read XML files that are large containing 400 outer element with extremely nested elements from provider server.

Tasks Involved :

1. Our server will run good amount quartz scheduling job for fetching XML file of different scenario’s from a server and process them differently.
2. Each job will run per minute to monitor the content, as real time content may change or not, which has to be displayed in real time to the end users.

Problem :
Reading, validating and processing large XML every minute to make them available to end user“even if file content is unmodified”. It will make unnecessary overhead for our server and consumption of IO operation even when data is not modified.

It’s like our server not able to find relevant data and ending up reading same data every time polluting the server environment with processing of large responses.

How to clean our server environment, from large unnecessary processing? Solution : HTTP conditional policy request using mod_policy

Yes you can make conditional request “If-Modified-Since” to provider to determine whether the file is being modified or not.

Request Flow :

1. Our server will make request via adding conditional header, “If-Modified-Since”, Using ApacheHTTPClient 4.3

         apacheHTTPClientAdapter.requestObject.setHeader("If-Modified-Since","Fri, 17 Jan 2014 19:10:11 GMT");
         //Note : I have used ApacheHttpClient 4.3 to make request
   

    
    

2. Or Using HTTP URL Connection :

        URL url = new URL({file});
        HttpURLConnection http = (HttpURLConnection)url.openConnection();
        Http. addRequestProperty ("If-Modified-Since", LastUpdate);
        http.connect();
   

3. Now, provider server mod_policy filter check for the GMT lastUpdated date you have specified in “If-Modified-Since” header value of request, so :
   • If the file last-modified is greater than lastUpdated than only provider server will pass original request via responding with 200 OK with the file content to our server.
   • else if the file is not modified, provider server will respond with response code HTTP 1.1/304 Not Modified without any file content to our server.
4. Isn’t it beautiful, We have saved our server for such large overhead of validating and processing unnecessary file and from polluting our server environment with unnecessary processing.

 
Hope this help and make you familiar with conditional request policies if you don’t know about it yet. There are many other policies like “If-Match”, “If-None-Match”, “If-Unmodified-Since”, “If-Range”. For more details refer : http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.24