====== AJAX - Make an AJAX request using XMLHttpRequest ======

To fetch the content of a URI using an **XMLHttpRequest** object in JavaScript.


===== Background =====

**XMLHttpRequest** is the JavaScript class which underpins the concept of AJAX (Asynchronous JavaScript and XML).  It allows a web page to load new data without reloading the page as a whole.  This in turn allows many important types of web application to be coded entirely in JavaScript when it would previously have been necessary to rely on technologies such as Java applets or ActiveX.

Despite its name XMLHttpRequest is not limited to XML: it can be used to fetch any type of data, including other text-based formats such as JSON, and binary formats such as PNG or JPEG.

It is good practice to transfer data in structured form where possible, in order to cleanly separate the presentation layer (on the browser) from the business logic (on the server).  Alternatively you can take the more direct approach of sending pre-formatted HTML or XHTML, which need only be parsed by the browser before being inserted into the document tree.

===== Scenario =====

Suppose you are developing a webmail application, and want the client to list the content of the user’s inbox.  It can obtain the necessary raw data from the server by making a GET request to the URL /folders/inbox, accompanied by a pre-existing cookie which is used to identify and authenticate the user.  The response to the GET request is an array of objects encoded as JSON.

===== Method =====

==== Overview ====

The method described here has six steps:

  - Create a new XMLHttpRequest object.
  - Specify what is to be fetched and how by calling open.
  - Specify the expected responseType.
  - Register an onreadystatechange event handler.
  - Initiate the request by calling send.
  - Wait for the event handler to be called.
  - Create a new XMLHttpRequest object

The constructor should normally be called without any arguments:

<code javascript>
var xhr = new XMLHttpRequest();
if (!xhr) {
  alert('failed to create XMLHttpRequest object');
}
</code>

For subsequent requests you can choose between creating a new XMLHttpRequest object or reusing an old one (see below).

==== Specify what is to be fetched and how by calling open ====

The XMLHttpRequest constructor returns a request object which is in the UNSENT state.  In order to progress to the OPENED state you must specify the URI to be fetched and HTTP method to be used.  This is done by calling the open method:

<code javascript>
xhr.open("GET", "/folders/inbox", true);
</code>

There are two required parameters and three optional ones:

  - the HTTP request method (as a string),
  - the URI to be fetched,
  - the async flag (optional, normally true),
  - the username (optional), and
  - the password (optional).
  - The HTTP request method tells the server what action to perform when it locates the resource identified by the URI.  You will probably be familiar with the GET and POST methods that can be used within an HTML <form> element, however XMLHttpRequest is not limited to these. See below for a description of the more commonly-used request methods.

The URI can be given in absolute or relative form, but if the request is made in the context of a web page then it must either conform to the same-origin policy [http://tools.ietf.org/html/rfc6454] or be allowed on some other basis such as CORS [http://www.w3.org/TR/cors/] (see below).  The W3C standard requires support for the http and https URI schemes only. Where other schemes are supported by the browser there are likely to be differences in behaviour (for example, due to not using HTTP headers and status codes).

Setting the third argument to true indicates that the transaction should be completed asynchronously, meaning that send should return immediately when called without waiting for a response from the HTTP server.  Synchronous requests are strongly discouraged when running in the foreground context of a web page, because the user interface is single-threaded and will become unresponsive if you cause the foreground thread to block.

You may encounter code where the third argument has been omitted.  This is fully acceptable, since it is an optional argument with a default value of true, however it is common practice to specify the required value explicitly.

The username and password arguments are relevant when using HTTP authentication methods which require them, such as Basic Authentication or Digest Authentication.  They can be omitted or set to null to indicate that no credentials are available.  If authentication is necessary to gain access to the requested URI then omission can result in the user being prompted by the web browser.  If you do not want that to happen then prompting can be suppressed by supplying an invalid username and password.  These will either be ignored, or result in an 401 (Unauthorized) response.

==== Specify the expected responseType ====

The default behaviour of XMLHttpRequest is to expect a text response from the remote server and present it to the caller in the form of a DOMString.  Since JSON is a text-based data format, one option is to receive it as text then parse it afterwards, however modern web browsers provide a shortcut with the potential for greater efficiency.  By setting its responseType property, you can inform the XMLHttpRequest object that a JSON-encoded response is expected:

<code javascript>
xhr.responseType = 'json';
</code>

This feature is not supported by all web browsers, but you can allow for that by providing a fallback mechanism which decodes by another method if necessary.  To prepare for this it is helpful to:

  * Avoid setting a responseType unless the XMLHttpRequest instance already has a property of that name.  This is because implementations with full or partial support for the responseType property will prevent you from setting an unsupported value, but implementations with no support will allow you to set anything.

  * Catch and ignore any exceptions. This should not be necessary, because the WebIDL bindings for JavaScript specify that assigning an invalid string to an enumeration type has no effect, however there are some browsers which throw an exception instead.

You can test whether the request object has a responseType property using the in operator:

<code javascript>
try {
  if ('responseType' in xhr) {
    xhr.responseType = 'json';
  }
} catch (e) {}
</code>

Response types allowed by the XMLHttpRequest specification are:

^responseType^decoded^parsed^response^
|arraybuffer|no|no|ArrayBuffer|
|blob|no|no|Blob|
|document|yes|as HTML or XML|Document|
|json|yes|as JSON|(varies)|
|text|yes|no|DOMString|

The default value (which is the empty string) has similar behaviour to text, but with some additional rules for determining the character set of XML documents.


==== Register an onreadystatechange event handler ====

Because the HTTP request occurs asynchronously, you will normally want to be notified when it has finished.  The most portable way to do this is by means of the onreadystatechange event handler.  This refers to the readyState attribute of the XMLHttpRequest object, which can take the following values:

|0|UNSENT|The object has been constructed.|
|1|OPENED|The open method has been successfully called.|
|2|HEADERS_RECEIVED|Reception of response headers is complete.|
|3|LOADING|Reception of response body is in progress.|
|4|DONE|Reception of response has either completed or failed.|

Of these, the state most likely to be of interest is DONE. Since it does not by itself distinguish between success and failure, it is then necessary to check the HTTP status code.  This can be found in the status attribute.  Values in the range 200 to 299 indicate that the request was successful:

<code javascript>
xhr.onreadystatechange = function() {
  if (xhr.readyState == 4) {
    if (xhr.status >= 200 && xhr.status < 300) {
      var folderIndex = (xhr.responseType == 'json') ? xhr.response : JSON.parse(xhr.responseText);
      processFolderIndex(folderIndex);
    } else {
      alert('failed to download folder index');
    }
    xhr.onreadystatechange = null;
  }
}
</code>

You may encounter scripts which treat a status code of 200 as the only successful outcome.  This will work under most circumstances, since the other codes in that range are rarely used, but there is some risk of unnecessarily rejecting a usable response.

The reason for resetting the event handler is to avoid a race condition which might otherwise cause the data to be processed more than once.  It is necessary because the value of readyState seen by the handler could be different from the value it had when the event was triggered.  The reset also avoids a memory leak in some browsers.

Fallback to JSON.parse is implemented here by testing whether the responseType was successfully set to JSON.  As previously noted, if this feature is supported at all then it should only allow supported response types to be set.  You may wish to provide a further fallback to eval in case JSON.parse is not available either.

==== Initiate the request by calling send ====

Prior to this point, nothing has been sent to the remote server.  To make that happen you should call the send function:

<code javascript>
xhr.send();
</code>

This has one optional parameter, which is a message body to be sent as part of the request:

<code javascript>
xhr.send(body);
</code>

Whether it is appropriate to send a message body, and what it should contain, will depend on the HTTP request method and the URI.  See below for further details.

There is a bug in FireFox 3.0 and below which causes send to throw an exception if you do not supply any arguments.  You can work around this by explicitly setting the message body to null:

<code javascript>
xhr.send(null);
</code>

==== Wait for the event handler to be called ====

Provided you did not ask for a synchronous connection, the send function should return almost immediately.  Your script can then go on to do other work if required, but it should ultimately return control to whatever called it within a reasonably short period of time.  The event handler will be called as and when appropriate without any further action on the part of your script.

==== Sample code ====

<code javascript>
var xhr = new XMLHttpRequest();
if (!xhr) {
  alert('failed to create XMLHttpRequest object');
}
xhr.open("GET", "/folders/inbox", true);
try {
  if ('responseType' in xhr) {
    xhr.responseType = 'json';
  }
} catch (e) {}
xhr.onreadystatechange = function() {
  if (xhr.readyState == 4) {
    if (xhr.status >= 200 && xhr.status < 300) {
      var folderIndex = (xhr.responseType == 'json') ? xhr.response : JSON.parse(xhr.responseText);
      processFolderIndex(folderIndex);
    } else {
      alert('failed to download folder index');
    }
    xhr.onreadystatechange = null;
  }
}
xhr.send(null);
</code>

===== Notes =====

==== HTTP request methods ====

The four key methods for implementing a RESTful web service are:

  * GET, for retrieving a data item (when no side-effects are required or expected);
  * PUT, for updating a data item, or sometimes for creating one (idempotent);
  * POST, for creating a new data item (not idempotent); and
  * DELETE, for deleting a data item (idempotent).

In the absence of a more suitable method, normal practice is to use POST for operations with side effects and GET for operations with none. Other useful methods include:

  * HEAD, for retrieving metadata about a data item without the data itself (usually to determine whether it has changed since it was last retrieved);
  * PATCH, for atomically updating part of a data item; and
  * OPTIONS, for determining which operations are allowed for the given URI without requesting any particular action to be performed.

Typically only a subset of these methods will be available at any given URI.  You can use any method that the HTTP server supports, with the exception of CONNECT, TRACE and TRACK (which are forbidden for security reasons).  Beware that even if the web service disregards the HTTP method (and therefore accepts any method), the choice that you make can still influence how requests and responses are cached.

==== Message bodies ====

Whether an HTTP request can or should have a message body, and what the message body should contain, is determined in part by the request method:

  * GET and HEAD should not have a message body, and if you try to supply one then XMLHttpRequest will ignore it.
  * DELETE and OPTIONS can have a message body, but that would be unusual.
  * PUT should have a message body which is the content that you want written to the URI.
  * POST should have a request body which is either the content you want added, or which otherwise describes the operation to be performed. For the latter it is common practice to use the same format as an HTML form (content type application/x-www-form-urlencoded or multipart/form-data).
  * PATCH should have a request body specifying the changes to be made to the given URI. You can use any format that the server is willing to accept. An example of a type designed for this purpose is application/json-patch+json (JSON Patch format).

Beyond this the HTTP specification places few restrictions on how the message body is used. For this reason, web services are usually accompanied by some form of API specification detailing what should be sent and how.

Where a message body of type application/x-www-form-urlencoded or multipart/form-data is needed, options include:

  * Using the FormData interface, in browsers which support it. This can be passed directly to XMLHttpRequest.send and is serialised as multipart/form-data.
  * Construction and submission of an HTMLFormElement using a hidden iframe. This method has good browser support, but is inflexible and potentially inefficient.
  * Writing code to explicitly construct the response. Though somewhat tedious, this is a reasonably straightforward process for either form type.

Note that provision of a message body does not preclude you from also encoding information into the path and/or query components of the URI if it is semantically appropriate to do so.  (This would not be possible with an HTML form, but there is no objection within the HTTP specification.)

==== The same-origin policy ====

When XMLHttpRequest is used from within a web page, it cannot be used to access arbitrary URIs.  The reason for this is to limit the potential for abuse. Without any restrictions an attacker would be able to:

  * Connect to addresses which are routable from the client, but not from the public Internet.  Examples include the loopback address, and RFC 1918 private address blocks (10/8, 172.16/12 and 192.168/16).  The same is true for URL schemes such as file: which implicitly refer to the local machine.
  * Bypass access controls based on the IP address of the client. The address at greatest risk here is the loopback address, as this is often treated as a trusted source requiring no further authentication.
  * Use cached credentials (such as cookies, or usernames and passwords for HTTP authentication) for sites that have been accessed using the same web browser.  In this case it is immaterial whether the targetted URL is local or remote with respect to the client.

To prevent this, browsers check whether the request is directed back to the site from which the web page originated. If so then the request is allowed, on the basis that the website owner has responsibility for ensuring that pages within the website do not invoke scripts which put the website at risk.  The details of this policy are documented in RFC 6454 [https://www.ietf.org/rfc/rfc6454.txt].  For URIs which contain a hostname there are three components which must match:

  * the URI scheme,
  * the hostname and
  * the port number.

Both the URI scheme and hostname are normalised to lower case before comparison.  The appropriate default port is used if none has been specified.  For URIs without a hostname there is significant variation between browsers, and you should not assume that any access will be available.

Points to note:

  * It is the origin of the web page which counts, not the origin of the JavaScript (which might be different).
  * Requests which cross subdomain boundaries do not satisfy the same-origin policy, even if the subdomains in question are located within the same administrative zone.

Consequences of the same-origin policy are that web applications cannot easily combine data from multiple servers, or be served separately from any web services that they make use of.  This is problematic because there is often a need to separate these types of subsystem for security or performance reasons. An exception to the same-origin policy has therefore been defined, known as Cross-Origin Resource Sharing (CORS).

CORS allows safe requests (such as GET and HEAD) to be made to any destination, but does not allow the script to see the result unless the server responds with a suitable Access-Control-Allow-Origin header.  The same applies to some less-than-safe POST requests, which are allowed due to pre-existing loopholes.  Other types of request are permitted, but are validated with the server before they are sent (a process known as preflight).

If CORS is unavailable then there are some alternative methods which can be used to circumvent the same-origin policy and achieve a similar outcome:

  * Install a reverse proxy between the browser and the relevant servers (or proxy the traffic through one of the servers), thereby making all necessary content accessible through a single domain name. Because this is a server-side solution it should work with any web browser.  The main drawback is the additional traffic and latency, particularly if the servers are not physically co-located.
  * Use a <script> element to fetch data formatted as JSONP [https://en.wikipedia.org/wiki/JSONP] (which is similar to JSON, but has padding to invoke a callback function once the fetch is complete).  This supports only GET requests, and compared to XMLHttpRequest is less satisfactory with regard to error handling and security, but provides much better compatibility with older browsers.
  * Programmatically populate and submit a <form> element.  The response can be directed to an <iframe> in order to avoid reloading the page, but gaining access to the content may be problematic as this is subject to the same-origin policy.

==== Circumventing inappropriate response caching ====

If a web server outside your control fails to set appropriate cache control headers then it is usually possible to prevent caching by adding a dummy parameter with a random value to the query component of the URI.  For example, the URI /folders/inbox might become /folders/inbox?dummy=a9e40717291db2f1.

Avoid doing this unless necessary, as it can adversely affect performance due to cache pollution.  It is better to provide correct cache control headers if you are able.

===== Variations =====

==== Reuse of XMLHttpRequest objects ====

There is no objection to reusing an XMLHttpRequest object provided that there is no temporal overlap between requests, however you should not do this with the expectation of any significant performance gain: the effort required to create a new object is negligible in comparison to sending the HTTP request and processing the response.  Furthermore you should not do this for requests which might otherwise be capable of executing in parallel, because having them share a single XMLHttpRequest instance would be likely to reduce performance.

An example where reuse makes good sense is in a polling loop used to implement HTTP server push.  The requests naturally occur in series, so there should be no risk of contention.  However, if an XMLHttpRequest object were needed for any other purpose in the same program then it would probably be best to use a separate one.

==== Legacy support for Internet Explorer ====

Internet Explorer lacks native JavaScript support for XMLHttpRequest prior to IE7.  It instead provides an ActiveX control named MSXML2.XMLHTTP with a similar API. Microsoft recommend using version 6.0 of this control if it is available, or failing that version 3.0, but not version 4.0 or 5.0.  If none of these are available then Microsoft.XMLHTTP can be tried as a last resort. The native JavaScript class should always be the first preference:

<code javascript>
function createXMLHttpRequest() {
  if (window.XMLHttpRequest) {
    return new XMLHttpRequest();
  }
  try {
    return new ActiveXObject('MSXML2.XMLHTTP.6.0');
  } catch (e) {}
  try {
    return new ActiveXObject('MSXML2.XMLHTTP');
  } catch (e) {}
  try {
    return new ActiveXObject('Microsoft.XMLHTTP');
  } catch (e) {}
  return null;
}
</code>

Further points to be aware of:

  * You may need to call open before registering the onreadystatechange handler. Normally this would make no difference, however open was originally conceived as a method for resetting the XMLHttpRequest object, and in the course of this resets the event handler.
  * In testing it was found that Microsoft.XMLHTTP does not support the in operator (it causes an error which cannot be caught with an exception handler), and does not allow onreadystatechange to be set to null (but you can set it to a function that does nothing instead).
  * Internet Explorer caches XMLHttpRequest responses, whereas most other browsers do not. This is not a bug, and it need not be a problem if you configure the web server to send appropriate cache control headers. You should do that regardless, because even if the browser does not cache the content then something else might. However, it has historically tended to be Internet Explorer usage which has caused this issue to manifest.
  * IE8 and IE9 do not support the JSON object in quirks mode.


===== Alternatives =====

==== Using jQuery ====

jQuery provides several methods for performing HTTP requests, the most generic being jQuery.ajax which has broadly similar functionality to a raw XMLHttpRequest object.  There are also a number of shorthand methods providing more specialised operations:

|jQuery.get|method = 'GET'|
|jQuery.post|method = 'POST'|
|jQuery.getJSON|method = 'GET'; dataType = 'json'|
|jQuery.getScript|method = 'GET'; dataType = 'script'; execute script|
|.load|method = 'GET'; dataType = 'html'; load HTML into element|

For example, given an empty <div> element with an id of inbox:

<code javascript>
<div id="inbox"/>
</code>

you could populate that element with pre-formatted HTML obtained from the URI /folders/inbox.html using .load:

<code javascript>
$("#inbox").load("/folders/inbox.html");
</code>

One of the main benefits of using jQuery is to abstract away differences between browsers.  The 1.x branch provides support back to IE6.

==== Using AngularJS ====

AngularJS provides a service named $http for making HTTP requests.  JSON-encoded responses are automatically detected and decoded.  For example, the following code fragment would fetch and decode the webmail inbox content as per the scenario above:

<code javascript>
angular.module("webmail", []).controller("messages", function ($scope, $http) {
 $http({method: "GET", url: "/folders/inbox"}).then(function (response) {
  $scope.messages = response.data;
 });
});
</code>

The main benefit here is that the fetched data can be declaratively bound to page elements without the need to perform any explicit manipulation of the DOM.  For example, to iterate over the inbox content and render it as an HTML table:

<code javascript>
<div ng-controller="messages">
 <table>
  <tr ng-repeat="message in messages">
  <td>{{message.From}}</td>
   <td>{{message.Subject}}</td>
   <td>{{message.Date}}</td>
  </tr>
 </table>
</div>
</code>


===== References =====

  * XMLHttpRequest Level 1, W3C Working Draft [http://www.w3.org/TR/XMLHttpRequest/] 
  * XMLHttpRequest, WHATWG Specification [https://xhr.spec.whatwg.org/]
  * XMLHttpRequest, Web API Interfaces, Mozilla Developer Network [https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest]
  * Using XMLHttpRequest, Web API Interfaces, Mozilla Developer Network [https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/Using_XMLHttpRequest]
  * A Barth, The Web Origin Concept, RFC 6454, December 2011 [https://www.ietf.org/rfc/rfc6454.txt]