Aller au contenu

From basic polling to server push

Introduction

What is the need ?

The Web was born around the client/server model, where the Web client requests a resource and the Web server serves this resource. In this model, the Web client is always the initiator of a transaction and there is no mechanism for the server to independently push data to the client without the client first making a request.

Modern web applications, including Web of Things applications, require the use of more advanced server push mechanisms. This is required for minimizing traffic and improving the user experience.

What you’ll build

We will demonstrate various possibilities for implementing such a push mechanism. The codelab starts with a simple application performing basic polling. In the basic polling mechanism, the client requests a resource at regular time intervals and the server immediately serves the resource to the client upon each request. This approach has important limitations and we will show different mechanisms that overcome these deficiencies: from long polling to web sockets.

What you’ll learn

  • How to implement various server push scenarios, both from a Web client and from a Web server perspective.
  • The advantages and limitations of each specific technology allowing server push.

You will demonstrate the technologies in consecutive steps. Some code blocks are provided for you to simply copy and paste.

What you’ll need

  • WebStorm.
  • Node.js framework.
  • Prior knowledge of JavaScript and Node.js programming, based on lectures, lab and exercise sessions.

Basic polling

The introduction of the XMLHttpRequest Standard in the late 90’s was one of the first steps for extending the simple request-response HTTP protocol for distributing structured documents to readers using a browser. This standard defines an API for Web clients using scripting for transferring data between a client and a server. Much later, the Fetch API - Web APIs was introduced as a more powerful and flexible tool for fetching resources over the network.

The most basic use of both standards is through the Web client repeatedly polling the server for an updated version of the resource, very often at regular time intervals. This mechanism is often called basic or plain polling and it is the mechanism that was implemented until now in your lab exercises.

In this part of codelab, we will implement a very simple Web client-server application for polling a game score and displaying it in the Web client. In this simple application, the Web server is responsible for updating the score and delivering it to the client - in the traditional simple request-response way.

The Web client is a simple HTML page with a button for launching score updates and displaying the score on screen. The page also displays how many successful requests have been made to the server.

With the WebStorm IDE, you may create the code below in files located in the same project. This will ease the development and testing of the different steps of the codelab.

The code for the Web client implementing basic polling is given below. This example uses the Fetch API - Web APIs for implementing the request to the Web server. The successive calls for updating the score are implemented in the showScore() function, with the call to the setTimeout() function. This means that each time a score is received from the server, a call for getting another update is scheduled after a given interval specified in milliseconds.

index_basic_polling.html
index_basic_polling.html
<!doctype html>
<html lang="en">
<head>
    <title> Getting the score </title>
    <!-- jQuery javascript integration -->
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
</head>

<body>

<button onclick=getScore()>Launch score update</button>
<p id="successfulCalls"> Number of successful calls is 0</p>
<p id="timeoutCalls"> Number of calls with timeout is 0</p>
<p id="score"> No score </p>

<script type="text/javascript">

  const interval = 10000;
  let baseUrl = "http://localhost:8080/basicpolling";
  let nbrOfSuccessfulCalls = 0;

  function status(response) {
    if (response.status >= 200 && response.status < 300) {
      return Promise.resolve(response.text());
    } else {
      return Promise.reject(new Error(response.statusText));
    }
  }

  function showScore(text) {
    nbrOfSuccessfulCalls++;

    // update the score
    $("#score").html(text);

    // update the number of calls value
    $("#successfulCalls").html("Number of successful calls is " + nbrOfSuccessfulCalls);

    // launch next polling
    setTimeout(() => getScore(), interval);
  }

  function showError(error) {
    alert(`Sorry, there was a problem: ${error}` );
  }

  // Method for getting the score
  function getScore() {
    fetch(baseUrl, {
      method: 'GET', // *GET, POST, PUT, DELETE, etc.
      mode: 'cors', // no-cors, *cors, same-origin
      headers: {
        'Accept': 'text/plain'
      }
    })
      .then(response => status(response))
      .then(response => showScore(response))
      .catch(error => showError(error));
  }
</script>
</body>
</html>

The Web server is implemented using a very simple Node.js Express application. The generic code for the web server listening to port 8080 is

update_server.js
update_server.js
// http server to use: needs to be modified for using different servers
const httpServer = require('./servers/long_polling');

// HTTP Server
const port = 8080;
httpServer.server = httpServer.listen(port, function () {
  console.log('HTTP server started...');
  console.info(`Your service is up and running on port ${port}`);
});

This generic code allows us to easily switch the implementation of the HTTP server in each step of this codelab.

For the basic polling case, the Express application code is

servers/basic_polling.js
servers/basic_polling.js
const express = require('express');
const cors = require('cors');

let score = "0:0";

process.stdin.on('readable', () => {
  let chunk;
  // Use a loop to make sure we read all available data
  while ((chunk = process.stdin.read()) !== null) {
    // simply update the score
    score = chunk;
  }
});
const app = express();

// allow all origins
app.use(cors());

// serve basic polling route
app.get('/basicpolling', function (req, res, next) {
  res.writeHead(200, { "Content-Type": "text/plain"});
  res.write(score);
  res.end();
});

module.exports = app;

This application serves the /basicpolling route and delivers the score as text/plain content. For the sake of simplicity and readability, this application does not check for content-type and does not handle other methods and routes for returning the correct return code.

For updating the score on the server side, we use the process console, so that you can modify the score by typing a new score value in the console of the Node.js application.

You may launch/debug both the Web client and the Web server applications. On the client side, the following HTML page should display in a tab of your browser:

Basic polling client

Basic polling client

If you hit the “Launch score update” button, the score should update to “0:0”. On the server side, you may open the application console and type a new score as shown below. By doing so, the new score should update on the Web client after a given time (at most the interval specified in the setTimeout() call).

Basic polling server

Basic polling server

From this simple example, the limitations of the basic polling mechanism should become very clear:

  • First the client should make requests at small time intervals if it wants to be updated soon after a change of a resource on the server.
  • Second the server has no means of updating the client upon a resource change.

This behavior is sketched in the diagram below:

HTTP polling

HTTP polling

If you observe the network traffic in the browser, you should see requests made at regular intervals as shown below. Requests last a very short time because they get an immediate response from the server.

HTTP polling traffic

HTTP polling traffic

Long polling

To overcome the deficiency of the basic polling mechanism as described above, Web application developers can implement a technique usually called HTTP long polling.

With this technique, the Web client still polls the server for getting an updated version of the resources, but the Web server does not reply immediately to the request. Instead, the Web server holds the request until new data is available on the server. Once new data is available, the server responds to the request. The Web client receives the new information and immediately - without timeout - issues another request. This sequence of operations is repeated again and again and this effectively emulates a server push - as soon as the server has new data available, it can notify the Web client. This behavior is shown in the diagram below:

HTTP long polling

HTTP long polling

For implementing this behavior in our simple “Get the score” application, there are only little changes required. The Web client must modify the url for the request and issue a new request immediately after getting the response from the previous request. For preventing timeouts, the server also sends empty messages before a timeout may occur. The client filters those responses for displaying timeouts.

index_long_polling.html
index_long_polling.html
<!doctype html>
<html lang="en">
<head>
    <title> Getting the score </title>
    <!-- jQuery javascript integration -->
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
</head>

<body>

<button onclick=getScore()>Launch score update</button>
<p id="successfulCalls"> Number of successful calls is 0</p>
<p id="timeoutCalls"> Number of calls with timeout is 0</p>
<p id="score"> No score </p>

<script type="text/javascript">

  const interval = 10000;
  let baseUrl = "http://localhost:8080/longpolling";
  let nbrOfSuccessfulCalls = 0;
  let nbrOfTimeOuts = 0;

  function status(response) {
    if (response.status >= 200 && response.status < 300) {
      return Promise.resolve(response.text());
    } else {
      return Promise.reject(new Error(response.statusText));
    }
  }

  function showScore(text) {
    // an empty text means that a timeout occurs on the server side
    // in this case, the server sends an empty payload
    if (text === "") {
      // update the number of timeouts value
      nbrOfTimeOuts++;
      $("#timeoutCalls").html("Number of calls with timeout is " + nbrOfTimeOuts);
    }
    else {
      // update the number of successful calls value
      nbrOfSuccessfulCalls++;
      $("#successfulCalls").html("Number of successful calls is " + nbrOfSuccessfulCalls);

      // update the score
      $("#score").html(text);
    }

    // launch next request immediately
    getScore();
  }

  function showError(error) {
    if (typeof error.text === 'function') {
      error.text().then(errorMessage => {
        //this.props.dispatch(displayTheError(errorMessage))
        alert(`Sorry, there was a problem: ${errorMessage}` );
      });
    }
    else {
      alert(`Sorry, there was a problem: ${error}` );
    }
  }

  // Method for getting the score
  function getScore() {
    fetch(baseUrl, {
      method: 'GET', // *GET, POST, PUT, DELETE, etc.
      mode: 'cors', // no-cors, *cors, same-origin
      headers: {
        'Accept': 'text/plain'
      }
    })
      .then(response => status(response))
      .then(response => showScore(response))
      .catch(error => showError(error));
  }
</script>
</body>
</html>

The Web server Express application serving the long polling request is

servers/long_polling.js
servers/long_polling.js
const express = require('express');
const cors = require('cors');

let score = "0:0";
let pending_response;
let pending_response_date;

process.stdin.on('readable', () => {
  let chunk;
  // Use a loop to make sure we read all available data
  while ((chunk = process.stdin.read()) !== null) {
    // simply update the score
    score = chunk;

    // if we have a pending request, send the data
    if (pending_response != null) {
      pending_response.writeHead(200, { "Content-Type": "text/plain" });
      pending_response.end(score);
      pending_response = null;
      pending_response_date = null;
      score = "";
    }
  }
});

const app = express();

// allow all origins
app.use(cors());

// serve long polling route
app.get('/longpolling', function (req, res, next) {
  // check if there is a new score available (data from stdin)
  // if yes, deliver it right away
  // otherwise store the response without answering
  if (score.length > 0) {
    res.writeHead(200, { "Content-Type": "text/plain"});
    res.end(score);
    score = "";
  }
  else {
    // store the response so that we can respond later (when stdin gets data)
    pending_response = res;
    pending_response_date = new Date().getTime();
  }
});

// for handling timeouts and send an empty response before it happens
// we run a function at regular interval that check for timeout and
// does send an empty response
const timeOutValue = 10000;
setInterval(() => {
  // close out requests that are withing 10 seconds to timeout
  if (pending_response && pending_response_date) {
    let elapsed_time = new Date().getTime() - pending_response_date;
    const allowed_margin = 10000;
    // we expect that app.server.timeout is set
    if (elapsed_time >= app.server.timeout - allowed_margin) {
      pending_response.writeHead(200, {"Content-Type": "text/plain"});
      pending_response.end("");
      pending_response = null;
      pending_response_date = null;
    }
  }
}, timeOutValue);

module.exports = app;

There are two important changes as compared to the application serving the basic polling:

  • A GET request to the ‘/longpolling’ route is getting an immediate response only if there is a new score. The score is initialized to “0.0” and it is reset to “” each time a response is sent to the client.
  • If a response is pending at the time a new score is entered by typing it on the console, it is then sent immediately to the client.

This behavior can be observed from the network traffic, as shown below (in this example, the score was changed twice). In this figure, you can observe that requests last much longer than for basic polling and that there is alway a pending request available for the server.

HTTP long polling traffic

HTTP long polling traffic

Note that a robust implementation should account for timeout in the fetch request on the client side. If no response is sent by the server after a given timeout time, then the fetch request will end with net::ERR_EMPTY_RESPONSE. In this case, a new request should be issued. This is implemented in our example with a call to setInterval that runs a check at regular intervals.

The long polling is thus a solution for immediate server push using a request-response mechanism. There are however various concerns for applying such a mechanism at a system level:

  • Scalability on the server side: maintaining pending responses will consume a lot of resources (memory, thread, etc.)
  • If the thread associated with the connection is blocked with the pending request, then thread exhaustion can occur very quickly.
  • How is a connection re-established if it is lost by the client (loose connections on the client side).

For these reasons and others that are not developed here, other solutions are probably required for a large-scale deployment. Among those solutions, we will present the SSE (Server-Sent Event) API and the WebSocket API.

Server-Sent Events (SSE)

For applications where the Web client mainly needs to get updates from a server, the Server-Sent Event API (SSE) provides an easy and efficient solution for the implementation of such a scenario.

When using this API, the Web client will create an EventSource instance that opens a persistent HTTP connection to the Web server. Using this connection, the server can then send events in the text/event-stream format. Details about the text/event-stream format are given in the lecture slides and in the format specifications.

On the Web client side, the use of the Fetch API for getting updates from the server is replaced by the use of the EventSource API. The client creates an EventSource instance by specifying its url (as for the Fetch request). This will open an HTTP connection to the server and once this connection is opened, the server will be able to send messages to the client. The client will receive the messages in the form of events in the text/event-stream format. For each type of event, the client may register a callback function. The events that can occur are:

  • ‘error’: fired when a connection to an event source failed to open.
  • ‘message’: fired when data is received from an event source.
  • ‘open’: fired when a connection to an event source has opened.

Additionally, other ad-hoc events will be created if the server sends messages with an event field set to a specific value (see the lecture slides and https://www.w3.org/TR/2009/WD-eventsource-20090421/#event-stream-interpretation for more details).

The code on the client side is:

index_sse.html
index_sse.html
<!doctype html>
<html lang="en">
<head>
    <title> Getting the score </title>
    <!-- jQuery javascript integration -->
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
</head>

<body>

<button onclick=getScore()>Launch score update</button>
<p id="nbrOfMessages"> Number of messages is 0</p>
<p id="score"> No score </p>

<script type="text/javascript">

  let nbrOfMessages = 0;

  function getScore() {

    let sseClient = new EventSource("http://localhost:8080/sse");

    // set the listener for receiving data notifications
    sseClient.onmessage = function (event) {
      if (event.data === "") {
        console.log("event with no data received");
        return;
      }

      let message = event.data;
      console.log(event.data);

      // update the message
      $("#score").html(message.toString());

      // update the number of calls value
      nbrOfMessages++;
      $("#nbrOfMessages").html("Number of messages received is " + nbrOfMessages);
    };

    // In case of error, stop the observation
    sseClient.onerror = function (event) {
      if (event.readyState === EventSource.CLOSED) {
        // Connection was closed
        console.log("Connection was closed");
        sseClient.target.close();
      }
      else {
        console.log("Error in sseClient");
        if (typeof sseClient.target != 'undefined') {
          sseClient.target.close();
        }
      }
      sseClient = null;
    }
  }

</script>
</body>
</html>

The important parts of the client code are:

  • The creation of the EventSource instance with its url.
  • The definition of the function to be called when a message is received from the server.
  • The definition of the function to be called upon error.

The Node.js Express application code is:

servers/sse.js
servers/sse.js
const express = require('express');
const cors = require('cors');
const SSE = require("sse-node");

let score = "0:0";

// sse options: ping will force a ping sent by server at the given interval
const sseOptions = { ping: 60000 };

// SSE client instance
let clientSSE;

process.stdin.on('readable', () => {
  let chunk;
  // Use a loop to make sure we read all available data
  while ((chunk = process.stdin.read()) !== null) {
    // simply update the score
    score = chunk.toString();
    // if we have a existing SSE client, send the data
    if (clientSSE != null) {
      clientSSE.send(score);
      score = "";
    }
  }
});

const app = express();

// allow all origins
app.use(cors());

// serve basic polling route
app.get('/sse', function (req, res, next) {

  // create the sse client
  clientSSE = SSE(req, res, sseOptions);

  // upon closing of the HTTP connection by the client simply log and reset score
  clientSSE.onClose(() => {
    score = "0:0";
    console.log("SSE client was closed");
  });

  // check if there is data from stdin
  // if yes, deliver it right away by sending a message
  // otherwise do nothing
  if (score.length > 0) {
    clientSSE.send(score);
    score = "";
  }
});

module.exports = app;

The important changes as compared to the long polling application are:

  • Upon GET request on the ‘/sse’ route, an instance of SSEClient is created. No response is ever sent to the client.
  • Any updated score is sent to the client as a SSE message, using the SSEClient instance.

Do not forget to change the Express application to be used in your server and test the SSE implementation of the “Getting the score” application. At startup, the score “0:0” should be displayed and any update on the server should be immediately available on the client application. If you observe traffic, you should see that an EventSource has been created for the entire duration of the application and that the time information is updated each time a message is sent from the server:

SSE traffic

SSE traffic

From this application, it should become clear that server-sent events are unidirectional, meaning that messages are delivered in one direction only, from the server to the client. This is shown in the diagram below:

SSE

SSE

So SSE is a good choice whenever there is no need to send data from the client to the server using a message form. This makes it a good choice for WebOfThings applications where a lot of data is delivered from things to web clients.

For applications requiring bidirectional messaging, another approach is required. A very popular approach is the use of the WebSocket API. We introduce this API in the next step.

WebSocket

The WebSocket API is a modern technology that makes bidirectional messaging between a Web client and a Web server possible. With this technology, the Web client can send messages to a server and receive responses driven from events on the server, without having the client poll the server for a response.

On the client side, the use of the WebSocket API is very similar to the use of the EventSource API. Once the WebSocket instance is created by the client, a number of callback functions are set to this instance. These callback functions will then be executed upon specific events.

One of the main differences between the EventSource API and the WebSocket API is that the WebSocket API is not built on the HTTP protocol but rather on the WebSocket protocol (RFC 6455 - The WebSocket Protocol). As stated in the specifications, the goal of this technology is to provide a mechanism for browser-based applications that need two-way communication with servers that does not rely on opening multiple HTTP connections (e.g. using XMLHttpRequest and long polling).

WebSocket client applications use the WebSocket API to communicate with servers implementing WebSocket. On its side, the server must listen for incoming socket connections using a standard TCP socket and following the protocol. For bridging from HTTP to WebSockets, a handshake happens between the client and the server. During this handshake, connection parameters are negotiated and, if each party agrees on the terms, the WebSocket is then opened and allows bidirectional communication between the server and the client.

The client code is very similar to the one of Server-Sent Events web client. The principle is the same:

  • The client must create a WebSocket instance with the corresponding url.
  • Once the instance is created, one must set callback functions for the different events

The code for the client is:

index_ws.html
index_ws.html
<!doctype html>
<html lang="en">
<head>
    <title> Getting the score </title>
    <!-- jQuery javascript integration -->
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
</head>

<body>

<button onclick=getScore()>Launch score update</button>
<button onclick=closeWS()>Close WebSocket</button>
<button onclick=sendDateMessage()>Send date message</button>
<p id="nbrOfMessagesIn"> Number of received messages received is 0</p>
<p id="nbrOfMessagesOut"> Number of sent messages is 0</p>
<p id="score"> No score </p>

<script type="text/javascript">

  let nbrOfMessagesIn = 0;
  let nbrOfMessagesOut = 0;

  let socket = null;

  function getScore() {
    // Create WebSocket connection
    socket = new WebSocket('ws://localhost:8080');

    // Connection opened
    socket.addEventListener('open', function (event) {
      socket.send('WebSocket opened!');
    });

    // Listen for messages
    socket.addEventListener('message', function (event) {
      // test the type of data
      console.log("Received data string: " + event.data);

      if (event.data === "") {
        console.log("event with no data received");
        return;
      }

      let message = event.data;
      console.log(event.data);

      // update the message
      $("#score").html(message);

      // update the number of received messages
      nbrOfMessagesIn++;
      $("#nbrOfMessagesIn").html("Number of messages received is " + nbrOfMessagesIn);
    });

    // In case of error, close the websocket
    socket.addEventListener('error', function (event) {
      console.log("Error");
      socket.close();

      // check the reason for the error
      if (event.code !== 1000) {
        // Error code 1000 means that the connection was closed normally.
        // check whether we are still connected to the Internet
        if (!navigator.onLine) {
          alert("You are offline. Please connect to the Internet and try again.");
        }
        else {
          // we could try to reconnect
        }
      }
    });

    socket.addEventListener('close', function (event) {
      // Connection was closed
      console.log("Connection was closed");
      socket.close();
    });
  }

  function closeWS() {
    if (socket != null) {
      socket.close();
    }
  }

  function sendDateMessage() {
    if (socket != null) {
      let msg = Date().toString();
      socket.send(msg);

      // update the message sent
      $("#messageOut").html(msg);

      // update the number of sent messages
      nbrOfMessagesOut++;
      $("#nbrOfMessagesOut").html("Number of sent messages is " + nbrOfMessagesOut);
    }
  }

</script>
</body>
</html>

For demonstrating bidirectional communication, this code also includes a button and a function for sending a message from the Web client to the Web server. In this case, the message received by the server is displayed in the console of the Web server application. This bidirectional communication capability is represented in the diagram below:

Bidirectional traffic

Bidirectional traffic

The code of the Web server application is:

servers/ws.js
servers/ws.js
const http = require('http');

// HTTP Server
const port = 8080;
const server = new http.createServer();

server.listen(port, function () {
  console.log('HTTP server started...');
  console.info(`Your service is up and running on port ${port}`);
});

// create the WebSocket server
const WebSocket = require('ws');
const wss = new WebSocket.Server({ server });

let score = "0:0";

process.stdin.on('readable', () => {
  let chunk;
  // Use a loop to make sure we read all available data
  while ((chunk = process.stdin.read()) !== null) {
    // simply update the score
    score = chunk.toString();
    // if we have a existing web socket, send the data to all clients
    if (wss != null) {
      wss.clients.forEach(function each(client) {
        if (client.readyState === WebSocket.OPEN) {
          client.send(score);
        }
      });
    }
  }
});

wss.on('connection', function connection(ws) {
  ws.on('message', function incoming(message) {
    console.log('received: %s', message);
  });

  ws.on('close', function close() {
    console.log('disconnected');
  });

  ws.send(score);
});

Note that the node application must be started directly with the “servers/ws.js” file and not with the “update-server.js” file. The HTTP server is created and started in the servers/ws.js file together with WebSocket server. As explained above, the WebSocket will listen for socket connections initiated by the WebSocket client.

You can observe in the server code that the server keeps track of clients’ sockets and sends data to all WebSocket clients. This mechanism is also required for keeping track of handshaking requests made by clients.

For illustrating the handshake required for establishing the WebSocket connection, you may observe network traffic at the time you hit the “Launch score update” button. The WebSocket handshake is started by the Web client that contacts the server and requests a WebSocket connection. The client sends a standard HTTP request for switching protocols and with headers that looks like:

Protocol switch

Protocol switch

Protocol upgrade

Protocol upgrade

The HTTP version must be 1.1 or greater and the method must be a GET method. The important parts of the Request headers pertaining to the WebSocket are

  • Connection: Upgrade
  • Upgrade: websocket
  • Sec-WebSocket-Key:…
  • Sec-WebSocket-Version: …
  • Sec-WebSocket-Extensions: …

When the server receives the handshake request, it should send back a dedicated response that indicates that the protocol will be changing from HTTP to WebSocket. The Response headers looks like:

Handshake response

Handshake response

For the sake of simplicity, we do not address extension requests here. We don’t address how the Sec-WebSocket-Key is handled, except that it is required for establishing a secure WebSocket connection between the client and the server.

Once the WebSocket connection is established, messages can be exchanged bidirectionally between the client and the server, as you can observe in the Messages tab of the Network window of your browser:

Websocket traffic

Websocket traffic

This illustrates the capability of the WebSocket framework of implementing bidirectional communication, unlike the SSE framework where communication is unidirectional from the server to the client. WebSocket is thus a solution in cases where bidirectional communication is required.