In-order responses for asynchronous work

Posted by David Owen on 2023-02-08

Sometimes we end up executing some asynchronous function several times in a row, but we need only the results of the last call. The difficulty is that some earlier invocations may finish after the latest. I encounter this most often in Javascript, when I call an API in response to ongoing user input, like for looking up an address. While debouncing can help reduce this problem (and should be done anyway to lighten the load), it does not eliminate the problem.

A simple way to do this is to use a counter to keep track of each request, and only process a response if it's newer than any other processed so far.

First, each time we prepare a request, we'll grab a new ticket from the counter. Whenever we process a request, we'll mark that ticket as handled. Both the counter and the last-handled ticket will be global state. The ticket associated with a particular request, however, will be tracked as local state; each request will get its own.

var ticket_counter = 0;
var response_ticket = 0;

function request_update() {
    ticket_counter += 1;
    const request_ticket = ticket_counter;

    const response = await fetch(url);

    // ...
}

At this point, we do the actual work. This is almost always asynchronous. Whether you use async/await or use a Promise API doesn't matter. We use await in this example for brevity.

Now, the current task is suspended and the runtime may execute other tasks. These other tasks may also run request_update() and increment the global ticket_counter. However, no task can change any other task's local request_ticket.

Once the work completes, the runtime will resume executing our task, returning from the await.

For now, we'll always handle the response and update response_ticket.

async function request_update() {
    // ...

    const response = await fetch(url);

    update_display(response);
    response_ticket = request_ticket;
}

The problem with out-of-order responses is now observable, not just by confusing the user, but by looking at the tickets. Specifically, while request_ticket always increases, response_ticket might go backwards.

We'll add some guards to handle this situation. Depending on your application, you might want to display results to the user as they come in, or you might want to wait until the final results arrive. We'll support either approach.

async function request_update() {
    const response = await fetch(url);

    const is_last = request_ticket == ticket_counter;
    const is_fresh = request_ticket > response_ticket;

    if (is_last || is_fresh) {
        update_display(response);
        response_ticket = request_ticket;
    }
}

If you want only final results, instead of incremental, change the condition to:

if (is_last) {
    // ...
}

Finally, we'll display and hide a spinner to let the user know whether updates are still waiting.

The final code (with no error-handling), is:

var ticket_counter = 0;
var response_ticket = 0;

async function request_update() {
    ticket_counter += 1;
    const request_ticket = ticket_counter;

    show_spinner();

    const response = await fetch(url);

    const is_last = request_ticket == ticket_counter;
    const is_fresh = request_ticket > response_ticket;

    if (is_last) hide_spinner();
    if (is_last || is_fresh) {
        update_display(response);
        response_ticket = request_ticket;
    }
}

This is suitable when you have only a single type of request for which you need to ensure in-order handling. If you have several, change the two global variables to be keys in a map, and identify that map in another map keyed off the type of request:

var tickets = {};

var ticket_counter = 0;
var response_ticket = 0;

async function request_update(name, url, update_fn) {
    if (!(name in tickets)) {
        tickets[name] = {
            counter: 0,
            response: 0
        };
    }

    const ticket_state = tickets[name];
    ticket_state.counter += 1;
    const request_ticket = ticket_state.counter;

    show_spinner();

    const response = await fetch(url);

    const is_last = request_ticket == ticket_state.counter;
    const is_fresh = request_ticket > ticket_state.response;

    if (is_last) hide_spinner();
    if (is_last || is_fresh) {
        update_fn(response);
        ticket_state.response = request_ticket;
    }
}

David Owen’s blog

In-order responses for asynchronous work

Trackbacks

Comments

Add Comment