I use async
/await
pretty often at work, but how the syntax is applied under the hood has always been a bit of a mystery to me.
Like many JavaScript developers, I know that await
allows us to write asynchronous operations in a synchronous-like manner. I also know that async
functions always return a JavaScript promise. I kind of know that async
/await
is built off JavaScript generators, but it’s not clear exactly how the two concepts might be connected.
This last point, in particular, is the bit that gets to me. What if we didn’t have async
/await
? How would we derive it from other JavaScript features? Could we build our own asyncify
function?
Iterators and Generators
Let’s start with a quick refresher on iterators and generators.
Generators are simply special iterators. An iterator is an object that implements the iterator API. This API expects any iterator object to contain a .next()
method which, when called, returns an object in the form of { done: [some boolean], value: [some value] }
.
Iterators generally deal with collections of data. For example:
// Given an array
const arr = [1, 2, 3];
// We can access its default iterator
const iter = arr[Symbol.iterator]();
console.log(iter.next()); // { value: 1, done: false }
console.log(iter.next()); // { value: 2, done: false }
console.log(iter.next()); // { value: 3, done: false }
console.log(iter.next()); // { value: undefined, done: true }
Generators are special in the sense that they allow us to go beyond working with static collections of data. With generators, we can repeatedly start and pause a function’s execution, which enables us to produce dynamic collections of data.
// Defining a generator function
function* addGen(x, y) {
const first = yield x;
const second = yield y;
return first + second;
}
// Instantiating a generator
const gen = addGen(1, 2);
// Running a generator
console.log(gen.next()); // { value: 1, done: false }
console.log(gen.next(5)); // { value: 2, done: false }
console.log(gen.next(5)); // { value: 10, done: true }
Running to Completion (Synchronously)
The ability to pause a function and continue executing it later is an incredible language feature. In fact, it’s crucial in the context of writing synchronous-looking asynchronous code –– but we’ll come to that later.
For now, let’s forget about asynchronicity and just figure out out how we can run synchronous generators to completion.
Recall that every .next()
call accepts arguments that can be freely defined by the developer.
// Defining a generator function
function* addGen(x, y) {
// `first`: Arg passed into gen.next() *after* yielding 1
const first = yield x;
// `second`: Arg passed into gen.next() *after* yielding 2
const second = yield y;
return first + second;
}
This is why our final call to gen.next
in the previous example returned 10
instead of 3
, despite the fact that addGen
was called with 1
and 2
:
// Instantiating a generator
const gen = addGen(1, 2);
// Running a generator
console.log(gen.next()); // { value: 1, done: false }
console.log(gen.next(5)); // { value: 2, done: false }
console.log(gen.next(5)); // { value: 10, done: true }
To make our generators run to completion without the need of any kind of developer-defined input, we’ll make it requirement that gen.next()
is always called with the previously-yielded value:
// Instantiating a generator
const gen = addGen(1, 2);
// Running a generator
const yielded1 = gen.next();
const yielded2 = gen.next(yielded1);
console.log(gen.next(yielded2)); // { value: 3, done: true }
If you think about it, this is practically equivalent to “converting” addGen
into something like:
function add(x, y) {
const first = x;
const second = y;
return first + second;
}
It’s almost as if we’ve removed the generator syntax and stripped addGen
of the yield
keyword.
Let’s define a conversion function, getGenRunner
, which returns a new function that executes the provided generator function as if it wasn’t a generator.
function getGenRunner(generatorFunc) {
return function genRunner(...args) {
const gen = generatorFunc(...args);
function processNextState(nextState) {
const { done, value } = nextState;
// Terminal case: Generator is done
if (done) {
return value;
}
return processNextState(gen.next(value));
}
const returnValue = processNextState(gen.next());
return returnValue;
}
}
const add = getGenRunner(addGen);
add(1, 2); // 3
In the above code, all genRunner
does is that it repeatedly invokes gen.next
with the previously-yielded value until the generator has been marked as done
.
At this point, you might be thinking, “Why go through all this trouble of turning generators into non-generators?” If building an asyncify
function is our goal, it seems like we’ve taken a step back rather than a step forward.
To see why, I’ll need you to bear with me for just a little while longer! Even though we’ve sort of hidden the fact that add
is built on top of a generator, we’ll soon be using these hidden generator features to handle asynchronicity.
Running to Completion (Asynchronously)
Consider the following code:
// Defining a generator function
function* delayedAddGen(x, y) {
const first = yield new Promise((resolve) => {
setTimeout(() => resolve(x), 1000);
});
const second = yield y;
return first + second;
}
// Instantiating a generator
const gen = delayedAddGen(1, 2);
// Running a generator
const nextState = gen.next();
console.log(nextState) // { value: <Promise>, done: false }
Running the generator for the first time yields an object wrapping a Promise. We can’t simply invoke gen.next
with this Promise. Instead, we’ll need to wait for the Promise to resolve and then continue the generator’s execution with the resolved value.
To forward the resolved value to the next generator call, we could subscribe gen.next
as a promise fulfillment callback on the next line:
// Running a generator
const nextState = gen.next();
console.log(nextState) // { value: <Promise>, done: false }
nextState.value.then(gen.next);
This is nice, but how do we integrate this with getGenRunner
? It seems that our requirements have expanded, as we now need to handle synchronously-evaluated values as well as asynchronously-evaluated ones (i.e. Promises).
Since we’re no longer only dealing with synchronous behaviours, it seems fitting to rename getGenRunner
to asyncify
. This is what we’ve been working towards the whole time –– asyncify
will help us convert generator functions into async
/await
-like functions.
function asyncify(generatorFunc) {
return function asyncifiedFunc(...args) {
const gen = generatorFunc(...args);
function processNextState(nextState) {
const { done, value } = nextState;
if (done) {
return value;
}
// Case 1: Yielded a Promise
if (
typeof value === "object" &&
typeof value.then === "function"
) {
return value.then(resolvedValue => {
return processNextState({
done,
value: resolvedValue
})
})
}
// Case 2: Yielded a non-Promise
return processNextState(gen.next(value));
}
const returnValue = processNextState(gen.next());
return Promise.resolve(returnValue);
}
}
const delayedAdd = asyncify(delayedAddGen);
delayedAdd(1, 2).then(x => console.log(x)); // 3
Note that I’ve called Promise.resolve
on the returnValue
. Since async
functions always return a Promise, we want to replicate that behaviour here as well.
In other words, even generators that purely contain synchronous operations will still be asyncify
-ed into Promise-returning functions:
const add = asyncify(addGen);
add(1, 2).then(res => console.log(res)); // 3
Considerations: What is this?
Our implementation of asyncify
is not terrible, but there are a number of things we ought to improve on.
First of all, let’s think about how context objects will be bound. What if our input generator functions contained references to this
?
function* contextualAddGen() {
const first = yield this.x;
const second = yield this.y;
return first + second;
}
The key here is to use .call
to bind the context of the underlying generator in asyncify
:
function asyncify(generatorFunc) {
return function asyncifiedFunc(...args) {
// Invoke `.call`
const gen = generatorFunc.call(this, ...args);
function processNextState(nextState) {
// ...
}
return processNextState(gen.next())
}
}
const myObj = {
x: 1,
y: 2,
add: asyncify(contextualAddGen)
}
myObj.add().then(res => console.log(res)); // 3
To learn more about binding generators, I recommend checking out this superb article by Ben Nadel.
Considerations: Error Handling
The other critical consideration relates to error handling. Given that we’re handling synchronous and asynchronous code paths, we want to make sure that both types of errors eventually cause the Promise returned by the asyncified function to be rejected.
function asyncify(generatorFunc) {
return function asyncifiedFunc(...args) {
const gen = generatorFunc.call(this, ...args);
function processNextState(nextState) {
const { done, value } = nextState;
if (done) {
return value;
}
if (
typeof value === "object" &&
typeof value.then === "function"
) {
return value.then(resolvedValue => {
return processNextState({
done,
value: resolvedValue
})
}).catch(err => {
// Invoke gen.throw
return processNextState(gen.throw(err))
});
}
return processNextState(gen.next(value));
}
try {
const returnValue = processNextState(gen.next());
return Promise.resolve(returnValue);
} catch (err) {
return Promise.reject(err);
}
}
}
There are two cases to handle.
First, if one of our yielded Promises gets rejected, we make sure to propagate that error to the generator in the .catch()
callback. We’ll use gen.throw()
, which allows us to fire errors at generator entry points (i.e. places where the generator resumes its execution).
Second, if any errors remain uncaught in the generator itself, then the asyncified function’s returned Promise must be in a rejected state –– so we return Promise.reject(err)
instead of returning a resolved value.
Ideally, the generator functions we pass to asyncify
should already contain their own try
/catch
blocks and handle any errors in advance for us. That said, we cannot assume that this will always be the case. Thus, it is necessary for us to guard against such errors within asyncify
itself.
Concluding Thoughts
With that, we’ve implemented a pretty decent version of async
/await
using our very own asyncify
function. To be clear, I don’t believe that the underlying behaviours surrounding async
/await
map entirely to the implementation I’ve outlined above –– certain aspects, such as error messages annd their associated stack traces, are likely to differ.
Regardless, the goal of this article was to hammer home the strong relationship between async
/await
and generator functions, and I hope it’s achieved that. In particular, I hope this article has helped you better appreciate the language features and coding patterns that have enabled us to reason about asynchronous flows in JavaScript a lot more intuitively.