Escape from Callback Hell

Escape from Callback Hell

·

14 min read

Follow me on Twitter @ javascriptual (78.9K followers) for tutorial updates.


Assuming you already have a solid grasp on async and await you've probably already overcome the problem of callback hell. So why this tutorial now?

Regardless of JavaScript as a maturing language many are starting to learn how to code from scratch as we speak. Inevitably on their journey they will too end up in Callback Hell. Or more likely .then() hell, as even beginners often jump direct into ES6+ features such as promises nowadays.

If you keep getting stuck in callback or then() Hell pattern it’s a good time for learning async and await. If you know you wanted to for a long time but for some reason…you just never got to it then this tutorial may be for you.

We will plan a tactical escape from callback hell with a promissified async/await solution. If you are looking to be employed in software industry this is a common stuck point.

I've been there before and it's quite embarrassing when you are the only person on the entire team who didn't spend the time to learn how async / await actually works. I don't ever want you to be in that spotlight.

First things first: Going back in JavaScript history

It's hard to understand because there is some JavaScript history behind it. The language evolved for a long time. New features were added over the years.

Going back in history to understand callbacks and promises is only first step. But once we tackle that, you will finally have the "ah ha" moment about how async/await actually works and why it works the way it does.

Only knowing about existence of keywords then, async &await is knowing nothing at all. You want to gain thorough understanding once and for all.

I know this tutorial is long but if you're new to webdev it'll save you time in long run. Note: all this wouldn't be possible for me to learn myself without help of my teammates & industry experience gained over the years.

Callbacks

Before we narrow down on the problem created by callback, Promise and .then() hell, we need to take a look at a few callback function examples first.

Large scale projects require executing multiple callbacks and dealing with them in an elegant way using clean code. If you're not prepared to deal with it you might get stuck for hours or commit crappy code that won't pass code review.

What the Hell are callback functions?

In JavaScript code written in a rush you will often find yourself going in circles because, basiucally, it's impossible to predict exactly when a callback event function will return.

Let's start with the very basics. When you have a simple function it returns the result using return keyword:

function message() {
    return "Hello world.";
}

message(); // Evaluates to "Hello world."

Most User Interfaces are generally event-based in some part due to human element. Who knows when user is going to click a button?

In many cases delays are due to the hardware (reading files in Node or using fetch API to execute an HTTP request, for example.)

When an event happens at a later time that's when a callback event function will execute to let us know that event has occurred. Or more precisely that it has "finished occurring" and we can collect some type of data from it.

(Mouse click coordinates, data of the file that was just read, return message from an HTTP request, etc.)

A mouse click produces a callback event.

function callback(event) {
    // do stuff with "event" object returned from click event
}

// Grab the HTML object of element <div id = "element">
const element = document.getElementById("element");

// Attach click event to the element
element.addEventListener("click", callback);

// Make sure to at some point remove the callback when done:
// (Requires passing same callback function in order to work)
element.removeEventListener("click", callback);

Within callback function in above example we can find out where mouse was clicked on the client area, etc. In this callback hell tutorial the important part to understand at this point is that an event returns at a later time in a separate execution thread (not to be confused with a separate process.)

Callback functions can be used for executing HTTP requests and tracking mouse click position on client area. In many cases it's unknown exactly when the callback will be executed. That creates a problem.

So what's the point of all this?

We simply don't know exact time. That's why we have callback functions. They "call you back" at a later time to let you know an event has occurred. (Although actual meaning is of course focused on calling a function at a later time. The phone call reference is just a metaphor I made up.)

JavaScript allows anonymous and arrow functions used as callbacks:

Perhaps this is more of an offside subject but it's good to know there are two types of functions in JavaScript that can be used as callbacks.

Most of the differences are syntactical. But how each function handles this object is different. Also, arrow functions don't have a constructor, can't be used to instantiate an object, and are not hoisted (because technically arrow functions are still just expressions.)

Regular anonymous callback function:

element.addEventListener("click", function(args) { /*...do stuff...*/ });

Arrow function (was added in ES6 specification)

element.addEventListener("click", args => { /*...do stuff...*/ });

Or arrow function with multiple arguments (notice added parenthesis:)

element.addEventListener("click", (arg1, arg2, arg3) => { /*...do stuff...*/ });

Note that arrow functions don't have their own this object. They borrow it from whatever this equals to outside of their own scope.

Note: the following examples do not explain how code should be written. However as someone new to web development and growing in your skillset, this is the pattern you will inevitably find yourself trying to implement before grasping promises and async. If you find yourself in that place when trying to solve a complex problem you've gotten yourself into Callback Hell.

The classic Callback Hell example

This is when you need to create numerous API calls in a row. That's because next API call requires data returned from the previous one. The first solution that comes to mind is simply nest API calls in their own callbacks:

(this is pseudo-code but could be any call that results in a callback.)

CallEndpoint(`api/getidbyusername/hotcakes`, result => {
    CallEndpoint(`api/getfollowersbyid/${result.userID}`, result => {
        CallEndpoint("api/someothercall/" + result.followers, result => {        
            CallEndpoint("api/someothercall/" + result.followers, result => {
                CallEndpoint("api/someothercall/" + result.followers, result => {

                });
            });            
        });
    });
});

There was a time in JavaScript history when this is all we had. This bad coding practice was actually the most "elegant" solution.

Later this led to creation of promises and .then() methods (and later async.)

This made code look a bit cleaner. But the problem persisted and transformed intself into thenable hell:

The .then() Hell

Same thing only with promises.

promise.then(resolve, reject) { }).then(result =>{
    promise.then(resolve, reject) { }).then(result =>{
        promise.then(resolve, reject) { }).then(result =>{
           /* then hell */
        });
    });
});

While it works, this isn't a great idea for writing code. If you find yourself writing code like this you will probably get fired after first code review.

All this means is simply that you're graduating to the next level.

It's time to start applying promises for their intended purpose in combination with async/await keywords. But first a couple more examples and then we'll dive right into the solution for escaping callback hell.

A Fetch API call in JavaScript produces a callback event.

Another popular example of a callback function is in a fetch API call.

Before showing the fetch call, create a helper function make() to package your JSON payload with redundant POST (or GET if you need it) object that basically tells the API the data is in JSON format:

// Redundant everywhere so put it in a neat make() function
let make = function(payload) {
    return { method: 'post',
            headers: { 'Accept': 'application/json',
                 'Content-Type': 'application/json' },
            body: JSON.stringify(payload) };
}

And finally the fetch event for "follow user" event in the server-side API:

// Sample payload for "follow user" API call
const payload = make({token: token, userid: 15427});

fetch(`https://${Site.www}example.dev/api/user/follow`, payload).then(
promise => promise.json()).then(json => {
  if (json.success) {
    /* update UI state */
  } else {
    /* something's wrong */
  }
});

Notice fetch API call returns a promise. And if you want to quickly find out how promises work here's a diagram:

image.png

By the way it's taken from my coding book JavaScript Grammar check it out.

So what is the difference between a callback and a promise? Actually, I had to look this up myself and Mozilla has a better, more detailed, answer:

Guarantees. Unlike old-fashioned passed-in callbacks, a promise comes with some guarantees: Callbacks will never be called before the completion of the current run of the JavaScript event loop. Callbacks added with then() , as above, will be called even after the success or failure of the asynchronous operation. ( source )

All Promise objects have then() method. We'll get back to that in a bit. But basically, promises are often used together with making an API call. And then decide what to do with the returned data: reject or resolve.

We'll take a closer look at reject and resolve Promise methods as part of our final solution as we get closer to the escape from callback hell.

In source code listing above API tried to follow a user. The payload included token of the currently logged in user and userid of the user to follow. Express on the server-side will crunch this data talk to a Mongo database and return state whether the user was successfully followed or not.

Reading a file in Node produces a callback event.

Let's say you are writing backend code in Node.

A common operation is to read a file:

/* Include file system module (read, write, delete, etc.) */
let fs = require('fs');
let app = express();

/* Read some file on the server */
app.get('/api/file/read', function (req, res) {

    /* read some file from filesystem */
    let data = fs.readFileSync(`/var/www/amazing.txt`, `utf-8`);

    /* send file to the front-end */
    res.send(data);
});

(only partial Express code shown to demonstrate server-side callbacks.)

On the front-end you would call fetch API to /api/file/read endpoint and your code will wait until file is read and returned from the server. (Express's res.send() command.)

But if amazing.txt is gigabytes long, it's going to take some time not only for HTTP request to take place but also to read the file.

Who knows when the callback will actually return? The setup is complicated by callbacks, Event Loop, network connectivity and hardware. There are simply too many unknowns. And this is part of the problem.

Even without HTTP requests and file reads the exact execution time of a regular callback function is unknown. It's being determined internally by JavaScript's V8 engine (in Chrome) inside of something called the Event Loop.

The Event Loop is core mechanism used by JavaScript to keep track of the API calls, Call Stack for tracking order of executed functions, events, micro tasks and macro tasks.

You don't have to worry about the Event Loop when coding. But questions about Event Loop (and stuff like prototype) are among some of the most common questions on tech job interviews. So I'd learn that separately. Event Loop does a lot of stuff in the background.

The problem of executing multiple callbacks at the same time

I know a callback function in itself is not a return value. But it can be potentially thought of as being similar to a return value in that (when promissified) can yield a return value.

The callback function executes at some point during the execution flow in the timeline of your program. Exact time is not known. That creates a problem.

What if we need to execute another callback event that relies on return value created from previous callback?

In previous callback hell example we tried to fetch a list of followers from a user, first we need to fetch the id of that user. And then pass it to another callback. But the code was ugly.

Final Solution to Callback Hell

First create a function to do all important work -- whatever it may be.

This function must return a new promise. I created one with new Promise.

I recently wrote code to get email list from giveaways marked by unique id. My CSS Visual Dictionary book yielded 420 giveaway entries. My Node API book giveaway produced 1300 entries. So I had to count them separately.

So I named it get_giveaway and required an id as one of its arguments:

function get_giveaway(id) {

    // Return new promise
    return new Promise((resolve, reject) => {

      // This is your thenable action (whatever it may be)
      DoThenStuff().then(result => {

        if (result.success)
            resolve(result); // return value passed to resolve() function
        else
            reject(result); // something went wrong


      }); // close DoThenStuff

    }); // close return new Promise

} // close get_giveaway

In this simple function, do your thenable action. Which will often be executing a fetch API call to the API endpoint server.

In this case I used a made up function DoThenStuff() as an example, that supposedly returns another promise, but it can be any "thenable" function.

Instead of return keyword use resolve(result). That's just how promises "return" a value if the function has succeeded. This is a thenable "return". It will return later. If there was an error, use reject(result) method.

Now the fun part. The key to make this work right is 2nd function. That's where you will do all of your thenable actions in a row and they will return in same order as you execute them :)

This is the magic of async / await

Now define a new function main() (name it whatever you want) and precede it with async keyword. Inside it add await in front of each promissified function we created earlier (they will "await" until previous one returns):

async function main() {

  const a = await get_giveaway(1);
  const b = await get_giveaway(2);
  const c = await get_giveaway(3);

   /* Code here will not execute until all promises return */

  const d = get_giveaway(4);

  // pass d to next func (if it needs it)
  const e = get_giveaway(d);

  // they're all here!
  console.log(a,b,c,d,e);
}

This code will literally wait until a callback returns until proceeding to the next.

Finally, somewhere else in your program:

// Wait for DOM elements to finish loading:
window.addEventListener('DOMContentLoaded', event => {
    main(); // do magic!
});

Then just call main() from anywhere in your program as a regular function. Boom. That's the pattern you're looking for. It's clean. It's asynchronous. But it's modular and imperative. (You simply write code in the order you would expect those commands to execute as if they were synchronous.)

The Event Loop will takes care of all the magic going on in between the calls. You'll simply just focus on programming as you normally would. At this point you can say you've successfully defeated Callback Hell.

Not shown here but notice how it is possible for async and await functions to be hidden in an isolated module. Once written you won't ever have to see its contents again. Simply call main() from anywhere in your code.

This way you get data privacy that comes with modular encapsulation and execute asynchronous code while writing it in regular imperative style.

This is how you were trying to write your program before understanding promises and how async / await works.

With this new simple two-function setup you can now do exactly that - retain asynchronous nature of your code and keep your code clean.

#Octopack is my coding book bundle.

Hey readers! At the end of each tutorial I advertise my coding book bundle. I don't know if it's going to work as I don't really do marketing well. But I decided to follow this protocol just to see what happens.

The pack includes CSS Dictionary, JavaScript Grammar, Python Grammar and few others. Just see if you might need any of the books based on where you are on your coding journey! Most of my books are for beginners-intermediate XP.

I don't know who is reading my tutorials but I put together this coding book bundle that contains some of the best books I've written over the years. Check it out 🙂 I reduced price to some regions like India, Nigeria, Cairo and few others. Prices are just so different here in the U.S. and all over the world.

You can get Octopack by following my My Coding Books link.

css book

css flex diagram