01/09/2020

Callbacks, Promises, Generators and Why a JS Developer needs to understand them.

I've had my fair share of up and downs with Javascript and it always is frustrating when the language you've been using for a while doesn't behave the way you assumed it would but I continued using JS for almost everything I built over the past 2-3 years.

Why Javascript?

I'm not biased towards the language, I would like to get back to being a C developer or maybe be a little modern and be a Go/Rust Developer but as I mentioned in my previous post, I've been a little bitch about it and keep running back to JS for moral support.

But, I still think that learning JS is a valuable skill. Learning any programming language at this point is.

We though are going to go with JS to start because I can explain stuff about it a little more than I can explain C, Go, or Rust.

It all Started with him... the dreaded one

Callbacks

Veterans love them, newbies fear them and others have no idea what's going on.

The thing about callbacks is that we are all using them in almost every JS codebase and still fail to realize that we are.

Anyway, getting to the basics.

What are they?

It's a function. A function trapped inside another one to be precise but it's still a function so let's treat them like one.

function functionOne(callbackFunction) {
  var a = 1;
  callbackFunction(a);
}

function functionTwo(num) {
  console.log(num);
}

// Variation One
functionOne(function (value) {
  functionTwo(value);
});

// Variation Two
functionOne(functionTwo);

Now, before I explain the above, I'm assuming you understand that functions in JS aren't considered different than general parameters and thus, you can pass them down to other functions.

This is also allowed in other languages as well so should not come as a surprise anymore to people who've been jumping languages or to people who've been dealing with JS a lot.

Let's go through the code snippet now.

We've got 2 functions to start with functionOne and functionTwo, functionOne takes in a parameter called callbackFunction which could be anything, a string a number or even a boolean or an object/array for that matter, but I'm going to keep it simple for us to understand and not add type checks at this point (which you should add if you are writing in just JS, ignore if you use TypeScript).

functionTwo on the other hand has the same parameter signature or accepts the same number of arguments as functionOne.

If we now look at the inner code of these two we see that one declares a variable a and executed callbackFunction and passed in that value (again, functionOne assumes that callbackFunction is going to come in as a function and so blindly executes it.

functionTwo's inner code is logging the passed parameter to the console/stdout(depending on where you are executing this snippet).

Execution

After the declarations we have 2 variations for the execution of our functions, one being a little verbose and the 2nd being my definition of readable code.

  1. The first variation basically calls functionOne and passes it another function as a parameter which is called an anonymous function (guess why) and this anonymous function surprisingly has a value parameter, we didn't declare it, so how does it get it? functionOne passes it to anonymous function when we made the callbackFunction(a) and callbackFunction is now pointing to our anonymous function because this is what we passed as a parameter and then we just call our functionTwo and pass it the received value.

  2. The second variation is used when there's only one function that needs to be executed with the incoming value from functionOne, you should still go with Variation one if you're going to use the value more than once. Now this works because we are still passing functionOne a callbackFunction which takes in one value and similar to the 1st variation, it accepts the value and runs its logic with it.

You can copy the above code and run it on any JS playground and you should see that the number 1 is printed twice.

Why use Callbacks?

As I said, you're using them everywhere in JS without realizing that you are but, as to why use them? It's a very simple answer.

Scoped Data Access

If you've not gone through the internals of JS this might be a little hard for me to explain but I'll give it a try.

Like most languages, you have data scopes that are maintained by the interpreter or compiler which is why you can access variables only under certain conditions.

If you go back to the above example, you can see that var a = 1 is defined inside functionOne and thus can only be used by functionOne's scope or by code that is inside functionOne but what do you do if you want that data to be accessible to other functions because if you write everything inside one function, then it beats the point of having functions and or thinking about creating modules altogether.

This is where callbacks excel and this is why JS is very async friendly.

async - asynchronous programming, I'll explain this in detail in another post.

When you write code with async programming in mind, the chances of you blocking the execution thread is very low, unless you hit a deadlock between two callbacks calling each other or you forgot to break a loop.

So if we go back to our example, we see that a is passed to functionTwo from functionOne's scope and then functionTwo just prints its. That is a very naive example and in real-life code, callbacks aren't that clean and easy to read.

If dealing with dependant data and working with data from the network, you'll probably see your code go south like this.

function dataFetch() {
  const data = someNetworkRequest();
  formatFetchedData(data, function (err, formattedData) {
    if (err) {
      console.error(err);
      return err;
    }

    processFormattedData(formattedData, function (err, processingResult) {
      if (err) {
        console.error(err);
        return err;
      }

      sendResultBackToServer(processingResult, function (err, serverResult) {
        // let's end this with a console.log
        console.log(serverResult);
      });
    });
  });
}

function formatFetchedData(param, callback) {
  //relevant code
}

function processFormattedData(param, callback) {
  //relevant code
}

function sendResultBackToServer(param, callback) {
  //relevant code
}

dataFetch();

A 3 level callback dependency can be readable but obviously, a complex app won't stop at 3 and while I could write something cleaner with an async chaining utility, a very famous one is async.js and we could use it's waterfall method to keep passing down upper dependencies to the lower functions, it's a little more manageable but still messy in larger codebases.

The Solution to the Living Hell

Enter Promises

The above-mentioned chaining is still the solution to avoiding the triangular callback code but with a little more magic handled by the wrappers.

You see, someone wrote a library called q which was the initial concept of how promises have grown to be today, this was followed by bluebird's promise polyfills which overall implement the same thenable paper.

Thenables

We are still going to have the callbacks in our life but we are going to put on a little makeup on them so we can bear them for longer sessions.

Thenables when explained simply, is a stateful container that can be chained by .then caller functions and each caller function creates another thenable. A recursive chain of wrappers to be precise.

I'll explain with the same example

//Variation One
dataFetch()
  .then(function (data) {
    return formatFetchedData(data);
  })
  .then(function (formattedData) {
    return processFormattedData(formattedData);
  })
  .then(function (processingResult) {
    return sendResultBackToServer(processingResult);
  })
  .then(function (serverResult) {
    return console.log(serverResult);
  });

//Variation two
dataFetch()
  .then((data) => formatFetchedData(data))
  .then((formattedData) => processFormattedData(formattedData))
  .then((processingResult) => sendResultBackToServer(processingResult))
  .then((serverResult) => console.log(serverResult));

//Variation Three
dataFetch()
  .then(formatFetchedData)
  .then(processFormattedData)
  .then(processingResult)
  .then(console.log);

If you understand the verbose example, you can understand how the other 2 variations work and this is obviously much neater than plain callbacks but as I said, we are still going to continue using callbacks because the language depends on it. The solution/promises are just a better way to handle it.

As visible, we still pass functions down to a wrapper/caller function that takes in the returned data and passed it to the next then in the chain because every function being called inside a .then is treated as another Promise and hence can be chained to be such.

I'll try to simplify how Promises work internally. The Promise constructor maintains a state for itself. pending | fulfilled | rejected these 3 per promise decide if the call was successful, or failed and based on it, they call a .then or a .catch.

new Promise(function (resolve, reject) {
  if (1 > 0) {
    return resolve(true);
  } else {
    return reject(false);
  }
})
  .then((value) => {
    console.log(value);
  })
  .catch((err) => {
    console.error(err);
  });

To explain this we'll consider the above example. I create a new Promise using the words new Promise and this constructor takes in a callback that is passed 2 params, resolve, and reject.

Pause.

At this point we have a Promise with the state pending because nothing has actually been a success or a failure and it'll stay that way till you, the person who's writing this promise decides.

resolve tells the constructor that the run was successful and you can execute your .then function's callback and pass it the data that it has received. In our case, true is passed to .then.

reject, on the other hand, calls the .catch with the passed value and this is where the state changes to rejected

You didn't mention the state changing to fulfilled !!

I know. Patience, human.

The fulfilled state is updated but under certain conditions, if there was only one .then call, then the main constructor is now fulfilled but if you chained it with more and more .thens then each one has it's own state and even though the first constructor might have resolved and changed to fulfilled you'll still see pending in the console because the chained ones each have their Promise instance which still has the state as pending.

In the end, we have a few callbacks and a constructor wrapped around our callbacks to make this chaining possible and code a lot more readable.

Generators

This is a huge topic so I'm going to explain it in another post sometime in the future but for now all you need to know is that generators are special kinds of functions that allow you to iterate over and over till you decide to end the function altogether. This is the actual concept that async and await works on. You can write your custom async-await implementation using a few generators and promises.

The Chosen One

Async|Await

Now even though I'm someone who likes promises more than I like async-await, mainly because I keep forgetting to add the async keyword on my functions and I'm too used to writing thenables to control my async flow, still as a programmer we gotta learn what's new. Isn't always better but if you know your options you can choose the ones that suit the condition.

As mentioned in Generators, you can create your a custom async-await wrapper if you'd like too since the interpreters will actually compile your async-await code into generators anyway.

Generators allow you to iterate over itself and used a keyword called yield which allows you to throw value out of the generator and take in another value for the next yield till you decide to end it's life with a return, much like normal functions.

function* infinite() {
  let index = 0;

  while (true) {
    yield index++;
  }
}

const generator = infinite(); // "Generator { }"

console.log(generator.next().value); // 0
console.log(generator.next().value); // 1
console.log(generator.next().value); // 2

With this in place, I can have one generator run, get an async value and yield it out of it, then pass it back to the generator and it can run the next yield scope and so on till it decides to stop. You need to understand that while yielding we are still resolving promises and async/await is nothing more than syntactic sugar for creating and resolving promises and as always, it's also based on the concept of callbacks and hence each of them creates a slight delay if compared to async functions written with just callbacks. But, developer experience and sanity need to be kept in check, else we'll have all JS developers in some Asylum yapping about callbacks all day long.

In technical terms, Javascript's concept of considering functions as first-class types improves the composition and this is something that functional programming languages generally have.

The composition works well but then you gotta limit the level of abstraction you create ( a rant for later )

Overall, a general idea of how the internals of the language dictate your control flow helps you make better choices and this can be seen by the concept of callbacks which is a spec that André Staltz came up with, which is creating a Pub/Sub model using just callbacks and using them for streams (Again, will make a post about this as well).

Make sure you don't create a callback hell inside a thenable though.

That was long....

Adios!

Have something you want me to write about or have a comment on one of my posts? You can send an email to ahoy@barelyhuman.dev