Node.js CommonJS Modules

The history of modules in JavaScript is quite a fascinating tale of its own, similar to the history of promises, taking twists and turns through the path of time and then eventually landing up as a feature in the core ECMAScript standard.

Back when Node.js was introduced in 2009 by Ryan Dahl, there was no standard module system in JavaScript. The server side, however, was obviously such a complex world to live in that there had to be one system designed for it — developers might've been OK not having modules on the browser but by no way was this going to work on the server end.

Consequently, initiated by Kevin Dangoor from Mozilla, a group was formed, dubbed as ServerJS (and then later on as CommonJS), to oversee the drafting of a specification for building an ecosystem of modules in JavaScript on the server side, to better compete with server-side languages such as Python, Java, and Ruby.

Kevin Dangoor even discussed some of the insights into the ideas that led to this development in a small blog article.

Anyways, Node.js eventually came in and brought with it a new module system in JavaScript, for exclusive use on the server side, referred to as CommonJS modules.

While today the recommended approach is to use ECMAScript modules because of the very fact that ECMAScript has formulated its own module system, it's still crucially important to be aware of CommonJS modules in Node.js,because currently it's the default module system of the Node runtime.

In this chapter, we shall discover a whole ton full of detail relating to CommonJS modules in Node.js.

In particular, we'll understand the CommonJS module specification; how its implementation works in Node.js; the require(), module, and exports identifiers available in a module file; the special __dirname and __filename variables; caching of modules; and a lot more.

As specified at the start of this chapter, CommonJS modules refer to one of the earliest module specifications in JavaScript, created solely for the server side.

Most importantly, CommonJS modules are NOT a part of ECMAScript (core JavaScript), so to speak; they represent a separate specification meant to define a module system in JavaScript outside of the browser.

It's different from the module system that is now defined in ECMAScript, which is referred to as ECMAScript modules, or ESM for short. (We'll discover ECMAScript modules in Node.js in the next chapter.)

The ideas put forth by the CommonJS modules specification for implementing a module system in JavaScript are really simple:

• Every single JavaScript file is a CommonJS module.
• To import another module within a module, we use a function require().
• To export given stuff from a module, we use module.exports or exports (the difference between them will be made clear below).
• module is an object containing some further details regarding the underlying module it's used in.

In addition to these, CommonJS modules also expose two special variables within them:

• __dirname holds the path of the directory of the underlying module file.
• __filename holds the path of the underlying module file.

These identifiers are available within a module file without us having to do anything extra.

Perhaps one of the paramount things to note regarding CommonJS modules is that they are synchronous in nature.

That is, when a require() call is made to import another module, the call doesn't exit until and unless the underlying module file is read from the file system, parsed by the JavaScript engine, and finally executed.

If that module file itself imports other modules (via further require() calls in it), then those modules have to be executed first before the module itself can complete executing.

On the server end, this module loading takes the matter of milliseconds, if not faster, as the module file typically resides on the same file system where the Node.js runtime is running, and loading it simply means finding and then executing it.

However, this idea couldn't be possibly taken to the browser, where loading a module file would mean to download a JavaScript file over the network, synchronously.

Let's understand these features of CommonJS modules in more detail with the help of some examples.

Before we can import stuff from any module, it's imperative to know about how to export it in the first place. After all, we need something to be exported prior to importing it, right?

Now, to export data from a CommonJS module, we have two approaches:

• Assign a value to module.exports
• Set a property on the exports object

Let's take a look over both these.

Assign a value to module.exports

The module object available in a CommonJS module contains information related to the module, for e.g. its set of exports, its name, its id, and so on.

The thing that interests us is its set of exports, stored in its exports property.

Exports in CommonJS are carried out in the form of an object and its properties.

To begin with, module.exports points to an empty object in a module.

This is illustrated as follows:

console.log(module.exports);

We're logging module.exports right at the start of our script. The console snippet below showcases the obtained output:

As can be seen, the returned object is an empty object (based on the Object interface).

If we wish to export one single value from a module, we should use module.exports.

A given CommonJS module file exposes its entire module.exports object to any module that imports it. As we shall soon find out, a require() call to a given module returns back the value assigned to module.exports in that imported module.

For example, let's say we wish to export the following object from math-constants.js, holding two constants, PI and E, corresponding to the values of Math.PI and Math.E:

const constants = {
   PI: Math.PI,
   E: Math.E
};

We'll assign this object to module.exports, as follows:

const constants = {
   PI: Math.PI,
   E: Math.E
};

module.exports = constants;

Assigning a value to module.exports in a CommonJS module could be assimilated to a default export in an ECMAScript module (although they both aren't the same thing).

Anyways, when we want to export multiple values individually from a module, we need to individually assign properties to module.exports or simply the exports object. This is discussed up next.

Set a property on the exports object

The exports identifier available in a CommonJS module is simply an alias of module.exports.

That is, both exports and module.exports point to the exact same object internally — mutating either of these causes the same underlying object to change.

The reason of aliasing module.exports as exports is to simplify accessing module.exports over and over again in a program when making multiple individual exports. Clearly, typing exports is much quicker than typing module.exports.

In the following code, we try to replicate the math-constants.js file above, albeit by assigning the constants individually using exports:

exports.PI = Math.PI;
exports.E = Math.E;

Let's inspect the value of exports and module.exports in this code:

exports.PI = Math.PI;
exports.E = Math.E;

console.log(module.exports);
console.log(exports);

{ PI: 3.141592653589793, E: 2.718281828459045 }
{ PI: 3.141592653589793, E: 2.718281828459045 }

Expectedly, both module.exports and exports holds the same object.

We can further confirm this using the === operator:

exports.PI = Math.PI;
exports.E = Math.E;

console.log(module.exports === exports);

true

Remember that module.exports and exports both point to an empty object to begin with; adding properties to either module.exports or exports is essentially the exact same thing.

However, obviously assigning a value to module.exports is NOT the same thing as assigning that value to exports. This is elaborated up next.

Assigning to exports doesn't have any effect!

Although a JavaScript language detail you'd probably know, it's worthwhile emphasizing on the fact that assigning a value to exports as a whole won't export the given value as intended. In fact, it won't export anything!

As an example, the code below doesn't export constants; instead it just exports an empty object:

const constants = {
   PI: Math.PI,
   E: Math.E
};

// This won't work as expected!
exports = constants;

The reason is very simple: exports is a module-scoped variable, and assigning a value to a variable in JavaScript does NOT have any reactive side effects on anything else in the program — it's just the variable that gets changed.

So, when we assign a value to exports, nothing happens because module.exports is still an empty object (or whatever it's set to by the underlying module).

To reiterate on this idea, because it's quite important to keep in mind, we have the following illustration:

Demonstrating module.exports and exports in a CommonJS module

This shows the configuration of module.exports and exports right at the beginning of executing a CommonJS module.

Right after we assign a value to exports, as follows:

exports = { x: 10 }

here's how the configuration then looks:

Demonstrating module.exports and exports in a CommonJS module

exports points to a new object while module.exports remains with its original value (rightly so, because we haven't assigned anything to it).

When the module is done executing, the value of module.exports is what's returned to whichever context called the module, and in this case this is the value {}.

So now that we know how to export data from a module, let's see how to import it in another one.

To import data in a CommonJS module from another module, we use the require() function.

The path to the module that we wish to import is specified to the require() function.

Now, there are three different kinds of modules that we can import:

• Core modules, also known as native modules
• Node modules, defined by a third-party and available for us to download via npm (more on npm in the chapter Node.js Packages)
• Local modules that we define ourselves.

For core modules, the string provided to require() is an absolute specifier. Some examples include 'fs', 'events', 'streams', etc. To make this distinction even clearer in code, we can precede the specifier with the text 'node:'. So instead of 'fs', we'd have 'node:fs'.

For Node modules, again, the string provided to require() is an absolute specifier but doesn't resemble the name of any core module. Some examples include 'express' (the third-party Express framework), 'morgan' (the third-party logging library), 'mysql' (Node.js MySQL driver), etc.

Finally, for local modules, the string provided to require() is a relative file path, beginning either with . (current directory) or with .. (parent directory). This path is resolved relative to the module that has the require() call.

For example, if we have the following directory structure,

and we are inside grandchild.js, the path to grandchild2.js would be ./grandchild2.js. Similarly, the path to child.js would be ../child.js. And that of parent.js would be ../../parent.js.

Simple?

For now, we'll only concern ourselves with local modules.

However, in the coming chapters, we'll be going to great lengths in using core modules in Node.js for performing various tasks, such as reading files, setting up streams, handling events, working with clusters, and much more. We'll also explore a bunch of third-party packages and consequently, with them, Node modules.

Anyways, let's now see how to import the data exported from the math-constants.js file that we created in the last section above.

Imagine we have a file main.js in the same directory as math-constants.js. The following code inside main.js imports this math-constants.js file and logs the two obtained constants.

const constants = require('./math-constants');

console.log(constants);
console.log(constants.PI);
console.log(constants.E);

{ PI: 3.141592653589793, E: 2.718281828459045 }
3.141592653589793
2.718281828459045

There are a couple of things to take note of here...

Firstly, since the file to be imported, math-constants.js, resides in the same directory as our main.js file, we use ./ in the path provided to require().

Secondly, the object returned by the require() call is simply the same object literal that we exported from math-constants.js.

Now because the value returned by require() is basically just an object in this case, we can skip creating constants in main.js and rather directly obtain PI and E from the imported module by leveraging the object destructuring assignment syntax in JavaScript.

Something as follows:

const { PI, E } = require('./math-constants');

console.log(PI);
console.log(E);

3.141592653589793
2.718281828459045

Let's consider another example, this time slightly more involved.

Suppose we have three modules, a.js, b.js, and main.js. main.js is the entry point to our program (i.e. it's provided to the node command in the terminal).

a.js has an export named a while b.js has an export named b. In main.js, our goal is to import both these modules and then log their respective exported values, a and b.

Here's the code of a.js:

exports.a = 'This is a';

And here's that of b.js:

exports.b = 'This is b';

Now, it's time to see the code of main.js:

const { a } = require('./a');
const { b } = require('./b');

console.log(a);
console.log(b);

The a.js file is imported and its export named a is extracted into main.js. Similarly, b.js is imported after this and its export named b is extracted. Finally, both these values are logged to the console to give the following:

So what do you say? Simple, isn't this?

Let's say we have a file main.js importing two modules, a.js and b.js, and logging the value of a exported by a.js:

const { a } = require('./a');
require('./b');

console.log(`a: ${a}, imported into main.js`);

The second call to require() is not behind any assignment statement here simply because we don't wish to extract anything from b.js (in fact, b.js doesn't export anything).

The module b.js itself imports a.js in it and logs its exported value a too:

const { a } = require('./a');

console.log(`a: ${a}, imported into b.js`);

Following is the definition of a.js:

console.log('a.js executing...');
exports.a = 10;

Now, a valid question that comes to mind is that would Node repeat the processing of a.js when it's imported inside b.js or not?

In other words, given that a.js has already been processed for an import in the main.js file, would Node redo it in b.js? Or would Node remember that it has processed a.js before and thus skip the processing, using the already processed result?

What do you think? Give it a guess.

Well, it's the latter — Node.js would know that it has processed a.js before and thus won't do so again; it'd just use the already processed result.

The console snippet below showcases the output of this program:

The single most paramount detail to note here is that the text 'a.js executing...' is output exactly once.

This confirms that a.js is executed exactly once even though it's imported twice, for the second import (in b.js) is resolved from the result of the previous one (in main.js).

Based on this behavior, we say that CommonJS caches modules.

As we can probably guess, this is done for two reasons:

• Performance — preventing reprocessing an already processed module definitely saves computational resources and speeds up module imports.
• Consistency — reusing the same result of an imported module means that its data could be used with predictability and consistency.

The second point here is worth explaining in more detail...

Suppose we have a module counter.js exporting a count variable, as shown below:

exports.count = 0;

This module is called upon in a script called work.js and its count updated in there. Here's the definition of work.js:

const counter = require('./counter');

counter.count++;

There is no destructuring assignment syntax here since we need to obtain a reference to the object exported by counter.js to be able to correctly mutate its count; directly updating count won't obviously work as expected.

Finally, we have a third module called log.js that simply logs the value of this count exported by counter.js:

var { count } = require('./count');

console.log(count);

Integrating all three of these modules is the main.js script below:

require('/work');
require('./log');

By virtue of the fact that CommonJS modules are cached, when we import counter.js in log.js after having imported it before in work.js, we get back the same object that's returned in work.js.

The effect of this is that updating the count from within work.js is visible in log.js since both the objects are essentially the same:

Had modules not been cached, importing counter.js in log.js would've re-invoked the counter.js module and returned a completely new object, with its count set to 0.

As you'd agree, this would've completely thrown away the highly desirable trait of consistency of importing the same module in two different locations and mutating its data with predictability.

If you're curious about how the aforementioned five special variables are available inside a CommonJS module or, in general, how CommonJS modules work, then this section is for you.

As stated above, everything inside a CommonJS module is scoped to that very module. Alright. Add to this the fact that the module gets access to five special variables from the runtime environment: require, module, exports, __dirname, and __filename.

How might Node.js possibly provide these five variables to a module?

Well, an ingenious thing that Node.js does here is to try to reuse existing features of JavaScript in order to provide these five variables to a CommonJS module. That ingenious thing is to use a function to define a module.

What? A function? Yes. For defining a module? Absolutely. How?

Let's find it out...

Suppose we have the following JavaScript file, a CommonJS module:

console.log('Hello World!');

The way Node.js makes this file be executed as a CommonJS module is as follows:

First, the file's code is wrapped up in an anonymous function definition with five parameters in the given order: exports, require, module, __filename, __dirname. This gives a string representing a JavaScript function definition, something as follows:

function(exports, require, module, __filename, __dirname) {
   console.log('Hello World');
}

Next, this string is fed into the V8 virtual machine to be parsed and converted into a function object to be called.

Finally, this function is called, supplying it a total of five values during invocation, in the given order: the exports property of a module object, a function to import modules, the module object (previously mentioned), the path of the underlying file, and the path of the underlying directory.

Can we confirm this somehow? A big yes.

Since a CommonJS module is merely a function, we can access the arguments object (available in JavaScript functions) to get direct access to all these five arguments:

console.log(arguments);

[Arguments] {
  '0': {},
  '1': [Function: require] {
    ...
  },
  '2': Module {
    ...
  },
  '3': ...,
  '4': ...
}

And now the intuition behind where these five variables come from in a CommonJS module is crystal-clear. A module is actually executed as a JavaScript function with five arguments at run time.

Without having to come up with any new logic for executing modules, Node.js quite smartly implements CommonJS modules using functions and some, good old parsing power from V8.

Isn't that remarkable?