JavaScript Iterators

On the basic level, an iterator is a JavaScript object that provides utilities to iterator over a given sequence.

On the technical level:

Now what is the iterator protocol?

The iterator protocol is simply a set of rules which must be obeyed by an object in order for it to be called an iterator.

The protocol states that the object must have a next() method. This method shall return an object with the following two properties:

1. done - a Boolean value indicating whether the iterator has reached its last value.
2. value - holds the next value in a given sequence.

Now, before we give a live example of an iterator, first try to make some intuition of these property and method names.

Suppose you have an iterator object iter. Calling next() on it shall give the next value in the sequence that it operates on - on the initial call this will be the first value in the sequence.

Then calling iter.next() the second time shall again return the next value in the sequence - on the second call this will be the second value in the sequence. And so on and so forth...

This goes on uptil the point the sequence doesn't contain anymore values.

The way both these ideas are represented by the iterator is as follows:

If done is true this means that we are done - no need to iterate any further! Similarly, if done is false, this means that we aren't done - we still need to go further!

With this disucssion in mind, let's consider a simple iterator and understand more closely that how it should work.

Suppose that we have an array nums and the task of iterating over all of its elements and logging them.

var nums = [1, 5, 16];

Now imagine that we also have an iterator object iter to iterate over this array.

The way the iterator will work is demonstrated as follows:

Calling iter.next() for the first time will return an object wrapping the first value in nums.

iter.next(); // {value: 1, done: false}

The object's value property will be equal to 1 and the done property equal to false, as we aren't done yet.

Calling iter.next() the second time will return an object wrapping up the second value in nums.

iter.next(); // {value: 5, done: false}

value will be equal to 5 and done equal to false, as we aren't done even yet!

Calling iter.next() the third time will return an object wrapping the third (and last) value in nums.

iter.next(); // {value: 16, done: false}

value will be equal to 16 and done will be equal to false.

Now you might be thinking that why isn't done equal to true here.

The thing is that we still need to go further in the sequence nums and check if something remains - if we get nothing further, only then we know we are done.

In short, we need to go one step further to confirm the end of nums.

Likewise, calling iter.next() the fourth time returns an object wrapping the fourth value in nums. Since there isn't any fourth value in nums, the value property is undefined and done is true.

iter.next(); // {value: undefined, done: true}

This signifies that iter has reached the end of nums and hence there is no need to iterate further.

Regardless, subsequent calls of iter.next() don't return null or an empty object; but rather they return the same object as before with value equal to undefined and done equal to true.

iter.next(); // {value: undefined, done: true}
iter.next(); // {value: undefined, done: true}
iter.next(); // {value: undefined, done: true}

This is how a simple iterator works.

Now, it's time for you to implement all this for real by creating a function makeIterator() that takes in an array and allows iteration over it just as highlighted above.

var nums = [1, 5, 16];
var iter = makeIterator(nums);

iter.next(); // {value: 1, done: false}
iter.next(); // {value: 5, done: false}
iter.next(); // {value: 16, done: false}
iter.next(); // {value: undefined, done: true}

function makeIterator(arr) {
   var i = 0;
   return {
      next: function() {
         if (i > arr.length - 1)
            return {value: undefined, done: true}
         return {value: arr[i++], done: false}
      }
   }
}

If you solved the task above, then you would've surely realised that it isn't easy to construct an algorithm that spawns an iterator.

We need sophisticated skills to do so and understand concepts like lexical scoping, closure variables to be able to fluidly construct the algorithm.

But the good news is that we don't need to create iterators for many data classes in JavaScript - they have their own defined mechanism to create iterator objects and likewise use the iterators to iterate over their instances.

This mechanism is the @@iterator() method.

Let's put up a discussion....

Many classes like Array, String, ArrayBufferView, HTMLCollection etc define a special method on their respective prototype objects that can be invoked to retrieve an iterator for those class's instances.

This method is the @@iterator() method.

As you know from the Symbols chapter, @@ means that we are talking about a well-known symbol - a property defined on the Symbol object.

In this case, @@iterator means we are refering to Symbol.iterator.

In other words, we can also call this method as the [Symbol.iterator]() method.

Anyways, the @@iterator() method defined on a prototype such as Array.prototype returns an iterator object for its respective instance (an Array object in this case).

This iterator object can be used exactly how we used iterators in the section above.

In fact, all iterators work the same way no matter what - if they are iterators it indirectly means they follow the iterator protocol and likewise would operate synonymous to each other.

Following we demonstrate an example using our old nums array:

var nums = [1, 5, 16];

var iter = nums[Symbol.iterator]();

iter.next(); // {value: 1, done: false}
iter.next(); // {value: 5, done: false}
iter.next(); // {value: 16, done: false}
iter.next(); // {value: undefined, done: true}

The most important line here is the second line where we get an iterator object out of the array's predefined @@iterator() method.

Let's understand what's happening here.

A variable iter is declared and assigned an expression. This expression is nums[Symbol.iterator](). The way it evaluates is described as follows:

1. The Symbol.iterator returns a symbol which is then first searched on the nums object (remember that an array is also an object and therefore can also have properties).
2. Since there is no match found for the symbol on nums, searching shifts to the prototype of nums i.e the Array.prototype object.
3. Here, indeed a symbol property - Symbol.iterator - is found and it holds a functional value. In other words, it's a method defined on the prototype.
4. Finally, this method is invoked, which results in an iterator object being returned. This iterator gets saved in iter and so the execution of line 2 completes.

With the iterator in hand we can then use it in any way we like to iterate over nums.

It's good to know that many classes in JavaScript have predefined iterator-spawning methods defined on them, by virtue of a property whose key is the value of Symbol.iterator (hence the name @@iterator).

But it would be even better if we could somehow understand the deal behind this symbolic property.

Why do all iterables, such as arrays and strings, have their iterators hidden behind this method? What's the truth of the @@iterator() method?

Well it's a short story...

This is a convention set by JavaScript, or to be more precise, by ECMAScript 6.

You need to iterate over a sequence - you need to call the @@iterator() method on it and use the returned iterator for your job.

Such simple!

Let's consider an example.

In the previous chapter we saw how the for...of loop works when given a sequence to iterate over. Now we shall understand precisely how it does that. Yeah, precisely!

Consider the code below:

var nums = [1, 5, 16];

for (var num of nums) {
   console.log(num);
}

We all know what would this output.

For the real deal, consider the following code which shows what's happening behind-the-scenes when we execute the for...of loop above:

var nums = [1, 5, 16];

for (
   // initialisation
   var iter = nums[Symbol.iterator](),
   next = iter.next(),
   num = next.value;

   // iterative check
   !next.done;

   // iterative expressions
   next = iter.next(),
   num = next.value
)

{
   console.log(num);
}

In the background the for...of translates to a simple for loop.

First this loop calls the @@iterator method on nums (in line 5) and then uses the returned iterator in iterating over it (the iterator is saved in iter).

After this, it calls the next() method on iter (in line 6) and saves its value in a similarly-named variable next. next will be used in retrieving the value and done properties.

Since num (be careful, we're not referring to nums) corresponds to each value in nums it is assigned the value property of next.

Furthermore, the iterative check statement (in line 7) simply means to keep the loop iterating if we aren't done i.e if done is false.

Finally, the last two statements are just given to reset the respective values (next and num) for the next iteration.

And so, this is what's going on when a for...of loop is encountered.

Let's consider another example, this time using a typed array.

Below shown in an example drawn from Task 2 in the previous chapter. We've got a Uint8Array with 3 byte values - 15, 192, 16, which we log using the @@iterator() method defined in the prototypal chain of the typed array.

var uint8 = new Uint8Array([15, 192, 16]);

for (var byte of uint8) {
   console.log(byte.toString(16));
}

In the background, this code translates to the following:

var uint8 = new Uint8Array([15, 192, 16]);

for (
   // initialisation
   var iter = uint8[Symbol.iterator](),
   next = iter.next(),
   byte = next.value;

   // iterative check
   !next.done;

   // iterative expressions
   next = iter.next(),
   byte = next.value
)

{
   console.log(byte);
}

Once again, first the iterator for uint8 is retrieved by calling the @@iterator() method on it; and then this iterator is used to retrieve the first value in uint8, to start with.

Afterwards we proceed just as before - continuing iterating if we aren't done and resetting the variables next and byte to point to the next value in the sequence uint8.

Isn't this pretty simple?

If you think so, go on and give a go to the task below.

var nums = [1, 5, 16];

for (var num of nums) {
   console.log(num);
}

var nums = [1, 5, 16];

// initialisation
var iter = nums[Symbol.iterator](),
    next = iter.next(),
    num = next.value;

while ( !next.done ) {
   // body of for...of
   console.log(num);

   // reset variables
   next = iter.next();
   num = next.value
}

One thing to clarify before ending this section is that due to the fact that for...of translates to a loop that makes numerous next() calls, it is definitely slower as compared to a normal for loop.

However, this performance degradation isn't noticeable - in fact, not even a degradation - for short sequences like the ones we saw above. Even sequences with over hundreds, or maybe even thousands, of items won't cause a glitch.

It's only a concern for sequences that have over a million of values; where each fraction of a second matters!

Iterator objects are verily one powerful idea born with the advent of ES6 that allow for easy and elegant iteration over given sequences.

However, the whole idea is still incomplete. In the next chapter on Iterables we complete the remaining bits and pieces of this chapter, by defining our own @@iterator() methods and using them to make given objects work with for...of.

And after that we'll explore another strongly tied concept to iterators, that is, generator functions. You would surely fall in love with generators - this is how lovely they are!

In short, the iteration run isn't over - but yes you've completed one-third of it!