JavaScript Buffers Basics

Talking from the perspective of JavaScript:

And absolutely nothing more than this.

A buffer can be simply though of as a shopping cart. On itself, the cart is only capable to hold onto items, nothing more than that.

A human or, for that matter, a robot is needed in order to use it for any fruitful purpose. They are the ones who put stuff in the cart, or take stuff out of it, or change the order of items and much more than that.

A buffer is exactly this - it is nothing more than a place where stuff can be stored.

A buffer stores raw binary data i.e a sequence of 0s and 1s.

The simplest unit of a buffer is a byte. A buffer is simple a sequence of bytes.

To put binary data in a buffer and then get that out of it, we ought to use something called a view. We'll discover this later in this chapter.

For now let's explore the core interface that introduces buffers into JavaScript.

So you want to use buffers in JavaScript to store binary data? Well for that you ought to be familiar with the ArrayBuffer interface.

The ArrayBuffer interface is the engine that powers the idea of easy-to-create buffers in JavaScript.

It's extremely simple to use - even for beginner developers.

To create a new buffer, we call the ArrayBuffer() constructor along with a size argument, as shown below:

var buffer = new ArrayBuffer(size);

The size argument is where you specify how much space you want for the buffer, in units of bytes. The default value is 0.

For example, to create a buffer of 4 bytes, we would be setting size to 4. Similarly, to create a 1 kb buffer, we would be setting size to 1000, and so on and so forth.

One thing worth noting here is that ArrayBuffer() alots exactly the amount of space given in size, to the newly created buffer; it can't be changed later on.

That is, if size is 4, the buffer created is exactly 4 bytes long - not lesser, not greater than that!

This means that calling ArrayBuffer(0), which is the default value of size, would createn empty buffer - not having even a small amount of space to store a bit!

Hence, whenever creating a buffer, do consider your byte size limits!

Why is it called 'ArrayBuffer'?

Anyways, a common question developers - or better to say, enthusiastic developers - ask is that why is ArrayBuffer called 'ArrayBuffer'.

What does the word 'Array' have to do here?

Well this a good discussion to have!

Recall the structure of an array. It is a whole block of data where individual elements are stored at contiguous positions known as indexes.

Quite similar is the case with an ArrayBuffer.

It is a block of memory, where individual bytes are stored at contiguous positions, in this case, known as byte offsets.

A buffer in JavaScript can also be called an array of bytes, or simple, a byte array.

So instead of calling it Buffer, the devs of API decided to give it a more specific name - a name which could emphasize on the fact that it's an array of bytes!

Anyways, on its own ArrayBuffer is useless. Recall that it's just a cart - it needs to be paired with something called a view.

Let's discuss on it!

As we have seen above, creating a buffer by calling the ArrayBuffer() constructor isn't any difficult at all.

Just decide on a byte size, pass it to the ArrayBuffer() constructor as a number argument, and you're done.

However, only this won't yield us anything fruitful!

Recall that a buffer is just a storage location on itself i.e it can't store (or retrieve) anything. To do so we need to use a view.

But why do we need such a storage mechanism?

Say you want to store the number 200 in 16 bits (2 bytes).

First the number needs to be converted into binary representation, then the remaining bits need to be filled with 0s until all the 16 bits are filled.

Then based on the endianess, the bytes need to be re-ordered accordingly and finally the number has to be stored in the buffer.

Also consider any occasions where input numbers are greater than their byte sizes, for example storing 3000 in a single byte. In such cases, modular arithmetic needs to be done on the numbers.

Apart from this, when reading a given block of bytes in a buffer, the bytes first have to be re-ordered based on the endianess, then the binary data has to be transformed into a decimal number based on the format of data, such as an unsigned integer, a floating point number.

All this rightly justifies the fact that a mechanism is indeed required to read data from and write data into a buffer.

Talking about the name, 'view' comes from the fact that the feature allows us to literally view data sitting inside a buffer.

Otherwise the data would merely be a piece of gibberish!

Alright, now that we are well-versed with what views are, let's explore the DataView interface.

The most basic view interface JavaScript provides at the dispense of developers, to interact with buffers, is DataView.

A DataView object is made to work with a given ArrayBuffer object.

To create a DataView object, simply call the DataView() constructor, along with passing it the buffer object whom you want it to operate on.

This can be seen below:

var view = new DataView(buffer);

The parameter buffer is an ArrayBuffer object which needs to be viewed.

Let's create a view for an array buffer of 2 bytes:

var buffer = new ArrayBuffer(2),
     view = new DataView(buffer);

With a view set up, now we just need to set or get the buffer's underlying data. And to do so, we use any one of the methods defined on the DataView interface.

Following are all the get methods, that serve to get data out of a buffer:

1. getUint8() - get a byte as an unsigned 8-bit integer.
2. getUint16() - get two bytes as an unsigned 16-bit integer.
3. getUint32() - get four bytes as an unsigned 32-bit integer.
4. getBigUint64() - get eight bytes as an unsigned 64-bit integer.
5. getInt8() - get a byte as a signed 8-bit integer.
6. getInt16() - get two bytes as a signed 16-bit integer.
7. getInt32() - get four bytes as a signed 32-bit integer.
8. getBigInt64() - get eight bytes as a signed 64-bit integer.
9. getFloat32() - get four bytes as a single-precision floating-point number.
10. getFloat64() - get eight bytes as a double-precision floating-point number.

Similarly, following are all the set methods, that serve to put data into a buffer:

1. setUint8() - set a byte as an unsigned 8-bit integer.
2. setUint16() - set two bytes as an unsigned 16-bit integer.
3. setUint32() - set four bytes as an unsigned 32-bit integer.
4. setBigUint64() - set eight bytes as an unsigned 64-bit integer.
5. setInt8() - set a byte as a signed 8-bit integer.
6. setInt16() - set two bytes as a signed 16-bit integer.
7. setInt32() - set four bytes as a signed 32-bit integer.
8. setBigInt64() - set eight bytes as a signed 64-bit integer.
9. setFloat32() - set four bytes as a single-precision floating-point number.
10. setFloat64() - set eight bytes as a double-precision floating-point number.

In the next chapter, we shall understand all these methods in detail.

For now, we'll just work with the first method in both these lists - getUint8() and setUint8().

Say we want to store the numbers 200 and 30 inside a buffer, each taking up a byte.

To put the numbers in the buffer we'll use the method setUint8(). It processes the given value, and then places it inside a single byte, as an unsigned 8-bit integer.

Like all set methods, it has the following general form:

DataViewObject.setUint8(byteOffset, value, littleEndian);

The first byteOffset argument specifies the byte offset, or in simple words the position where to put the given value.

The second value argument takes the data to be put into the given byte offset.

With this information in mind, now we can finally put the numbers 200 and 30 inside a buffer.

var buffer = new ArrayBuffer(2);
var view = new DataView(buffer);

// put data into the buffer
view.setUint8(0, 200);
view.setUint8(1, 30);

In line 4, we put the number 200 in the first byte of buffer and in line 5, we put the number 30 in the second byte of buffer.

Now let's retrieve these values using the corresponding get method for setUint8() - getUint8().

The method getUint8() retrieves a given byte's binary data and processes it into an unsigned 8-bit integer.

DataViewObject.getUint8(byteOffset, value, littleEndian);

This time, byteOffset specifies the position where to start the retrieval of data.

Now, coming back to our example, to retrieve the numbers 200 and 30, we have the code shown below:

// get data out of the buffer
console.log(view.getUint8(0));
console.log(view.getUint8(1));

In line 2, we retrieve the first byte of buffer which returns 200; and then in line 3, retrieve the second byte of buffer which returns 30.

And this is the basic illustration of how we interact with an ArrayBuffer object using the DataView interface.

In the next chapter, as we've said before, we shall explore the DataView interface from crust to core. We'll see the details of all its get and set methods, and then look over a simplification done to this view model i.e typed arrays.

Finally, we shall understand what's the purpose of the last littleEndian argument of all the DataView's get and set methods, when we unveil the concept of endianness, in the last chapter.