DocsDeep diveHistory

History

Some contents are borrowed from https://xcoder.in/2017/07/01/nodejs-addon-history/

The Feudal era: Using v8 C++ headers directly

In very early ages, developers were using v8/Node C++ headers directly to build Node.js native addon.

Handle<Value> Echo(const Arguments& args)
{
    HandleScope scope;
 
    if(args.Length() < 1)
    {
        ThrowException(
            Exception::TypeError(
                String::New("Wrong number of arguments.")));
        return scope.Close(Undefined());
    }
 
    return scope.Close(args[0]);
}
 
void Init(Handle<Object> exports)
{
    exports->Set(String::NewSymbol("echo"),
        FunctionTemplate::New(Echo)->GetFunction());
}

This code snippets defined a simple Node.js function: echo. It always return the first argument passed in. And it equals to this simple Node.js codes:

exports.echo = function () {
  if (arguments.length < 1) throw new Error('Wrong number of arguments.')
  return arguments[0]
}

If you publish these codes as a npm package, it can only work with node 0.10.x.

But why? The short answer is v8 and Node.js API’s change fast. For example in Node.js 6.x, the way of define JsFunction changed:

Handle<Value> Echo(const Arguments& args);    // 0.10.x
void Echo(FunctionCallbackInfo<Value>& args); // 6.x

So native packages developed in this way can only support only few versions of Node.js, when the API of v8 or Node.js changed, these packages couldn’t be compiled any more. And if maintainers updated the API to latest Node.js and v8, the package couldn’t be compiled under the older Node.js, again.

The Castle era: Native Abstractions for Node.js

Back to 2013, with the fast iteration of the Node.js and v8, packages used the old way to build native addon grow with pains. And NAN came out. It’s shorten for Native Abstractions for Node.js.

NAN was built by Rod Vagg and then Benjamin Byholm. NAN was belong to Rod Vaggs’ GitHub account from the beginning, and transferred to io.js organization in the dark age of Node.js split to io.js and Node.js；After they got back together，NAN finally transferred into Node.js organization.

After NAN came out, the develop experience in native addon packages came to the Castle age, and last to nowadays.

It’s still a litter abstract for the full description of NAN: Native abstractions for Node.js. To be specifically, it’s a bunch of C macros. You can define a JavaScript function like this for example:

NAN_METHOD(Echo)
{
}

The macro of NAN will be expanded to different CPP codes in during compiling according different Node.js version:

Handle<Value> Echo(const Arguments& args);    // 0.10.x
void Echo(FunctionCallbackInfo<Value>& args); // 6.x

NAN_METHOD will be expanded by NAN to the codes snippets below.

There are tons of macros in NAN rather than NAN_METHOD, developers can using it to do almost anything.

For example the Nan::HandleScope allow you to declare handle scope, Nan::AsyncWorker allow you to spawn task on libuv.

So in the The Castle age, here is what the c++ native addon look like:

NAN_METHOD(Echo)
{
    if(info.Length() < 1)
    {
        Nan::ThrowError("Wrong number of arguments.");
        return info.GetReturnValue().Set(Nan::Undefined());
    }
 
    info.GetReturnValue().Set(info[0]);
}
 
NAN_MODULE_INIT(InitAll)
{
    Nan::Set(
        target,
        Nan::New<String>("echo").ToLocalChecked(),
        Nan::GetFunction(Nan::New<v8::FunctionTemplate>(Echo)).ToLocalChecked());
}

The benefit of writing codes in this way is because the codes could auto upgrade with the NAN upgraded, the codes could be compatible with every versions of Node.js.

Even a good thing like the NAN has a mission, and anything outside of that mission will be gradually stripped away. Versions such as 0.10.x and 0.12.x, for example, should be retired, and the NAN will gradually drop compatibility and support for them.

Age of Empires: ABI-compliant N-API

Since the release of Node.js v8.0.0, Node.js has introduced a brand new interface for developing C++ native modules, N-API.

According to the official documentation, it is pronounced with a single N, plus API, which means that the four English letters are pronounced separately.

How does this differ from the previous three eras? Why would it be a further age of empire?

First of all, we know that even under NAN development, code written once needs to be recompiled under different versions of Node.js, otherwise Node.js won’t load a C++ extension properly if the versions don’t match. In other words, write once, compile everywhere.

N-API, as compared to NAN, black-boxes all the underlying data structures of Node.js and abstracts them into the interface of N-API.

Different versions of Node.js use the same interface, which is stably ABI-compatible, that is, the Application Binary Interface (ABI). This allows compiled C++ extensions to be used directly without recompilation, as long as the ABI version number is the same across Node.js versions. In fact, Node.js that supports the N-API interface does specify the current ABI version used by Node.js.

In order to achieve the hidden goal above, the posture of using N-API looks like this:

  • Provide the header file node_api.h.
  • Any N-API call returns a napi_status enum to indicate whether the call was successful or not.
  • The return value of N-API is occupied by napi_status, so the real return value is inherited from the incoming arguments.
  • All JavaScript datatypes are wrapped in the black box type napi_value, no longer types like v8::Object, v8::Number, and so on.
  • If the function call is unsuccessful, the napi_get_last_error_info function can be used to get information about the last error.

For more details about functions of N-API, visit its documentation, but for now, let’s take a look at something a little less abstract to give you an impression of N-API.

Module initialization

In the Feudal and NAN eras, module initialization was left to the macros supplied by Node.js.

NODE_MODULE(addon, Init)

In the current N-API, it becomes a macro of N-API.

NODE_MODULE(addon, Init)

Accordingly, this initialization function Init will be written in a different way. For example, it is written in two different ways in the feudal era and in the NAN era:

// Feudal style
void Init(Local<Object> exports) {
    NODE_SET_METHOD(exports, "echo", Echo);
}
 
// NAN style
NAN_MODULE_INIT(Init)
{
    Nan::Set(
        target,
        Nan::New<String>("echo").ToLocalChecked(),
        Nan::GetFunction(Nan::New<v8::FunctionTemplate>(Echo)).ToLocalChecked());
}

The Init function should look like this when it comes to N-API:

void Init(napi_env env, napi_value exports, napi_value module, void* priv)
{
    napi_status status;
 
    // Description constructs for setting exports
    napi_property_descriptor desc =
        { "echo", 0, Echo, 0, 0, 0, napi_default, 0 };
 
    // set "echo" into `module.exports`
    status = napi_define_properties(env, exports, 1, &desc);
}

napi_property_descriptor is a description structure for setting object properties, which is declared as follows:

typedef struct {
  const char* utf8name;
  napi_value name;
 
  napi_callback method;
  napi_callback getter;
  napi_callback setter;
  napi_value value;
 
  napi_property_attributes attributes;
  void* data;
} napi_property_descriptor;

So the desc in the Init function above means that something called “echo” is set under the object to be installed, the function is Echo, all the other getters, setters, and so on are empty pointers, and the property is napi_default.

Declare Functions

Do you remember the two previous function declarations? Move over for the third time:

Handle<Value> Echo(const Arguments& args);    // 0.10.x
void Echo(FunctionCallbackInfo<Value>& args); // 6.x

In N-API, you no longer need to have a C++ background, C is sufficient. Because in N-API, declaring an Echo looks like this:

napi_value Echo(napi_env env, napi_callback_info info)
{
    napi_status status;
 
    size_t argc = 1;
    napi_value argv[1];
    status = napi_get_cb_info(env, info, &argc, argv, 0, 0);
    if(status != napi_ok || argc < 1)
    {
        napi_throw_type_error(env, "Wrong number of arguments");
        return 0; // `napi_value` is actually a pointer, returning a null pointer means no return value.
    }
 
    return argv[0];
}

Step-by-step analysis of the above code:

  • napi_get_cb_info Gets information about the parameters of the current function request, including the number of parameters and their bodies (which are represented as an array of napi_value).
  • To see if there is an error in the call (status is not equal to napi_ok) or if the number of parameters is less than 1.
    • If there is an error in the call or the number of arguments is less than 1, an error object is thrown at the JavaScript level via napi_throw_type_error and returned.
    • Proceed if there are no errors.
  • Returns argv[0], the first argument

Conclusion

This session explains the change in approach to native C++ module development in the Node.js:

  • From node-waf to node-gyp, it’s a change in build tools, maybe GN or something else in the future.
  • From code-breaking to the advent of NAN, the Node.js community has seen its fair share of loves and hates, all the way to the new kid on the block, N-API, which has brought new blood into the development of native C++ modules.

I hope this helps you understand the sour history of Node.js native module development, and the reasons and background for the emergence of N-API.

Native moduleRelease native packages