Updated 26/9: Updated the recommended approach for maintaining backward compatibility

To handle messages, a node registers a listener for the input event. Whenever that event is triggered, the registered callback function is called with the message:

this.on('input', function(msg) {

})

Within this function the node can do whatever work it needs to do and at some point it might call node.send() in order to send on a message. I say ‘at some point’ because the runtime doesn’t know what the node plans to do. There are a number of possibilities, for example:

• It might be an entirely synchronous function and call node.send() before returning.
• It might perform some asynchronous action and call node.send() in the future after the event handler function has returned
• It might call node.send() multiple times
• It might not send anything

This makes it impossible for the runtime to know with any certainty when the node has finished handling a message. It also means, when a node sends a message, it isn’t possible to know why it sent it.

Why does it matter?

Node-RED has coped perfectly well without the runtime knowing these things so far. However, as we look beyond version 1.0, there are a number of new features that will need this extra information.

For example, I mentioned in an earlier blog post that we are looking at how the runtime can better monitor messages passing through a flow and provide a standard way to timeout any node that takes too long.

Another feature being looked at is being able to more gracefully shutdown flows - so that a flow can stop accepting new work, but allow in flight messages to finish passing through the flow.

These things will only be possible if the runtime knows when the node has finished with a message.

What’s changing?

To solve this problem, we’re introducing a new signature for the input event callback function as part of the 1.0 release.

this.on('input', function(msg, send, done) {

})

In addition to receiving the message object, the function is also given its own send and done functions to use for that message.

The send function is a drop-in replacement for node.send(). The key difference is that when its called, the runtime will be able to correlate it back to the original msg object that was received.

The done function must be called when it has finished handling the message. It takes one optional argument, an error object if the node has failed to handle the message for some reason.

For example:

this.on('input', function(msg, send, done) {
    // do some work with msg
    someImaginaryLibrary(msg, (err, result) => {
        if (err) {
            // Report back the error. This is equivalent to
            //    node.error(err,msg)
            // but with the timeout handling dealt with as well
            done(err);
        } else {
            msg.payload = result;
            send(msg);
            done();
        }
    })
})

Adding the Complete node

One of the uses for this new API is to enable flows to be triggered when a node with no output has completed - such as the Email node. This is where the new ‘Complete’ node comes in.

If a node calls done(), it will trigger any ‘Complete’ nodes in the workspace that have been configured to target that node. If done is called with an error, then it will trigger any ‘Catch’ nodes, as with existing calls to node.error(err,msg).

Backwards compatibility

As I write, there are over 2200 node modules in the flow library. Clearly they are not going to update to this new callback signature overnight. But that’s no problem as the runtime is able to detect if a node is registering an ‘old’-style or ‘new’-style callback function and handle it accordingly.

Our hope is that node authors will migrate their nodes over time to this new format.

We also realise that not everyone is going to upgrade to Node-RED 1.0 straight away. So what happens if a node uses the new callback signature, but it gets installed in a pre-1.0 version of Node-RED?

For that to work, the node can be written defensively to work in either case:

let node = this;
this.on('input', function(msg, send, done) {
    // If this is pre-1.0, 'send' will be undefined, so fallback to node.send
    send = send || function() { node.send.apply(node,arguments) }
    // do some work with msg
    someImaginaryLibrary(msg, (err, result) => {
        if (err) {
            // Report back the error
            if (done) {
                 // Use done if defined (1.0+)
                done(err)
            } else {
                // Fallback to node.error (pre-1.0)
                node.error(err, msg);
            }
        } else {
            msg.payload = result;
            send(msg);
            // Check done exists (1.0+)
            if (done) {
                done();
            }
        }
    })
})

We’ll have all the proper documentation for this in place for the 1.0 release.