Introducing post-robot — smart cross-domain messaging, from PayPal

krakenjs/post-robot

This article is part of a series on PayPal’s Cross-Domain Javascript Suite.

Over at PayPal, one of the biggest problems we’ve been trying to solve is, “How do we create reliable, quality components to be loaded into our merchants’ websites and web-apps?”

Gone are the days of “redirect your customers to PayPal and we’ll redirect them back to you” — instead, we want to push our checkout experience further up into merchant sites, seamlessly and securely, so customers never have to switch contexts in order to pay.

This article is part-one in a technical series on how we’re working on accomplishing this, and how we’re transitioning to a “cross-domain component” model, and open sourcing the tools we’re using to do so.

I touched on what I mean by cross-domain components in a Node Summit talk earlier this year. There are a bunch of complexities involved in building these kinds of components. The domain boundary is a killer. Site-speed issues become compounded by iframes and popups. Interaction bugs crop up from running on third-party sites, alongside a wild ecosystem of existing javascript. All of these factors, and more, make this a non-trivial task.

I touched on a lot of these difficulties in my talk, but for now I want to take a deep dive into how we tackled the problem of messaging between different windows and frames, on different domains, in order to build cross-domain components that can reliably communicate with their parent, and other frames. This turned out to be a pretty tricky problem to solve.

But it’s pretty easy right? Just use window.postMessage…

Well, yeah, window.postMessage works, technically speaking. In fact, it’s really the only way to message across two different domains, unless you start throwing servers into the mix.

The problem is, window.postMessage is a primitive — just like many other browser apis, it works, but using it in a scalable and reliable way is an exercise left up to the reader.

If you haven’t seen it already, post-messaging looks like this:

// In the listening window
window.addEventListener('message', function(event) {
    console.log('I got a message:', event.data);
});
// In the sending window
window.parent.postMessage('Hello other window!', '*')

So what’s the problem with this?

• It’s 100% fire-and-forget. This is the worst part. I can send a message, but I have no idea if the window I’m messaging has closed or changed location, or if my message got through, or if the other window even set up a message listener, or if the listener error’d out… or any number of things that could go wrong with asynchronous messaging.
• I’m only allowed to set up a global message listener, which accepts any messages sent to my window, from any other window that chooses to message mine. I can’t even pick my own event name — it has to be `message`. So if I want to set up different messages for different use-cases or different operations, I have to be careful to use some kind of convention, like adding an `operation` field to each one of my messages. God help me if I have multiple libraries on my page all sending and receiving messages in different formats.
• I have no way of getting a response or callback back from the other window, without having it literally just send another message straight back to mine. And because I only have a global message handler, I need some iron-clad way to match up a response to the message I sent up originally, which means persisting some kind of unique id and keeping a bunch of state, and generally exposing myself to all kinds of potential bugs and spaghetti code.
• I can only send strings through this api. Modern browsers support sending objects, but if I want to support Internet Explorer, I still have to serialize everything to a string. That’s not a huge deal — I can just JSON.stringify everything, but it also still means I can’t do more advanced things like passing across functions or other data types very easily.
• Internet Explorer (even Edge) doesn’t support sending postMessages between a parent window and a popup, only between iframes. So, good luck if you want to build a component that needs to be rendered in a popup, rather than in an iframe. For security reasons, we have a number of cases where a popup is the only mechanism for rendering a component inside a merchant page.

So let’s build a wrapper around postMessage?

OK then!

That’s what we did — we tried to come up with the sanest and simplest possible api for asynchronous cross window messaging, and we ended up with post-robot!

Introducing post-robot

krakenjs/post-robot

The idea behind post-robot was to try to solve as many of these problems as possible, with the simplest possible interface. We wanted something that would:

• Support a client/listener pattern, where the listener could receive messages and send responses back to the client with minimal effort.
• Handle any errors resulting from a message being sent (or failing to be sent), and allow the sender to gracefully recover from those errors. That way, if the message didn’t get through for any reason, the client would be able to fail gracefully.
• Automatically serialize and deserialize messages
• Support promises and async/await, along with callbacks, for recieving responses
• Allow setting up a secure message channel between two windows, through which messages would be guaranteed to be sent and received to-and-from the specified windows and domains.
• Allow transparently passing a function from one domain to another, and allow it to be called back in the context of the original window (more on this later).
• Work with Internet Explorer, and enable sending messages between a window and a popup, not just between iframes

How did we achieve all this?

Client/Listener with messages and responses

First things first, let’s get the basic api working. I want a listener to receive messages:

postRobot.on('getUser', function(event) {
    return {
        id:   event.data.id,
        name: 'Noggin the Nog'
    };
});

And I want to be able to message that listener from another window:

postRobot.send(win, 'getUser', { id: 1337 }).then(function(event) {
    console.log('Got user:', event.data.name);
});

Perfect! Now I can send a message from one window to another, with a specific event name and some data, and based on that message my other window can return a response which gets passed back directly as a promise.

The event I get back has the same `source`, `origin` and `data`attributes as a regular postMessage event, but the serialization and deserialization is taken care of automatically, and my message is scoped to a particular event name, which I can use to namespace messages. I can send as many messages as I want, in whatever order I want, and they’ll all come back to the right place.

What about error handling?

OK, let’s add some:

postRobot.send(win, 'getUser', { id: 1337 }).then(function(event) {
    console.log('Got user:', event.data.name);
}).catch(function(err) {
    console.log('Something went wrong:', err.stack);
});

Now, if anything goes wrong sending my message, I’ll be able to gracefully handle that error case.

In the background, when post-robot sends a message to another window, the other window will immediately send back an `ack` message to indicate the message got through. After it’s done that, it has as long as it needs to send the full response — but if the original window doesn’t get an `ack` after a short amount of time, it will know something is preventing messages from getting through. Whatever that problem is, it can be handled as an error case by the caller.

You can also add a timeout for the full message response, and handle that as an error case:

postRobot.send(win, 'getUser', { id: 1337 }, { timeout: 5000 });

There are a bunch of other reasons my message could error out:

• Maybe the other window was closed
• Maybe it was redirected to a different page
• Maybe the listener was never set up on the other window
• Maybe the event loop in the other window is backed up and not responding

The great thing is, in any of these circumstances, the message sender will know for sure if the message has got through, and be able act accordingly. It’ll even error out if the message handler itself throws an error:

postRobot.on('getUser', function(event) {
    throw new Error('oops');
});

So in theory, as far as error handling goes, all my bases are covered.

What about promises and async/await?

We’ve already seen promises being used when sending messages. But we can also use a promise to return the response asynchronously:

postRobot.on('getUser', function(event) {

    return getUser(event.data.id).then(function(user) {
        return {
            name: user.name
        };
    });
});

This even works for rejected promises. In this case the promise fails, and the caller in the other window gets an error:

postRobot.on('getUser', function(event) {
    return getUser(event.data.id).then(function(user) {
        if (!user) {
            throw new Error('No user found!');
        }
    });
});

How about async/await? Well, it’s all promise based, so without any additional work I can throw in async/await. First let’s set up an async listener:

postRobot.on('getUser', async ({ source, origin, data }) => {

    let user = await getUser(data.id);

    return {
        id:   data.id,
        name: user.name
    };
});

Then send a message to it and await the response:

try {
    let { source, origin, data } =
        await postRobot.send(win, `getUser`, { id: 1337 });
        
    console.log(source, origin, 'Got user:', data.name);
} catch (err) {
    console.error(err);
}

Since postMessaging is already an asynchronous api, adding support for promises and async/await to postRobot was essential for us — if only because the benefits for error handling are very difficult to live without.

But we’re still listening for messages from all windows…

This is true. We want to be able to limit our messages to the exact domain and window we expect them to come from, otherwise we run the risk of getting spoof messages from any other window or frame that might be loaded at the same time as us, on any untrusted domain. Alternatively, I could message a window, and send privileged data to a domain or window I didn’t expect.

So let’s set up something a bit more secure. If we know the specific window and domain we want to listen to, we can set up a channel directly between them.

First, let’s set up the listener:

postRobot.on('getUser', {
    window: childWindow,
    domain: 'http://zombo.com'
    
}, function(event) {
    
    return {
        id:   event.data.id,
        name: 'Frodo'
    };
});

Then let’s message it:

postRobot.send(win, 'getUser', { id: 1337 },
    { domain: 'http://zombo.com' }).then(function(event) {
    
    console.log('Got user:', event.data.name);
});

Now I can sleep easy knowing that it’s not possible for a rogue window to send spoof messages to my listener, or for my messages to be read by the wrong window or domain under any circumstance.

I can even use a nice shorthand if I want to set up multiple listeners or clients. In the listening window:

var listener = postRobot.listener({
    window: childWindow,
    domain: 'http://zombo.com'
});
listener.on('getUser', function(event) {
    return {
        id:   event.data.id,
        name: 'Frodo'
    };
});

And in the sending window:

var client = postRobot.client({
    window: window.parent,
    domain: 'http://zombo.com'
});
client.send('getUser', { id: 1337 }).then(function(event) {
    console.log('Got user:', event.data.name);
});

How about passing functions?

This is my favorite feature of post-robot.

Passing functions is something that at first seems straightforward. Couldn’t I just do:

myFunction.toString()

and pass it along in the data object to the other window?

Well, I could, but crucially it’s not the function’s code I’m worried about sending across — it’s the ability of the function to be called in its original window, along with whatever closure scope it has.

So if I do something like:

var myUser = getSomeUser();
postRobot.send(win, 'passUser', {
    
    id: user.id,
    logout: function() {
        return myUser.logout();
    }
});

I want the other window to actually be able to call the function I’m sending them, in the context of the original window. Since that original function has `myUser` in its closure scope, it can then call

myUser.logout();

So, when I pass a function in the payload of a post-robot message, it will actually take care of this for me! What happens is:

• post-robot takes the function, and persists it locally in the sending window under a unique id
• It then sends an object in place of the function to the other window, containing that id, for example:

logout: { __type__: '__function__', id: '34fd3e4gf' }

• On the other window, post-robot recognizes this object, and deserializes it into a special wrapper function
• When someone calls that wrapper function on the other window, post-robot returns a promise to them.
• post-robot then takes the arguments and the function id, and passes them back to the original window in a post message
• The original window looks up the original function, using the id, then calls it, then sends the function’s return value back to the calling window
• post-robot on the calling window resolves the promise that we returned to the caller, with the return value.

So on the window which has been passed a function, it ends up just looking like:

postRobot.on('passUser', function(event) {
    
    var user = event.data;
    user.logout().then(function() {
        console.log('The user was logged out!')
    });
});

It feels like I’m just calling a regular promise-returning function on my window, but actually I’m calling a function on an entirely different window on an entirely different domain.

The cool thing about this is, I can expose little chunks of functionality from one window to the other, as functions, and they can be called from the other window. My code doesn’t even need to be aware that behind the scenes it’s triggering post messages.

This way, I could even set up my entire application such that it only has a single post-message listener to expose the initial set of functions I want to expose, and the rest of the cross-domain functionality is passed across within those functions.

This is secure, too: only functions that I pass explicitly in a post message become available to the other window, and even then, none of the closure scope or anything else is made available to that other window. All the other window can do is call the function and pass arguments back to the original window.

On top of that, the original window, which owns the original function, will ensure that only the window to whom it passes a function can send a message back to invoke that function — so it’s not possible for a rogue window to guess a function id, and invoke it arbitrarily. I do still need to make sure I specify the domain I want to send functions too, however, to ensure that only a given domain is allowed to call functions that I pass.

All in all, this allows for some pretty cool composition of cross-window, cross domain code, now that I’m dealing with function calls, not just message listeners and senders.

Cool! But what about IE?

I won’t go into too much depth on this, because IE isn’t the sexiest thing to talk about. But post-robot provides a pretty neat way to get messaging working from a parent window to a popup.

The trick is, it sets up a hidden, background iframe on the parent page, on the same domain as the popup window. The popup window then sends messages to that iframe, then those messages are proxied up to the parent window.

post-robot handles all of this transparently — all you need to do is point it to this iframe, and it takes care of the rest. In the background, it actually uses function-passing (explained above) to set up a secure tunnel from the parent window through to the popup.

Give it a try!

If you’re writing any kind of cross domain system, and you need to do messaging from one window or frame to another, give post-robot a try. Hopefully it’ll make your life a little more sane!

Pull requests are also totally welcome, if you’re interested in improving the library.

What’s next?

Next, I plan to write in more depth about xcomponent, and how it leverages post-robot to allow for the creation of truly cross-domain components, with props and data-down actions-up, all across the domain boundary.

—

Thanks!

Daniel, PayPal Checkout team