Publisher/Subscriber Pattern

Recently as part of rebuilding our interview process we were considering and additional question to test general architecture and problem solving skills.  It needed to be something quick (Could fit in the span of 10-15 minutes, to fit into the existing interview schedule), something that requires understanding one or more architecture pattern, and had enough complexity to trip people up if they move too quickly through it.

My suggestion to this was the Publisher/Subscriber pattern, a classic pattern that you can implement in-stack in JavaScript easily if you have a clear grasp of the concepts.  Beyond that, it's also and extremely useful pattern that deserves to be understood. So let's take some time to talk about, and implement, the Pub/Sub pattern!

The pattern, what it is and why we care

As developers one of our chief concerns is building maintainable and resilient software (Sitting only behind "working" in order of importance!), as one of the ways we can tackle both is to create clear, enforceable boundaries between our concerns. Why should code that deals with, say, student information have any direct connection to code that deals with college information? By creating boundaries we prevent the leaking of responsibilities, which helps make sure that our code stays maintainable.  It also forces us to take a closer look at how we communicate between concerns, do we keep using direct method calls forever linking the code? Do we use RESTful end-points to communicate information?  What do we do if we need to trigger more than one concern to act on our response?  This is where the pub/sub model comes into play.

The basic thought process behind pub/sub is to provide a consistent interface for async communication between concerns without creating concrete connections between those concerns using an event stack.  The general patterns goes as follows:

1. Concerns subscribe to events and provide a callback for each event.
2. When appropriate, any concern can publish an event with a payload.
3. For each callback tied to the event, the payload is passed in and executed.

The system is managed by a central point, assuring that at most there is only the one shared resource.  By creating this architecture we can easily separate our concerns by clear boarders, and with a sophisticated enough Pub/Sub implementation (Such as a message queue) you can create resilient systems that can recover from unexpected errors such as a database vanishing or a server being temporarily unavailable.

Like anything else this is not a silver bullet, but we'll talk more about that later.  For now, let's build a simple Pub/Sub implementation in JS to get our feet wet.

Building our implementation

Before we begin lets set some ground rules for ourselves.  Our implementation should allow for the following features:

1. Allow a callback to be registered for an event
2. Allow for events to be published with a payload
3. Allow for concerns to unsubscribe from an event easily

The first two are obvious requirements, the third one can sneak up on us as a requirement though.  There are times when a concern might want to decide that it no longer needs to listen for a specific event, so we should allow it that courtesy.

OK, now that we have our basic requirements let's do some initial setup:

export default Publisher = (function(obj) {
    return obj
})({})

A quick iife gets us started.  We'll be adding our functions into the passed in obj parameter.  When we're finished the exported Publisher object should be able to meet all of our requirements!

Next, let's find a way to track our events.  Ideally, events should always be a String or Symbol (or an enum in Typescript), so let's do our best to enforce that here. Given that we have many different events to track we should avoid an Array.  While we could certainly find a way to make that work it wouldn't be pretty or efficient.  A Map could work, but those allow for the key to be just about anything, and we want to keep ourselves limited. That leaves us with an Object as our base, so let's implement!

export default Publisher = (function(obj) {
    const subscriptions = {}
    return obj
})({})

Awesome.  By creating this subscriptions object we can rely on closure to make it available to our upcoming functions.  We don't freeze it because we need to be constantly manipulating it as we add new events!

Next let's talk about how we're going to add and track the event callbacks!  Obviously we'll need a subscribe method, which should take in the event and callback, but how do we store it after that?  We can't just store the callback attached to the subscriptions object by the event because more than one callback can be tied to an event.  The obvious choice here is an array, so for now we'll move forward with that and see what happens!

export default Publisher = (function(obj) {
    const subscriptions = {}
    
    obj.subscribe = (event, callback) => {
        if (!subscriptions[event]) {
            subscriptions[event] = []
        }
        subscriptions[event].push(callback)
    }
    
    return obj
})({})

Great! Now we have a clear way to subscribe to an event.  We find the event in our subscriptions object, if there isn't an event for that already we create a new array to store the callbacks, then we push to it! We now have a nice clean way of adding subscribers!

Now what about publishing events??  We know that we want events to listen for when they fire, and to pass the payload into any listening callback, so what does that look like in our code?  This one is easy, we just need a method called publish that accepts an event and a payload, and then iterates over that event's subscribed callbacks!

export default Publisher = (function(obj) {
    const subscriptions = {}
    
    obj.subscribe = (event, callback) => {
        if (!subscriptions[event]) {
            subscriptions[event] = []
        }
        subscriptions[event].push(callback)
    }
    
    obj.publish = (event, payload) => {
        if(subscriptions[event]) {
            subscriptions[event].forEach(callback => callback(payload))
        }
    }
    
    return obj
})({})

There we go, we're most of the way to our implementation!  We're able to both subscribe to events with Publisher.subscribe(event, callback) as well as publish events with Publisher.publish(event, payload), so the first 2 of our 3 requirements are next.  Now what to do about our last requirement, the ability to unsubscribe?

Well what's going on in our code right now when we subscribe?  We push values into our array right?  Well Array.push returns the index of the value we just inserted, so let's make use of that.

export default Publisher = (function(obj) {
    const subscriptions = {}
    
    obj.subscribe = (event, callback) => {
        if (!subscriptions[event]) {
            subscriptions[event] = []
        }
        const index = subscriptions[event].push(callback) - 1
        return () => subscriptions[event].splice(index, 1)
    }
    
    obj.publish = (event, payload) => {
        if(subscriptions[event]) {
            subscriptions[event].forEach(callback => callback(payload))
        }
    }
    
    return obj
})({})

OK, here we go, our subscribe function now returns a function we can use to unsubscribe! This is a solid pattern and it prevents us from having to sully Publisher with another method.  Our unsubscribe function itself is just splicing out the offending index, what could go wrong???

let s1 = Publisher.subscribe("test", () => console.log(1))
let s2 = Publisher.subscribe("test", () => console.log(2))
let s3 = Publisher.subscribe("test", () => console.log(3))

Publisher.publish("test")
//Console prints:
//1
//2
//3

s2();
s3();

//Console prints:
//1
//3

Uh oh.

We called our unsubscribe function, what happened???  Oh right! Because we spliced out the middle index when we called s2 we were left with a subscription array of size 2 instead of size 3, so s3 had nothing to remove!  OK, quick regroup then on how we're storing our subscriptions.

We need something different to store our callbacks in, something where we're not reliant on an index.

export default Publisher = (function(obj) {
    const subscriptions = {}
    
    obj.subscribe = (event, callback) => {
        if (!subscriptions[event]) {
            subscriptions[event] = {}
        }
        const symbol = Symbol()
        subscriptions[event][symbol] = callback
        return () => delete subscriptions[event][symbol]
    }
    
    obj.publish = (event, payload) => {
        const holder = subscriptions[event]
        if (holder) {
            Object.getOwnPropertySymbols(holder)
                .forEach(key => holder[key](payload))
        }
    }
    
    return obj
})({})

Here we go! Now instead of storing the callbacks in an array we can store them in an Object!  The object will be keyed with Symbols and because the events don't need to be executed in any particular order it's not a problem if the keys don't reflect that!  Now our unsubscribe function just needs to delete the callback by its symbol!  Our publish function also needed a slight update.  We need to get all of the Symbols from our holder and then use those to get our callbacks, but it's mostly the same. Let's try our same test from above and see what happens.

let s1 = Publisher.subscribe("test", () => console.log(1))
let s2 = Publisher.subscribe("test", () => console.log(2))
let s3 = Publisher.subscribe("test", () => console.log(3))

Publisher.publish("test")
//Console prints:
//1
//2
//3

s2();
s3();

//Console prints:
//1

Perfect.  Now we have a nice, simple, pub/sub implementation.

But at what cost?

So you want to go all-in with pub/sub and asynchronous communication, huh?  Great! Now let's talk about the problems you'll be facing!  There are a number of questions any time you move away from synchronous actions.  If you're making a call between concerns to get data, how do you represent that since you aren't just getting an immediate response?  What do you do to chain actions together?  How much is too much separation?

There is a lot of literature out there about asynchronous design, far more than I could ever write about and with far better explanations than I could give.  There are patterns, improve architectures, and plenty of best practices out there to help you along your path, and I highly suggest you dive in deep before going too far into implementation.

And that's it!  Hopefully this was a nice, high level dive into pub/sub and it's implementation!  Be sure to join us next time as I write about interfacing your Windows PC with a tree in the forest.

About the author