Picture this: You’ve been building node backends with Express or Koa, and you’ve found your groove in code organisation, structure and abstractions. Adding routes is always clean and you can do it at warp speed. Then, all of a sudden you have a new requirement involving some realtime functionality. No problem you think, Express and Koa both have web socket extensions. So you set it up, and that’s where the nightmare begins.
- Your beautiful code structure is now lying in pieces on the ground because sockets are event driven and persistent, unlike your existing routes
- The middleware patterns you have set up are gone, and you’re now processing, validating, and reacting to raw data that comes in through the socket
- The whole thing is a sort of tumour on the side of your once graceful app
If any of that sounds familiar to you, then rest assured: You’re not the only one. The truth is realtime apps are really tricky to get right, especially for developers who are used to building stateless REST APIs.
The real problem is that it’s the wrong abstraction level.
What do I mean by that? Essentially, sockets are the lowest level persistent communication mechanism you could conceivably work with on the web. You’re essentially doing two things: Pushing data down the socket, or pulling it out. Compare that with what we’re used to with even the most basic frameworks: Routing, request/response (or context) objects, built in async flow, etc. It’s a far higher abstraction to work with because you can ignore most of the details of what really goes into to dealing with HTTP requests.
Can we bring Web Sockets up to the same level? That’s exactly what I’ve been working on with Hexnut.
Hexnut is a middleware based, super lightweight framework. It can work hand in hand with frameworks like Express and Koa, even sharing the same underlying server object! It only really has two important concepts:
- When the client connects to the server, a special Context object is created which lives for the lifetime of the connection. The ctx object (as it’s typically named) is used to communicate with the client, and to build up state as messages go back and forth.
- Connections, messages, and closing events are handled in a middleware chain. When these events come in, certain properties on the ctx are set to indicate what kind of event it was, and the same ctx object is passed to middleware functions. The middleware can be used to set properties and to send messages to the client, and it can also choose whether the next middleware in the chain should be triggered.
For those who just want to see some code, let’s see how a basic server would look in Hexnut. First let’s install some dependencies from npm:
And then the server code:
Even as basic as this is, it should already feel like an improvement over the tangle of event listeners and global state you’d typically find. But to really begin to see the benefits, we should add some more middleware.
The next snippet is a little longer, but don’t worry — we’re gonna break it down.
Before we go into the code, I just want to point out what you’re probably already wondering, which is: Why is there a hexnut package and a hexnut-handle package?
Well, hexnut-handle is just a very simple helper package that you could easily write on your own (no seriously, check the code — it’s like 30 lines). Hexnut is designed to be a super simple core, with pluggable middleware allowing you to customise the application to tailor your specific needs. That’s exactly what hexnut-handle is — a reusable middleware helper.
Back to the code above! So the first couple of middlewares should be familiar if you’ve worked with web frameworks before. We basically add some functionality to easily send json messages to the client (lines 7–12), and also something that will automatically try to parse json messages from the client (lines 15–25).
The signature of every middleware is always the same.
Notice that middleware can be an async function, so you can easily await if needed! When you use the functions in hexnut-handle, they are also just transformed into this same signature.
From line 35 onwards, we first set up a counter at connection time, and then handle messages that increase or decrease the counter. hexnut-handles .matchMessage() should feel a little bit like setting up routing in Express/Koa, because basically that is what it is; We are defining when a message should be handled by a particular middleware, and if it doesn’t match, Hexnut just tries again with the next middleware instead.
Lastly, starting on line 61 there is one middleware that catches any message that wasn’t already handled, and in our little app that’s considered to be an “error”, and we inform the user.
Simple as it is, there are four major benefits we are getting here.
- Separation of concerns
- Predictable information flow
Although everything is now in one file, we can easily separate the controllers into their own files and folders. This structure helps us to make sense of code by allowing pieces to be decomposed into chunks.
Just like you have different routes in a http web framework, for sockets we can create a similar idea by matching messages. This simple idea can be built upon further to create more sophisticated systems, but even on it’s own it separates different functionalities into their own space. This is of course important because there’s no better way to create hard to debug code than by mixing a bunch of unrelated logic in the same place!
Basically a combination of Structure and Separation of concerns. We can predict how the data will flow through our app by following where a message will be processed and what calls will be made from there.
A Middleware architecture naturally leads to code reuse, especially with utility middlewares like those above, which add general methods to the ctx object, or middlewares that pre-process messages.
Contrast this to the approach of something like socket.io, one of original and most popular solutions for Web Sockets. Socket.io puts an emphasis on creating rooms and namespaces where users connect and are able to send messages to the server and to each other (ostensibly). Communication is event driven, which means that beyond this idea of rooms the developer must add their own structure. This makes it very difficult to adhere to the principles above, when the real problem you’re trying to solve is your application logic!
None of this is to say socket.io is bad. Another large draw of socket.io is that it can progressively enhance your Socket, so if your user’s browser cannot use real Web Sockets, they can fall back to long polling in the background. This is no small feat, and the way it works is very impressive! But the web has moved on, and this kind of polyfill solution is not applicable for many users now.