The article shows how to apply the CQRS pattern with Blockchain technology to solve a business case that it’s not possible using the native features that the Blockchain platform offers.
You will find in this article:
- Problem context: Why we have this problem. From business side and technical side
- Example scenario: Everything seems easier if we have an example
- Which Blockchains can take advantage of this approach
- Introducing CQRS concepts
- Explaining why CQRS and Blockchain
- Explaining why Event-Sourcing and Blockchain maybe not
- Source code example to solve the example scenario
- Recap of what has been explained
- List of compatible blockchains with this approach
I will change the article when I find things that are wrong!!
📢 ⚠️ The article might be ignoring some Blockchain platforms that might solve better the problem-solution described in this article because I might not be aware of. In that case, I will appreciate a comment to update the article with that new information. The Blockchain industry changes fast and be up to date is nearly impossible without full dedication. Thanks for your consideration.
ℹ️> You might want to skip the next part which is more the context of the solution, if you want to read the technical approach jump to “The Case” section
In order to know why we might need CQRS models on Blockchain, let me explain the problem we want to solve first.
All products are the specific solution implementation over a problem space. The solution space has to be guided via the problem space, otherwise we might be building something that it’s not needed by the user.
The problem space is your domain. A domain is what the business wants to solve. We might have the case of an asset management platform. The domain is the asset management.
We might want to divide the domain into sub-domain to have a better way to organize our solution space, a basic Divide and Conquer strategy. We might want to define/implement each differently on depending on the requirements. It might be more familiar to you if you know Domain-Driven Design or Microservice Architecture, but it’s not required to follow this article.
We want to define those sub-domains because, one reason but not the only one and nor the most important, it will just couple one sub-domain with Blockchain. So, we contain all possible problems with Blockchain on that specific sub-domain and not let all the application suffer of that undesired coupling.
You might want to use Blockchain on more than one sub-domain, it might be needed, but I would suggest you to not start that big since you might need first get experience with Blockchain and, if at the end you need to drop that implementation, you will have to switch that sub-domain solution to another. So, controlling the accidental complexity that Blockchain brings to the system into just one sub-domain will be enough work and it will give you enough headache, don’t start with two sub-domains using Blockchain. Just a recommendation. 🙂
On each sub-domain, you will have different classes that represents a business concepts.
That business concept, we will refer it as a Domain model. Domain model is a conceptual model of a domain that contains both data and behaviour. The Domain model contains the code to solve the business problem. So, it gives value to the platform since it solves what the user need. As a developer, and as a company, you want to invest your resources on those models because, usually, it’s from where you get money.
We can agree that a Domain Model is always incomplete and imperfect, and some times useful. So, we need to iterate fast over our assumptions and adapt the business needs to the real user needs, that they might change faster than the company solution.
In order to know that we are building the correct tool, we want to have short delivery times and a fast feedback loop. The faster is this feedback loop, more competitive is your product in the market. In essence, being agile.
In the few cases, the solution is to use Blockchain technology. Which are that those cases? It’s out of scope for this article. 🙃
Let’s assume the solution space requires an immutable ledger to interact with 3rd party actors over a shared & trusted network. An actor in the system is anyone that interacts with the system we have defined.
Our case scenario is a company that provides an Asset Management System. As a development team, we have seen that Blockchain seems to be a promising technology that fits our initial needs. So, we do a research of which Blockchains we could use.
Some Blockchains have a better approach than others to solving any business problems. Let’s group two main ways to solve it, Smart Contract and Predefined features/contracts.
Smart contract approach, Ethereum style. It helps represent the whole business model. Important info for the reader, I’m not an Ethereum expert ⚠️.
Predefined features approach, NEM style. It provides a set of predefined transactions that combined brings a higher way to interact. Problem, the domain model cannot be defined to match our business like we might do with traditional tools. Important info for the reader, I’m more familiar with NEM Blockchain 🙂.
- Short development cycles
- Business Logic upgradable easily and at any moment
- Rich domain model
- Immutable data store
- Being used by anyone in the network
There is different ways to implement a Domain Model with Blockchain. In my humble opinion, all methods I know are too expensive and the development time too high compared to “traditional” technologies. Be sure Blockchain cost overhead is worth.
As development team, we have chosen the NEM Blockchain due to its focus on Assets.
So, which options we have to build a useful Domain Model? Because the first business requirement is use tokens, we find that NEM provides this model in its core. So, we have already what we need using NEM Blockchain. Path of Domain Model to Fits the current Blockchain Model, we have been lucky :-)
ℹ️> In case of NEM2, aka Catapult, you will have the option to extend the Blockchain behaviour for private chains, but I want to share the case where you haven’t this option to help projects using other Blockchain platforms that hasn’t the option to extend the on-chain behaviour but they provide a transaction that can contain a raw text message.
There is a business shift! The business team has seen that the customers need non-fungible assets. An asset is non-fungible when it is unique, has an unique owner.
- Fungible Asset: Tokens, there’s 100.000 X tokens, they are all equal between them.
- Non-Fungible Asset: A Car, you own the Car with ID 1202.
The business needs to put the product fast to the marked to verify the business assumption of value proposition. That’s why the Product Owner shared the concern that they need to try things fast and pivot faster with the development team.
After the first deployment, the development team realizes the NEM Blockchain transactions doesn’t represent all things that the business needs, for example, it hasn’t the native support for Non-Fungible Assets.
As development team, we have two options:
- Rewrite everything to use a Blockchain with Smart Contract support. It implies learning a new technology again, its limits and re-adapt what we have with the new technology. If we are lucky, we just have few things, but if had 2–3 components (mobile app, web app, server), re-write some of that logic might be painful and a resource waste. ❌
- Adapt the way we interact with the Blockchain to have more freedom defining Domain Models. The development team has acquired experience with the Blockchain and they know the limits of that technology plus they had started estimating better the features related to Blockchain, we don’t want to start this process again. Plus we can reuse part of that we already have. ✅
📢 We choose the 2nd option because otherwise this article doesn’t make any sense. By when you finish the article, if you still think I should explain why you shouldn’t migrate to a Smart Contract Blockchain, just leave a comment and I will reply you! 😁 Just take into account that there’s no perfect solution, just some useful.
Let’s procedure with the solution
The development team has to model a non-fungible asset on top of a blockchain that does not offers it natively, it looks for alternatives.
In case you haven’t time to read that full CQRS article, here a brief, incomplete and probably poor introduction but still useful to follow the article.
Using the CQRS pattern, the development team could create a virtual representation of a non-fungible assets using NEM native features, like transactions messages.
Command Query Responsibility Segregation pattern says that you can use a different model to update information (Command) than the model you use to read information (Query). (source)
So, we will use Commands Models to update the state and Query Models to fetch the state.
CQRS is often used with Event-Sourcing, but it’s not necessary to be used together. Event-Sourcing ensures that every change in the state is captured in an event object and persisted.
Event-Sourcing has some desired benefits that we want to apply on Blockchain.
- Complete Rebuild: We can rebuild the application state re-running the events from the event log on an empty application.
- Temporal Query: We can determine which were the application state at given point.
- Event replay: If some past event was incorrect, we can compute the consequences and apply a fix.
With Blockchain technology, some points are more difficult than others.
What’s an Event Log? It’s a database where all events are persisted.
A Command and an Event doesn’t contain logic, just data. They are Data Transfer Objects (DTO).
A Command shows the intention of changing the state. An Event shares that something happened. There’s a huge difference. A Command might fail, meanwhile an Event reflects something that already happened in the system.
Recap from where we come, we had the need to represent a business thing that should change its state over time and still use Blockchain.
A CQRS Model helps us to define a set of Commands that we will store on the Blockchain and publish our Query Models with a public API.
Storing the Commands on-chain helps us to re-create the state and audit the state cache that a third party is providing to know it’s valid or invalid.
Sharing the commands help us to quickly let others know how to interact with our Domain Model though a trusted network without the need of sharing our Domain Model internal logic.
As a team, we can choose the level of shared information that we want.
Case — Sharing Commands: people will need to a) rely on our API to read b) emulate our model logic in case something is wrong and prove we are cheating
Case — Sharing Commands & Model Logic: people will a) rely on our API to read or b) create an application to execute our model logic and don’t rely to our API.
Case — Sharing all the application: people will a) rely on our API to read b) start all the application on their end and don’t use us.
There’s a grey limit about sharing the Model Logic and everything.
In our case, we opt to share the Commands & Model Logic. We want others to be able to execute this by themselves if they want, and rely to us for some specific logic, like complex Query Models.
With CQRS model, we are able to scale differently the Command Model and the Query Model. We want the system to respond to complex query fast, but we don’t mind to have slow write since the asset creation or management isn’t as frequent as reading the data.
Also, this approach allow us to choose just share the Commands if we would like to. It’s very important to a company choose what to share and what not to share.
The imaginary company differentiation is how to manage the information, not controlling the asset creation.
We will use a NEM Account to store all commands.
Using Blockchain as Event log has some undesired implications.
The event’s order matters. In case you want to re-create the state of your application, the events must be stored in the exact way they have been produced.
Imagine your application announces the events #1, #2, #3, but they got included #2, #1, #3. The ledger is immutable, right? I don’t know how to solve this case if the ledger/blockchain doesn’t support a way to ensure that they got included in that exact order. It should be on-chain logic in order to support being used as Event Log.
In order to ensure that required property, you will find that you have to wait until the transaction is included in a block. In the best case just be included in one block, in the worst case, you should wait until X (depends on each Blockchain) blocks have been confirmed. It could the application waiting between 1 minute to 10 minutes or more. 😖 It forces the application to be adapted to how the blockchain works and its (not that good) user experience.
Imagine the user interacting with an application waiting a minute to have the action confirmed. A nightmare. 👎
On the other hand, we have that the interaction happens inside an application. It might not be ideal to concentrate the interaction on an application. We want to lead others interact with us thought the Blockchain. It might require fully open the source code to obtain the trust of the others. Not all companies see this business model suitable for them.
Because of the previous reasons, I suggest to not use Blockchain as Event Store yet. Hopefully they will get mechanisms in the future to help us use it in that way.
📢 In case you know a Blockchain that has a restriction of not accept Y transaction before X, please, leave a comment in the post. 😄
Using the previous pyramid, with event sourcing we are forced to share Commands & Model Logic. Because what it’s stored in the chain is the result produced after executing the Model Logic.
We take a look into the specification. It says:
A possible UML diagram could be
️⚠️ The following code is extracted from the nem2-nonfungible-asset library implementation. It might have changed by the time you read the article. I suggest you to check the files referenced below to have a more accurate code example.
How a Command looks like:
It does 3 things:
- Be created with the required information (lines 5–7)
- Create a specific Transaction of NEM2 with a specific text message (lines 36–41)
- Recreate from a Transaction (lines 9–23)
Notice that some information is missing until it’s pushed into the network, like the signer (owner).
In the future, we might like to store the transaction info inside the Command to know that they are executed in order.
The Domain Model, Asset, is designed as:
We have a way to re-create the state given all commands (lines 12–23). They delegate into the apply method where, because of TypeScript limitations, maps the commands into the methods that have the logic of apply the change.
On each apply method, we will place the business logic. In case it fails, we should have different strategies to report the failure to the creator. I will explain this in another post.
How to read from the Blockchain
We fetch all Transfer Transactions (line 20) and we get each Command (lines 21–27). After that, we re-construct the state of the Asset.
The most critical is being unable to sort by transaction height, we want to read from older transaction to newer. It’s still not implemented on the current Asset Repository.
We might need to fetch all transactions in memory and then reverse the order meanwhile those features aren’t implemented.
- State cache persisted
- Query models on that state cache
- Incremental command application via server side notifications (WebSockets)
- What if a command is included before another announced previously?
We assume that it will be concurrency on the writing and they might cause undesired command application. In case of the Domain Model has dependency on the order, it might cause some race-condition problems, but it might not be the case. We are aware of that it might happen and we develop our Domain Model taking that into account.
- The writing is as slow as the Blockchain Block
Yes, it’s. When we had chose a Blockchain/DLT technology, we were aware of the consequences. We cannot make it go more faster on writing, but with CQRS, we can scale how we read. The use case should have this properties, more reads than writes.
We tried a first iteration of a CQRS Model with Blockchain. It seems to be viable to apply and extend the features.
The solution proposed isn’t fully implemented and there’s still room for improvement.
Give it a try on your next library and share your experience with us and others!
You can check how the library evolves here.
📢 Add a comment with the Blockchain or Distributed Ledger that you think it could benefit from this approach and it’s not listed yet. If you think you could write the same library for another Blockchain, feel free to and share your lib here, so others could find how to implement CQRS models on more than one Blockchain.