CrudQRS, a pragmatic approach to CRUD and CQRS using LightSwitch
July 4, 2012 7 Comments
After last month’s tales of advanced client-side customization, I originally intended to share a server-side tale about “Nancy”‘s LightSwitch application. However, since HTML5 has entered the LightSwitch town as the new deputy, the MSDN team and I rapidly changed course; long story short: it’ll be a while before I will cover this in-depth on MSDN so I decided to throw a short summary on my blog, share the idea, perhaps start some discussions, … By the way, I love experimenting with LightSwitch to see how far I can push it. To quote Michael Washington from a mail after I wrote about MyBizz Portal (which happens to be the v1.0 of the application that’s running at Nancy’s company): “experiments that take LightSwitch to the next level… Way beyond…”
Short recap: what’s REST?
RESTful services, OData, CRUD, entity-based UI, I’m not sure what this is called when talking about architectural patterns. The idea is that your application consist out of multiple “resources” or “entities”. It’s custom to think of these as the “nouns” in the story of your business: “Customer” “Invoice” “Shop” etc. Traditionally, there are 4 verbs in the story: “Create”, “Read”, “Update” and “Delete”. Both the nouns and the verbs are found throughout the application:
- Client side: “CustomerList”, “Search invoices”, “Add a new Contact”, etc
- In the transport protocol (OData): get, post, put, delete (verbs) and URIs (nouns)
- In the database: “Update dbo.Customers set (..) where (..)”
This traditional approach offers quite some benefits, but has some downsides too:
- About the database: you’ll traditionally find a normalized database that’s optimized for all 4 verbs equally: creating, updating, reading and (not always) deleting data. A normalized database is great for supporting applications where all 4 verbs are equally important, but this is not always the case. Think about scalability: some systems (Twitter & Facebook, for example) need a magnitude larger capability for querying, then creating and updating the data. Think about performance: how long will a query take “that returns the average increase in revenue per site, per month, over the last 15 years, compared to the average money spent on training per employee during the same period”? When this “reading power” becomes a real requirement, you’ll find benefits in adding a denormalized database to your application. Welcome to the world of BI (datawarehouse, SSAS, Cubes, ROLAP or MOLAP, etc). Once you have all this BI power, isn’t it a huge waste not to reuse it in the queries that fill the screens of your client application?
- The same scalability can be extended to your services too. If a service can handle all (CRUD) verbs regarding a particular noun, then they can’t scale to reflect the need for “reading power” being magnitudes smaller or larger than the need for the other verbs.
- Also, being limited to just 4 verbs means it’s hard to capture the intent of the end-user, the business process that drove a certain change on an entity. A customer isn’t “created”, he/she “joins a mailing list”. A customer doesn’t “update”, he/she “moves house”. In the latter case, the business might want to send an info-brochure to the new address to inform the customer of the nearest shop? Perhaps you can this process is free but only (maximum) once a year, otherwise the customer has to pay some administration costs. In these cases, there’s a distinct difference between how the application should react when “the customer’s address contained a typo”, “the customer moved” or “the name of the street was changed”. This difference can’t be expressed by the noun+verb combination “Update Customer”. By limiting the verbs to a vocabulary of only 4, you’re hiding what really happened.
Short recap: what’s CQRS?
CQRS addresses these scalability issues and issues to capture the intent of the end-user in particular. CQRS is an architectural pattern that advocates the benefits of having ”Commands” (C) and “Queries” (Q), and clearly splitting the responsibilities of the two (Responsibility Segregation, RS).
You can read more about this architectural pattern on Udi Dahan or Martin Fowler or Greg Young ‘s blogs. Although not *strictly* required, you’ll often see this responsibility segregation reflected front (a “task-based UI”) and end (at least two databases, one normalized and one denormalized) which I would recommend to fully enjoy this powerful architecture.
Notice that this is a totally different approach from CRUD. There’s no “CUD in CRUD” == “Commands” and “R in CRUD” = “queries” equation to be found.
- The commands represent actual business processes. ”MoveCustomerCommand” vs the CRUD “UpdateCustomer”.
- The queries have dedicated services, and sometimes (I recommend it) dedicated databases.
This system has a lot of benefits and doesn’t suffer from the same scalability issues or troubles trying to capture the intend of the user, but it comes with its own set of downsides:
- It’s more work to set up & maintain.
- The two databases can be out of sync, resulting in differences when reading the data straight after submitting a command (although end-users have gotten used to their data being async really, think about sending a tweet for example).
- Introducing a “reinventing-the-wheel” vocabulary (read about SOAP vs REST: Vocabulary re-use vs. its arbitrary extension: HTTP and SOAP)
- I doubt many end-users will enjoy a UI that has a button for invoking the “MoveCustomerCommand”, followed by a button to invoke the “ChangeCustomerStreetNameCommand” and a button “ChangeCustomerAddressBecauseItContainsTypoCommand”.
- LightSwitch doesn’t support CQRS out-of-the-box, so you’ll find it very counter-productive to force a LS UI&service to this model.
Introducting CrudQRS, a pragmatic approach to CRUD and CQRS
LightSwitch v1.0 made it rather difficult to blend different LightSwitch apps together, but because of the switch to OData, it’s quite easy to blend the middle-tier of different LightSwitch applications and call one from the other. To benefit from the advantages CRUD and take advantage of some CQRS benefits, you need:
- One very normal LightSwitch app. This is the CRUD-part. The “nouns” are domain models that encapsulate the business rules, validation, etc. However (using some manual plumbing because there’s no “generic” access to the write code extension points), when an entity is saved, it persists this change to the database, but also logs it to a generic auditing table:
- One LightSwitch data-service that hosts the “generic auditing table”, it’s largely based on Paul Van Bladel’s articles on generic auditing trails.
- One long-running process (of any kind) that takes the entries from the auditing table, and processes them by updating the data in:
- One denormalized LightSwitch data-service. Basically, this service is optimized to provide all the “reading power” you need, in custom reports, in Excell power-pivot tables, in specialized screens (in the normal LS app described in the first item that might have performance issues because there’s a lot of different entities being shown on each line), …
The second data-service might seem like an overhead at first: why wouldn’t you update the denormalized data-source directly? Well, because
- The denormalized data-source is geared towards reading only. Thus, updating might take a lot of time. You do not want your end-users to wait for this, hence the need of some kind of asynchroneous process, some kind of queuing system. I could have chosen many solutions for this (MSMQ for example), but abusing a LightSwitch OData service for this got me up and running in under 15 mins.
- You get the added value of having an auditing table for free. This table can also be useful in the LightSwitch app, to do… well… auditing
And that, my friends, completes the circle of CrudQRS, an architecture made (almost) entirely in LightSwitch, with the advantages of building out CRUD-based applications really fast, but is scalable and optimized for complex queries like a CQRS system would be.
I’m currently experimenting with a second table in my LightSwitch data-service (2.) that hosts a command table, because not everything is easily expressed in CRUD. These commands can be “instantiated” by the user by creating reusable modal screens. When I’ve played with this a bit more, I think Nancy’s app will be the pragmatic mixture between CRUD and CQRS that I intended it to be.
Just like “pure-CRUD” or “pure-CQRS”, this semi-in-the-middle-architecture comes with benefits and downsides. Choosing the “right” architecture or tooling is always a “scenario-dependent” choice. It worked for Nancy. I’m sure I’ll have many more applications where it’ll work. I’m just curious: do you see any applications where this might be the choice of architecture for you? And: would you be interested if I covered this more in-depth?
Keep rocking LS!