Navigating Babylon Part II

How to Introduce DomainSpeak in Testing

First, let’s start with a quick overview of the problem I discussed in Navigating Babylon Part I. Microservices create efficiencies in development in a world dependent on remote work environments and teams. Unfortunately, the separation of workers and teams results in the tendency for microservices to encourage the development of multiple languages or dialects that obfuscate communication and further complicating testing. We have our anti-corruption layer and we don’t want to pollute our code by spilling in sub-system language.

A Domain-specific Vocabulary for Testing: DomainSpeak

There is, however, a pragmatic solution: we can build on the anti-corruption layer by creating tests in a specific language that has been created to clearly describe business concepts. We can and should create DomainSpeak, a domain-specific vocabulary or language, to be used for testing. Once we publish information in this language it can be shared across microservices and thus influence the workflow. Periodically, as is done in the English language, we may need to improve definitions of certain vocabulary, by re-defining usage and disseminating it widely, thus influence its meaning.

How will this DomainSpeak improve testing?

For integration tests, all the different dialects should not permeate your integration tests. You need to be very clear that a word can only have a single meaning. This requires a two-part process:

  1. You need to verify that you are not inconsistently naming anything inside an actual test; and,
  2. You need to do translations in an anti-corruption layer so everything inside is consistent.

What does DomainSpeak look like in a practical sense?

When you consider how brands influence pop culture, it is through language.

In the business world, marketing professionals use domain specific languages to create a brand vocabulary or a BrandSpeak. All major and influential brands and even smaller yet influential brands have a specific vocabulary, with specific definitions and meanings, to communicate their brand to the public. All communications materials are integrated into this system.

Brand specific, intentional vocabulary, has the ability to invade and permeate. Many people are completely unaware that it was a DeBeers commercial in the 1940s that created the cultural tradition “a diamond is forever.” Other examples, “Don’t mess with Texas” came from an anti-litter campaign and although we know it’s a marketing ploy, just about everyone is on board with the idea that “What happens in Vegas, stays in Vegas.” On an international level, if you order a “coke” you will most likely get a carbonated beverage, but you won’t necessarily get a Coca-cola.

As I referenced in my first discussion on Navigating Babylon, I recommend implementing a mapping layer between the microservices and the test cases. Next, when deciding to address the language used, we take it a step further. Now focus in on the language or the DomainSpeak and how this domain-specific vocabulary improves the associated output of the test cases. This means that for example, a Customer, a User, and a Client all have specific meanings and that they cannot be interchanged.

What is the process to create this language?

The initial process is an exploratory step. To create your own DomainSpeak your testing department will need to communicate regularly with the business owners and developers. Their goal will not be to dictate what words the business owners and developers use, but to learn what words already have meanings and to document these usages. The more your communicate, recognize and document adopted meanings, the more you will discover how, where and why meanings differentiate.

For instance, the business may see a Customer as a User with an active subscription, whereas a microservice might use the words interchangeably as they do not have the concept of a subscription. You will also notice that sometimes situations may give rise to conflicting meanings. A developer may have picked up the word “Client” from a third party API he integrated with for “User,” whereas the business may use “Client” for a specific construct in a billing submodule for “customers of their customer.” In such situations, to avoid confusion and broken stuff, you will need to specify which definition is to be used and possibly introduce a new concept or word to account for the narrowing of the definition. Perhaps the “customers of their customer” will now be a “vendee” instead of a “client.” Don’t dismay if there is not an existing word that accurately matches your concept, you can always create a new word or make a composite word to meet your needs.

Indeed, by being consistent and by distributing your use of language to a wide audience you can introduce new words and shape the meaning of existing words. This means that your tests have to be very close to a formal and structured form of English. This can be accomplished by using Aspect-oriented testing or by creating fluid API wrappers on top of the microservices. Aspect-oriented testing would look like this (cucumber syntax):

Given a User
When the user adds a Subscriptions
Then User is a Customer
Whereas a fluid API would be something like this (C# syntax)
User user1 = UserManager.CreateNewUser();

This creates a lot of focus on syntactic sugar* and writing code behind the scenes to ensure that your code incorporates your business logic (your test) and looks the way you want it to. Every language has their own way to solve this challenge. Even in C, you could use macros to take a pseudo code and turn it into something that would compile, and chances are that your results would be far superior to that of your current language usage.

For my uses, the cucumber syntax, with a side of extra syntactic sugar that allows me to define variables, is very effective. (I will get into this in more detail another day.) Whichever language you use, keep in mind that the goal of creating a DomainSpeak vocabulary is not to make your code look pretty, but rather to ensure that your code communicates clearly to the business and developers and that meanings are defined with precise and consistent language.

The End Goal is Efficient Quality Assurance

The goal, after all, is to improve productivity and deliver a quality product. Clear communication will not only benefit your team internally, it will also influence other teams. By communicating your results in consistently clear and concise language to a wide audience, you will influence their behavior. You will be able to efficiently respond to questions along the lines of “we use ‘customer’ for all our ‘users.’” You will also be able to easily define and answer where the rule may not hold and why you use the word you use. Again, the goal is not to dictate to folks what words to use, but to explain the meanings of words and to encourage consistent usage. Adoption will follow slowly, and over time usage will become a matter of habit. Over time services will be rewritten and you should be able to delete a few lines from your anti-corruption layer every time one gets rewritten.

*syntactic sugar allows you to do something more esthetically, but not in a necessarily new way. Different ways of saying the same thing. Just looks different. Not important because it’s not new, but significant because it makes things more readable and understandable. Clean up language/clearer code and therefore easier to find a bug.

If you enjoyed this piece please share it with your colleagues! If you have something to add, please join the discussion!

Navigating Babylon: Part I

Navigating Babylon

“What do you mean?” is a common phrase, it is the communication equivalent to a checksum; making sure that the words are interpreted correctly. This is what testing does, it ensures that the business concepts are interpreted correctly. There are bugs where the code does not do what the developer intended, these have been addressed by tools and methodologies, and are becoming rare. The bugs that we want to talk about here are the ones where it does do what the developer intended, but not what the business desired.

What better tooling has not solved is the misinterpretation of requirements.

Again we are back to the problem of working with humans. Errors that result from misinterpretations are becoming the dominant target in testing, but to find them, it is imperative that the language that describes the tests is unambiguous. This is becoming increasingly challenging as distributed teams create isolated dialects.

Let’s look at two different ways to overcome communication problems. One option is to isolate ourselves from dialects using the anti-corruption layer pattern. A second option is to make language across teams cohesive by sharing information with clear and specific language. Effective, clear communication flows naturally and increases efficiency, which is our ultimate goal.

Microservices are adapted to today’s remote workforce and they are increasingly used because of their efficiency. Microservices on the development end reduce complexity and decrease the need for constant communication between teams. An unfortunate side effect is that over time individual microservices naturally go their own way and as a byproduct, a microservice specific code is created using unique language/dialect/slang. Microservices tendency to promote the creation of distinct vocabulary and meanings increases the likelihood for mistakes/bugs and broken tests.

Improving on the Monoliths of the Past

One of the benefits of monolithic architecture is that everyone spoke the same language. Yes, the user object was enormous and was far more complex than necessary, but everyone spoke the same language because everyone used the same object. Everyone agreed that User.Id was the identifier.

The intention behind microservices is to make life simpler and reduce complexity, but at the same time, they have increased complexity by creating many teeny distinct monoliths that naturally encourage speaking slightly different languages or dialects. We have gone from the Pyramids to the Microservices Towers of Babel. Another analogy might be to say that we have created geographically isolated human populations that encourage the development of distinct dialects and regionalisms. Where we once had User.Id, we may now find one microservice has chosen User.Identifier, while the next microservice has settled on Customer.Id, and yet in another one we find Agent.Uid.

Despite this drawback, Microservices are still a natural efficiency.

The solution is not to eliminate microservices, as they fit the increasingly popular corporate structure of remote work. Conway’s Law states that companies design software that mirror their internal structure, and as the individual developers become more isolated by remote work it makes sense that an architecture is used that supports the build out of small isolated components. Microservices are born as the way to accommodate the distributed nature of teams. However, as each service is built, we start to see slightly different terminology, slightly different assumptions, methods for error handling, exception behaviors and so on.

The Dangers of Misinterpretation

Writing the code for integration tests within microservices, we often end up with something like this:

Assert.Equal (a.SerialNumber, b.AssetId);

One service calls it a serial number, the other an AssetId. Technically, it is just a small issue that is easily understood in a conversation, but potentially grounds for a larger problem. Problems like this are only amplified when the respective developers work in different parts of the world and several different time zones. Sure, the company can clock development time 24 hours per day, but developer A has to wait 12 hours for developer B to get in and by that time developer A is already back in bed. And so, developers put in their “one line fix,” but the test cases that integrate these services tend to repeat this incongruent language.

There are patterns that solve this, the Adapter design pattern or the Anti-Corruption Layer domain driven design concept both look at solving these issues. The core idea is that you take the service that speaks a different dialect and wrap it in some code that takes care of all of your mapping for you. Effectively creating a layer where you put in all your “one line fixes.”

If one service thinks a field is a GUID, and the other a String, you convert it in there to a common format across all tests. The same goes for nullable fields, or fields named slightly different (Id, id, Uid, UserId, Identifier, UserIdentifier, uIdentifier, etc.). Create a layer for each service, even if there is no special mapping, to isolate your test cases from all the different dialects. Then create your own version of the message objects that have consistent naming, and then have code that maps your test objects to the microservice objects. If you use reflection to map the fields that match, you should be able to achieve this with relatively little code.

Now you might look at this process and think: “Wouldn’t it be easier if they just all named it the same thing?” Unfortunately, the answer is actually “no.” This problem is the direct result of the communication structure of an organization. To change the structure of communication you will have to change the organization, which is rarely a pragmatic solution.

Harnessing Babylon

Let’s look at two plausible solutions that are significantly more pragmatic than changing your overall organizational structure:

  1. Create a Department of Naming Stuff (Dons): Need a new field, ask the Dons for the name to use. They will look to see if it is already in the big book of names for stuff and, if not, they will add it and name it.
  2. Direct and consistent communication: Have developers communicate directly. If people talk to each other they start to adopt the same words for the same things, the closer the interaction the closer their use of words correlates.

Option 1

Languages (the human used kind, not the computer used kind) have this same problem. The Dons is comparable to the official Oxford Dictionary. Of course, we may have the problem that the US and UK don’t see eye to eye on what dictionary to use; especially on the common informal words. The result is that in most scenarios, the experience of the Dons is likely to be heavy handed. And, the day the Dons makes a poor naming decision, the result will be to create a ridicule of the concept.

By the time the problem is advanced to the stage where this looks like a good idea you will find that retrofitting to a standard language is cost prohibitive. Especially as the problems it causes means the project is buggy, late and over budget.

Option 2

And, given our pretense for remote work, the second option is inherently impractical as developers work in different locations and across time zones. Option two might also continue to promote regionalisms, slang, and dialects that develop when people who live in close proximity start to adapt words to have specific meanings, meanings which are often not shared across the wider organization.

Breaking out of Babylon

Fortunately, we are no longer limited to two solutions. We have a third solution that first entered the scene with radio and television, but that has now become nearly universal thanks to the digital age. This third type of technology is a unifying language. In daily life, it is what is known to us as Pop Culture.

A Unifying Language

Pop culture targets media for consumption by large segments of the American population. By favoring words with specific meanings, pop culture means that widely distributed words and meanings become adopted not only into American English but around the world. Netflix has members located in 190 countries around the world and Facebook has nearly 2 billion worldwide members. We can google truthiness, take a selfie, and Facebook the results. Pop culture introduces new words, redefines words and narrows the meaning of words by repetitive exposure to specific audiences.

We can do the same thing with microservices: we can create our own culture. We can create and distribute test results with consistent specific meanings. By creating our own uses and definitions, we can appropriate the language that we need and define it for our specific purposes. QA then becomes the company’s “pop” culture influencer and over time effectively influences the meanings that people associate with specific words. This is not a quick process, and measuring change will be difficult.

To be continued next week: How to Create a domain-specific vocabulary or DomainSpeak.

Please share this article and join the discussion in the comments!