Why I emphasize DSLs for Domain Experts

… and not so much for software developers

I have recently been asked (by Jurgen Vinju) why I always promote DSLs for domain experts and non-programmers, and not so much for developers. So in this post, I’ll explain my position.

Before we continue: I am not at all suggesting that developers should not use DSLs. In fact, there are many cases where it would be useful. This text is about why it is harder to convince them.

Defining Terms

We should probably define what we mean by developer. To me, a developer is somebody whose main skill is software engineering (or development or craftsmanship, if you prefer those terms). They know how to write robust and maintainable code, understand patterns, know relevant technologies and languages, are up to speed with modern libraries and frameworks and understand and care about non-functional concerns. Importantly, they can deal with complexity, for example, by defining their own custom abstractions with the help of general-purpose programming languages. A significant subset of developers define themselves specifically through this ability to conquer complexity.

Of course there are people who use general-purpose languages to implement functional requirements in a particular domain. For example, in one of my current projects, the people who we built the DSL for currently use a general-purpose language to implement tax calculations. I see them really as domain-experts who — by necessity, lacking a better alternative — use the tools of developers. I do not really see them as developers in the sense of the above definition. And many of them don’t see themselves as developers either.

We must also define the notion of DSL in this context. You can argue that developers use lots of DSLs, languages that are (perhaps) not Turing complete and fill a particular niche in the developer’s toolbox: HTML, CSS, SQL, Matlab, make, maven and gradle are some of the examples that come to (my) mind. However, while these are DSLs by most definitions, in the context of this discussion, they are out of scope. The reason is that these languages are not developed by the organization or team. They are just there, and most developers don’t even consciously recognize them DSLs. Oh, and I don’t consider fluent-API-style internal “DSLs” as DSLs either. They are just APIs with a particular syntax.

Why developers are hesitant to use DSLs

So, getting back to the original question at the top: I am indeed skeptical regarding the development and use of (custom-developed) domain-specific languages for people who consider themselves skilled and competent software developers (and are not really those “programming domain experts”). Let’s explore examples and reasons.

Reduced Benefit

The most important one, in my opinion, is that the benefit just isn’t as great. If you convince your domain experts to use a DSL to formally specify functionality (instead of writing requirements documents), you extend programming-style activities to much earlier in the overall development process (often called front-loading). This can hugely increase productivity and quality of the whole process. Replacing a library or framework used by software engineers by a language that has better syntax and potentially better static checking just does not have that same leverage.

Workable Alternatives

The second reason hides in the previous paragraph: of course developers use lots of reusable abstractions in the form of frameworks and libraries. They might have less convenient syntax as a good DSL and are worse regarding analysis and error reporting, but developers are able (and willing) to deal with this. Domain experts might not be.

Integration

More technical reasons: for developers, integration with existing libraries, frameworks and developer tools is mission-critical. Doing all of this for a DSL can be a lot of work. Domain experts usually do not rely on such a wide array of technology, and they prefer to work in a more self-contained environment, such as a DSL and its IDE.

Cultural Reasons

Other reason are cultural. A DSL by definition restricts the user in some way (even though it also provides lots of convenient shortcuts for important behaviors). But developers usually do not want to be restricted, they want to be able to “express everything” if necessary. And while incremental language extension in the style of mbeddr or KernelF-based DSL allow users to “express everything” by reverting back to the base language, it is still not quite as seamless as with frameworks and libraries. There are more cultural obstacles. Developers often don’t like to rely on tools they consider off the mainstream, which language workbenches typically are. And many developers continuously try to optimize their CV, and to make themselves attractive to potential future employers. It’s important to have stuff on your CV that is recognized, proprietary DSL of course are not. Note that these statements here are not judgements — but they limit the interest in DSLs.

But Developers do use DSLs, don’t they?

Of course, many developers build there own utilities to make their task easier, some of them with quite sophisticated specification syntax that should be considered DSLs. But usually these address only a part of the development workflow (interface definition, state machines, and the temporal specification in the example below), and integration with the rest of the system happens on the level of the generated code — developers don’t mind looking at that, in contrast to domain experts.

Examples

Let’s look at some examples and see what we can learn. The first one is near and dear to my heart: mbeddr. This is a set of incremental extensions to the C programming language to simplify embedded programming. While we learned a lot building it and developed lots of skills and utilities on the way, mbeddr itself is a failure: we were not able to achieve meaningful adoption with embedded software developers. We probably made mistakes in marketing and sales, but we very definitely ran into a cultural barrier as well: many embedded programmers really want to know every bit personally. This collides with the notion of abstraction and code generation. By the way: mbeddr is used very successfully by itemis internally to develop embedded software. So it’s not like it doesn’t work!

Another failed example: one of the earliest uses of MPS was within Jetbrains. They had developed a kind of statically typed JavaScript and DSLs for web UI development. After a while, and after developing actual products, they abandoned this approach. The reason was that the developers preferred to use the native web libraries directly. We can only speculate about the reasons, probably several of those I’ve mentioned above played a role. One particular point I heard is that the MPS-based approach relied regeneration, whereas the native one just required pressing Cmd-R in the browser to reload the page. Apparently the benefit from the DSL did not outweigh this (really not that huge) slowdown.

Another one, this time not a failure. At my current customer, developers have to deal with bi-temporal logic. They have developed a little specification language to specify the temporality of the data that would then be used within a procedure. While there is a simple specification language, this tool really falls more into the category of utility and it’s not a full-blown DSL. This one was used in production, by technical developers.

As soon as the developer’s task becomes more than writing code and tests, the picture changes a little bit. For example, at OHB, the C code they write has to conform to architectural standards governed by ESA, and the developers have to document their conformance. At least some of the developers there are willing to use (and build) their own DSL to automatically generate all the stuff required from them beyond code. I can report a similar example from the healthcare domain, where we built a DSL that would eventually be used by healthcare domain experts. But initially it was also used by developers, sometimes pair-programming with the doctors, and apparently they liked it as well. Again, healthcare regulations require lots of derived artifacts, metrics and documentation, and the benefits of a DSL become more obvious. There are also examples for DSLs in the automotive domain, for describing interfaces, state machines and contracts. But as I have said above, these languages and their generators play only a relatively small role in the overall development activities of the engineers.

There’s another example of successful use of DSLs for developers: language engineering! Many language workbenches. including MPS, are effectively collections of DSLs for language definition, and the language engineers happily use them (the counterexample is Rascal which relies more on libraries). But language engineers are kinda biased regarding DSLs and might not suffer from these cultural issues I mentioned above.

Wrap up

So where does that leave us? Building your own DSL is a harder sell when you target users whose primary skill is software engineering, for technical and cultural reasons. In addition, there’s usually not as much leverage compared to building one for domain experts. That’s why I emphasize DSLs for non-programmers.

Acknowledgements: I want to thank Tamas Szabo, Sergej Koszejev and Klaus Birken for feedback on the text that helped me sharpen my thoughts and improve the text.