Tool Lock-in vs. Semantic Lock-in

There’s a lot of discussion about lock-in relative to vendors and tools. I am confronted with these discussions in the context of domain-specific languages, because they rely on tools like Jetbrains MPS or Eclipse Xtext. If you build and use DSLs, you are locked into these tools. Is that a risk that should be taken? What happens if the tool “goes away”, then it is no longer supported, and you are (more or less) forced to move on?

Let’s explore the two implementation options that are usually available when we implement subject-matter intensive systems.

The first one is a DSL. You analyze your domain and understand the core abstractions. You implement those as a DSL in MPS or Xtext. This means you explicitly capture the structure in a metamodel and then implement validation rules, scopes and the execution semantics with the means of type systems and generators or interpreters. Then you capture your subject matter as an instance of that DSL. In the case of MPS, stuff is stored as (more or less) unreadable XML files in git, and editing and processing the models requires the use of MPS.

The usual alternative starts similarly by analyzing the domain and identifying the core abstractions. But then, instead of implementing those formally as a metamodel/language in MPS, you encode them in Java (or any other GPL) as classes and other abstractions available in your language of choice. You try to keep technical concerns out of separate as a matter of good design. Finally, you implement the subject matter as source code, relying on those previously defined classes.

The discussion around lock-in stems from the assumption that you don’t need a specific tool in the Java-only approach. You can edit the code with any of several (ok, more like two :-)) IDEs, or even your favourite text editor. And the probability that the Java compiler or runtime (or the equivalents for other mainstream GPLs) are no longer supported, is very low — certainly much lower than that of MPS or Xtext. So far, the argument is sound of course.

But let’s look at the degree of lock-in into tools like Xtext or MPS. As we have said, the models are based on a language that is formally defined. We have access to the structure and the semantics of the model — because we ourselves have defined the language, so we know what stuff means! So if the tool goes away, it is true that the effort for language implementation is lost, we have to reimplement it in some other tool; there are no meaningful standards that allow for automatic exchange of language definitions between DSL tools. But if we use DSLs in context where they make sense, then a lot more effort has gone into the models (not the language definition). The bulk of the subject matter is therefore formally captured as an instance of your language and can always be exported in some readable format (XML, JSON, whatever) and imported in a successor tool. The subject matter is never locked into the tool! In addition, you can also do more things with the subject matter you have captured formally using the DSL. You can generate other artifacts that you had not expected when you made the DSL/Java decision.

Compare that to the alternative where the subject matter has been encoded in GPL code. Doing anything with that subject matter except compiling and then running it is very hard because you have usually lost a lot of the
domain semantics in the process of encoding into GPL code. Therefore, automatically deriving other artifacts is anywhere between very hard and impossible. This is ultimately the problem with legacy systems: the subject matter is “buried” in source code, and you have to spend a lot of effort to (manually or automatically) reverse engineer it out of that grave. Yes, we also have to dig a bit when the DSL tool goes away, but at least the structure and semantics of our “knowledge” is still fully recoverable.

So, while it is true that using tools like Xtext or MPS lead to a degree of tool lock-in (not 100%, because you can always export a conveniently machine-readable version of the subject matter!), encoding your subject matter into source code leads to semantic lock-in. I argue that the negative consequences of this semantic lock-in are in many cases more significant than those of the (degree of) tool lock-in.

Avoiding that semantic lock-in is a major reason for using DSLs and software development approaches like those proposed by the Subject Matter First manifesto.