Do not completely synchronize read-only modules and models

[benedekh]

In the followings, the term "cloud" can be interchangeably used with "modelix model server".

Problem

Sometimes our users would like to refer to SNodes in their models, which are not part of their Project, but are coming from SModels in pre-installed plugins.

So far such referred SNodes were synched to the cloud with their full structure (properties, refereces, and their children recursively). Moreover, their parent SModel / SModules were also synched to the cloud completely (including all model imports, module dependencies, language dependencies, content, etc). Synching all those elements recursively to the cloud is an exhaustive task, especially if the transitive Module Dependencies are built in a way, that basically the whole application has to be synched.

As an example, imagine an MPS-based application, where the user just includes the root SModule in a DevKit (that are coming from pre-installed plugins) and this root SModule transitively includes all sub-modules the application is composed of. If the user puts a Module Dependency or a DevKit dependency for this SModule inside their Project, then all SModules will be synched to the cloud, which can be very exhaustive.

Solution

Since the aformentioned SNodes, SModels and SModules are coming from pre-installed plugins, they are read-only by default (thanks to MPS). It means, that since their structure cannot be changed, we do not have to replicate them in the cloud, but we could just save some "special references" (c.f. MPSNodeReference, MPSModelImportReference) for them which say that these elements are in MPS and we persist their MPS-based references in the cloud. It means, that when those elements are downloaded from the cloud, then we just resolve them locally in MPS, because we assume that they exist (since they are coming from the pre-installed plugins).

It simplifies the implementation a lot as we can see in the followings:

References for read-only SNodes

If an SNode 'A' refers to a read-only SNode 'B', then we create a serialized MPSNodeReference, which includes the MPS-based node reference (identifier). We put this serialized reference in a NodeReference, that is used as the Reference Target in the cloud.

When such reference arrives from the cloud, then we decide if the serialized reference refers to an INode (because INode references can be also serialized) or to an MPS-node. If it refers to an INode, then we fetch that node and transform it to an SNode (just like before). If it refers to an MPS-node, then we resolve that node locally in MPS. (For details see lines 94-111 in ModelixTreeChangeVisitor.)

Model imports for read-only SModels

If an SModel 'C' imports a read-only SModel 'D', then we create a serialized MPSModelImportReference, which includes the MPS-based model reference (identifier). We put this serialized reference in a NodeReference, that is used as the Reference Target in the cloud.

When such model import arrives from the cloud, then we decide if the serialized reference refers to an INode or to an SModel in MPS. If it refers to an INode, then we fetch that node and create a ResolvableModelImport from it (because the referred model might not have been transformed yet; just like before). If it refers to an SModel in MPS, then we resolve that SModel locally. For details see lines 163-186 in ModelTransformer.

Module dependencies for read-only SModules

If an SModule 'E' puts a dependency on a read-only SModule 'F', then not too much changes. Read-only SModules will not be transformed to INodes (see lines 149-154 in ModuleSynchronizer), but only the ModuleDependency will be created. The ModuleDependency has a UUID field which contains the UUID of the read-only SModule. There is an extra isReadOnly property in the ModuleDependency, that is set to true if the target SModule ('F') is read-only.

Note that the isReadOnly property is an extra, that is not part of the original ModuleDependency Concept definition. However, it does not cause any problems, even if someone uses the generated API for this Concept. That's because the generated API ignores those properties and references that are not part of the definition and can therefore transform the INode to TypedNode (N_ModuleDependency), even if it has some extra properties in its property store.

When such module dependency arrives from the cloud, then we check if its isReadOnly property is true. If it is true, then we do not try to transform the target module from INode to SModule, because we know that it must exist in MPS. Instead, we just simply create the ModuleDependency in MPS like before. For details, see lines 164-171 in ModuleTransformer (where we skip the target module transformation if the module dependency is read-only), and lines 213-220 in Module Transformer (where we create the Module Dependency in MPS, just like before).

Languages and DevKits

Languages and DevKits are not synched to the cloud, only the SimpleLanguageDependency and the DevKitDepepdency are created. Nothing changed here, even though Languages and DevKits are also read-only.

Read-only SModels and SModules

• The ModelSyncAction is disabled for read-only SModels, because we are not synching such models.
• The ModuleSyncAction is disabled for read-only SModules, because we are not synching such modules.
• The UnbindModelAction is disabled for read-only SModels , because we do not synch them, therefore no binding is created for them.
• The UnbindModuleAction is disabled for read-only SModules, because we do not synch them, therefore no binding is created for them.

Testing

I tested the code manually in the following way:

1. I synchronized an SModule to the cloud that had an SModel with several SNodes. One of the SNode had a reference to a read-only SNode.
2. I set this reference to null and then to the original read-only SNode to see if the changes are synched to the cloud as a deletion or MPSNodeReference creation operation.
3. I put a model import in the SModel to a read-only SModel that was installed in MPS (one of the mbeddr SModels) and checked if the model import was synched to the cloud as a MPSModelImportReference.
4. I removed the aforementioned model import and checked if it was removed from the cloud.
5. I put a module dependency in the SModule to a read-only SModule that was installed in MPS (one of the mbeddr SModules) and checked if the synched Module Dependency's isReadOnly property was set to true.
6. I removed the aforementioned Module Dependency and checked if it was removed from the cloud.

To test the change detection from the cloud (i.e. when something changes in the cloud, then this change must also appear locally), I implemented some pieces of kotlin code in a separate intelliJ, that could do the following actions separately, when they are run:

1. Set a specific node reference to null or to a serialized MPSNodeReference. (To simulate unsetting and setting a read-only SNode reference).
2. Add / remove a read-only model import in an SModel. (To simulate adding/removing read-only model imports).
3. Add / remove a read-only module dependency in an SModule. (To simulate adding/removing read-only module dependencies).

In a separate MPS, I had the aforementioned SModule, SModel opened which was my test example. In intelliJ, I ran the aforementioned actions separately (one-by-one) and in MPS I observed in debug mode if the changes were correctly executed in the ModelixTreeChangeVisitor (that is responsible for handling changes from modelix into MPS) and the effects appeared in MPS as expected. If there was some mismatch then I fixed the code an reran the manual test scenarios until no bugs remained.

During testing, I discovered some corner-cases that I forgot to fix earlier:

• It is a bad idea to use SNode, SModel, SModules as keys in Maps, because these objects are not stable and may change at runtime without any warning. One should use SNodeReference, SModelReference, SModuleReference instead. (8a84c5e, MODELIX-787)
• Some convenience refactoring to expose the SRepository in the ServiceLocator. This saved some code, because we do not have to go to the MPSProject anymore to get the SRepository. (ba8a6f3)
• I forgot to remove some elements from the cache, after the corresponding MPS elements were deleted. (dd4218b)
• I reclassified an error to a warning, because in some cases it notifies us more about a side-effect than about a problem. I.e. when we delete a parent node, then it might be that by the deletion of the parent the children are also implicitly deleted. So when the event about the deletion of the children come from modelix, then they do not exist anymore in MPS, so it's not a problem that we could not delete them. (4c82dd4)

Auxiliary information

• The merge request (MR) replaces Do not completely synchronize read-only modules and models #126, that was an earlier approach for the same problem. However, the new implementation is simpler and more robust.

[slisson]

🎉 This PR is included in version 0.9.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀