I am interested in learning
      whether there is het another "meta"-level on top of that :-)
Good, keep us posted! We happen to be doing the same thing in house the
last week(s)... More difficult than I thought at the beginning to get it
"right" _and_ "reusable" in more contexts than just this one of controller
management.

So, are you also implementing something similar?.

yes! Azamat is taking the lead in this effort, and he showed me some
prototype. It is not yet clear when something workable can be released.

Are there any additional valuable lessons that can be shared?.

I don't know what you call 'valuable' but the main reason that started
Azamat's work is to get a computational support for "tree-structured
algorithms with multiple sweeps", such as:
- Model-Predictive Control
- Bayesian networks, Markov random fiels, Factor graphs
- control diagrams
- tree-structured dynamical systems
The common thing behind all these use cases are the things I mentioned:
separating the schedule from the model and the execution; generate code
from a domain-independent 'template implementation' with domain-dependent
plug-in functions; (immutable) data management; deployment onto various
software "containers".