RFC: no-parsing `@components`

[cstjean]

This PR rewrites the @components parsing logic. Instead of parsing the RHS of each equation in the @components block, it evaluates it normally as plain julia code. Because the new code is a lot simpler, it was relatively easy to improve the flexibility of the model construction interface (see below) along the lines of https://github.com/SciML/ModelingToolkit.jl/issues/3791

There's still a lot of work to do, so I'd like to get some feedback before pushing forward (... or giving up). Is that a reasonable direction to take?

The good

• Net reduction in model-parsing code
• Can now change structural parameters with __ syntax: @mtkcompile my_model = MyModel(sub_component__N=10)
• Can now change subcomponents with __ syntax: @mtkcompile my_model = MyModel(sub_component__subsub_component=MyAlternativeComponent()). I believe this fixes #2038.
• No parsing on the RHS of the components means that the RHS can be arbitrary Julia code, which is a lot more intuitive and general. For example,

  my_interpolation = LinearInterpolation(df.data, df.time)
  @mtkmodel MassSpringDamperSystem2 begin
      @components begin
          src = Interpolation(itp=my_interpolation)
  

  can now be

  @mtkmodel MassSpringDamperSystem2 begin
      @components begin
          src = Interpolation(LinearInterpolation, df.data, df.time)
  

• This also supports heterogeneous arrays, as in resistors = [FixedResistor(), VariableResistor()].
• Nested if blocks now work inside of @components

The bad

Model metadata for components @test A.structure[:components] == [[:cc, :C]] now yields [[:cc, :unimplemented]]. (But System metadata is fine). How bad is that? Where is that information used? Maybe Dyad?

Potential workaround: bring back #master's parsing code just for metadata parsing. Maybe if I strip that code of everything except the metadata part, it would be reasonably small, and we can give up on more complex cases (that aren't supported ATM anyway)?

The ugly

To support plant__component__subcomponent => AlternativeComponent(), I used ScopedValues to thread those parameters. I believe that it's broadly fine, but there might be edge cases I haven't thought about.

TODO

If this PR looks appealing, then the remaining items would be

• [ ] Clean up, comment
• [ ] Conform to the style guide
• [ ] Validate kwarg names (at the moment misspelled constructor kwargs are ignored)
• [ ] Squash, or redo the commit history
• [x] Figure out a solution for the commented out tests (from https://github.com/SciML/ModelingToolkit.jl/pull/3995)
• [ ] Test against ModelingToolkitStandardLibrary
• [ ] More tests

Let me know what you think! If this is too disruptive, I can tackle another direction. Most of these features can be implemented without getting rid of the RHS parsing code, but it's a lot more work.

[cstjean]

I may have some understanding of the current test failure. It seems that, with

@mtkmodel SimpleLag begin
    @structural_parameters begin
        K # Gain
    end
end

@mtkmodel DoubleLag begin
    @parameters begin
        K1, [description="Proportional gain 1"]
    end
    @components begin
        lag1 = SimpleLag(K = K1)
    end
end

When passing the parameter K1 to SimpleLag as a structural parameter, K1's metadata gains a SymScope => ParentScope(LocalScope()) entry. Is that correct? I'd appreciate a pointer to the logic that achieves that; it's not in model_parsing.jl AFAICT.

On this branch, SimpleLag's K is the same object as DoubleLag's K1, so it does not have a SymScope entry. This is perhaps why it errors with double_lag₊lag1₊K1 is present in the system but double_lag₊lag1₊K1 is not an unknown.

[cstjean]

I figured it out (@named uses default_to_parentscope) and fixed it in https://github.com/SciML/ModelingToolkit.jl/pull/4029/commits/4086714d2cdae37f2f693d2af46d2592dfe30c5a

[AayushSabharwal]

This is... interesting. Given that @mtkmodel is being deprecated in v11, I don't mind this as long as it passes tests. It might be worth holding off a bit, I'll try and get v11 out ASAP and this can be retargeted there. The parsing code hasn't changed in v11, we're just not maintaining the @mtkmodel syntax.

[ChrisRackauckas]

We could spawn it off to a separate subpackage. I don't think anyone really plans to maintain it from the core SciML group but it could stay alive in that way, just we shouldn't use it in the docs.

[cstjean]

This is... interesting. Given that @mtkmodel is being deprecated in v11, I don't mind this as long as it passes tests. It might be worth holding off a bit, I'll try and get v11 out ASAP and this can be retargeted there.

Thank you for the heads up! I wasn't aware, but I found the discourse thread. The decision isn't surprising, but it's a bummer for us, as @mtkmodel is great for communicating with non-coding engineers. I appreciate the "soft deprecation" part.

V11 is breaking, presumably? Would it be a problem to take the opportunity to give up on Model's subcomponent metadata A.structure[:components] == [[:cc, :C]]? Having to support that negates some of the benefits of this PR, and I couldn't find where it's used. Anyway, @component won't support anything like that, I assume?

[AayushSabharwal]

V11 is breaking, presumably?

Yes, partially to support Symbolics@7 and partially to de-complicate a lot of the semantics surrounding parameters and initialization.

Would it be a problem to take the opportunity to give up on Model's subcomponent metadata A.structure[:components] == [[:cc, :C]]?

I think we can get rid of a lot of that metadata.

Anyway, @component won't support anything like that, I assume?

No, it's fairly barebones. But as in the discourse thread, there's ways to make it more useful.

@mtkmodel won't be part of MTK in v11, but will be split out into a separate package along with tests. It will still live in this same repo, and I'm more than happy to review PRs to it. However, it won't receive more development from the core maintainers. I agree that a lot of it can be made simpler (i.e. not re-doing @variables parsing) and it can definitely live on as a community project.

[cstjean]

@mtkmodel won't be part of MTK in v11, but will be split out into a separate package along with tests. It will still live in this same repo, and I'm more than happy to review PRs to it. However, it won't receive more development from the core maintainers. I agree that a lot of it can be made simpler (i.e. not re-doing @variables parsing) and it can definitely live on as a community project.

Thank you for the clear decision. I'm guessing that its tests will pass at 11.0 release, but may break once MTK internals start changing?

I'm wondering how much non-MTK-core community support will come up. It's complex macro code. On our side, we care about having such a macro, but it's not clear whether I'd prefer to maintain the current macro code, or start over with a simpler "convenience macro" along the lines discussed in the discourse thread.

Maybe I'll start a discourse post to figure out a way forward and see what others think.

[cstjean]

It will still live in this same repo

Somehow I missed that part. This suggests that it'll still be part of the MTK test suite throughout v11? If so, that sounds great. I'll wait until v11 to retarget this PR and see if it can be made to pass all tests (minus model metadata), as you suggested.

Looking forward to it!

[AayushSabharwal]

This suggests that it'll still be part of the MTK test suite throughout v11?

Yes, it will.