Add yaml-based Model loader?

[sdrave]

I would like to retire discretizers.disk and replace it by a generic config file format. Something like

type: StationaryModel
operator:
    type: LincombOperator
    operators:
        - L1.mat
        - L2.mat
    coefficients:
        - diffusion[0]
        - diffusion[1]
rhs: F1.mat
output_functional: C.mat
products:
    l2: l2.mat
    h1: h1.mat
parameters:
    diffusion:
        shape: 2
        range: [0.1, 10]

I haven't thought about the details yet, but the general idea would be to have a generic file format, which can be used to load arbitrary Models from disk. Any comments, @pymor/pymor-devs? I think one might also argue that the same could be achieved with few lines of Python code and that we actually do not need such functionality.

If nobody disagrees, I would start by moving discretizers.disk to the graveyard.

[pmli]

I guess this would be especially convenient for parametric models. I'm wondering how multiple matrices contained in a single Matlab-style .mat file would be handled.

[renefritze]

I think one might also argue that the same could be achieved with few lines of Python code and that we actually do not need such functionality.

There certainly are potential pymor application users who would rather fill in a config file template than write the equivalent 5 lines of code. So I'd say this functionality is not a strong "need", but a nice to have.

[MagnusOstertag]

Hi,

I think adopting this feature is pretty useful for a more widespread use of pyMOR and I'd like to contribute here. I'm pretty new to pyMOR having read mainly the documentation, but I'm willing to put in a few days work here.

So to be on the same page here: the loader should be able to read and construct a subclass of pymor.models from a description of the model and its parameters in a .yml file. The parameters keyword further describing an object of pymor.parameters which can be referenced in the parameters of the class. Where in the package do you like the method? Should I also write tests e.g. with the models from the tutorials? Do you think there should also be a write to file functionality for the models?

Maybe you can give me some feedback and pointers here. If you don't think this is doable in my time budget or think another issue might work better please also let me know...

[renefritze]

I think adopting this feature is pretty useful for a more widespread use of pyMOR and I'd like to contribute here. I'm pretty new to pyMOR having read mainly the documentation, but I'm willing to put in a few days work here.

Awesome! If you haven't already, maybe skim the developer docs.

So to be on the same page here: the loader should be able to read and construct a subclass of pymor.models from a description of the model and its parameters in a .yml file.

Yes. Where the type setting determines the subclass.

The parameters keyword further describing an object of pymor.parameters which can be referenced in the parameters of the class.

Yes on the reflecting a pymor.parameters.Parameters object. Not sure what you mean by the can be referenced bit.

Where in the package do you like the method?

I'd start with a new module pymor.models.loader. The name isn't really important at first anyhow.

Should I also write tests e.g. with the models from the tutorials?

That would indeed be a very good idea. If you write tests that need to load data, have a look at pymortests.fixtures.operator. We're using pytest-datadir there to transparently determine a location for a test(-fixture) to load data from. This is much more robust than trying to figure out a safe save location manually.

Do you think there should also be a write to file functionality for the models?

I'd say nice to have, but not necessary. Could also come later. It would be a useful tool for testing however. Checking whether a model can round trip through solve->save->load->solve and still produce the same results could be a great integration test.

If you don't think this is doable in my time budget or think another issue might work better please also let me know...

This is somewhat hard to say from my end, since it greatly depends how familiar you are with python, YAML, testing and so forth. However, I don't think you should worry too much about whether you can finish everything we may want from this functionality within your timeframe. If you feel up for it, just get started. If you want to learn more about the stuff needed to accomplish this, just get started. Even if you just get some foundation going and have to stop, we'll still welcome that contribution and it'll be much less work for somebody else to pick up from there than start from zero.

[MagnusOstertag]

I have started a pseudocod-esk draft pull request, as maybe s.o. can give me some general feedback whether I'm on the right track here. Especially, I'd like some feedback whether the general form of the .yml file is correct:

type: <subclass of pymor.models>
<operator>
parameters:
  <arguments>

With <arguments> being one or many statements of the form
<argument>:
  <value>
OR
<argument>:
  - <value>
  - <value>
  - ...

With <operator> being one or many statements of the form
operator:
  type: <subclass of pymor.operators>
  <operator>
  <coefficients>
OR
<name>:
  <matrix>
OR
<name>:
  - <matrix>
  - <matrix>