ModelParameters.jl icon indicating copy to clipboard operation
ModelParameters.jl copied to clipboard

Is there a reason: no method matching Param(::Vector{String})

Open alex-s-gardner opened this issue 1 year ago • 20 comments

Another awesome package, thanks!

In this example:

using ModelParameters

Base.@kwdef struct Project{A}
    update_geotile_missions::A = Param(["icesat", "icesat2"],)
end

model = Model(Project(), )

The following error is returned:

ERROR: MethodError: no method matching Param(::Vector{String})

Closest candidates are:
  Param(; kwargs...)
   @ ModelParameters ~/.julia/packages/ModelParameters/Lckhh/src/param.jl:135
  Param(::NT) where NT<:NamedTuple
   @ ModelParameters ~/.julia/packages/ModelParameters/Lckhh/src/param.jl:103
  Param(::Number; kwargs...)
   @ ModelParameters ~/.julia/packages/ModelParameters/Lckhh/src/param.jl:139
  ...

Stacktrace:
 [1] Project()
   @ Main ~/Documents/GitHub/Altim.jl/src/junk1.jl:5
 [2] top-level scope
   @ ~/Documents/GitHub/Altim.jl/src/junk1.jl:10

my expectation was that model Parameters were highly generic, allowing for a diverse Parameter types. Is it on the user to implement methods for things like vectors of strings?

alex-s-gardner avatar Aug 21 '24 20:08 alex-s-gardner

Could I ask what the intent here is?

Did you want a parameter that is specifically that string vector, or did you want a parameter with two options?

ConnectedSystems avatar Aug 21 '24 20:08 ConnectedSystems

I need the parameter to be an iterable of strings.

alex-s-gardner avatar Aug 21 '24 21:08 alex-s-gardner

I may not be fully understanding the use case, but if the vector doesn't change, do you need to wrap it in a Param?

That said, I agree in principle that this is something Param could do - the string could be a value sampled from a list of categoricals for example; a CategoricalParam if you will. I actually need something similar myself, eventually.

ConnectedSystems avatar Aug 21 '24 21:08 ConnectedSystems

Maybe I'm using ModelParameters incorrectly... my intention is to store all of my model "parameters" and "options" in ModelParameters objects so that I have all of my specific model run information in a single object with associated metadata.

Is this the intended use case?

alex-s-gardner avatar Aug 21 '24 21:08 alex-s-gardner

Technically you could (for real-valued parameters at least), but it's intended more for parameters that are to be sampled from something.

If the parameter doesn't change then it's constant for the purpose of your model.

A quick workaround is to give the Param a 0 value and map that to the actual value you want to use with a Dictionary...

ConnectedSystems avatar Aug 21 '24 22:08 ConnectedSystems

Yeah, its not really worth being a parameter unless you or some optimiser will want to change it.

It's not really a package to define and manage all you models inputs - just to let you update and change Real and Number parameters once they are in the model structure.

rafaqz avatar Aug 22 '24 00:08 rafaqz

I guess a Dict() is the way to go, or do you have any recommendations?

alex-s-gardner avatar Aug 22 '24 00:08 alex-s-gardner

Maybe I don't really get what you're trying to do

rafaqz avatar Aug 22 '24 11:08 rafaqz

@rafaqz the intent is to attach metadata to model parameters while also making them composable via Model().

ConnectedSystems avatar Aug 22 '24 11:08 ConnectedSystems

Oh, if that metadata relates to specific model parameters you can add arbitrary fields to them...

Like Param(x; mymetadata= ["icesat", "icesat2"]) and :mymetadata will be a column in the Model table.

rafaqz avatar Aug 22 '24 11:08 rafaqz

Yes, but it'll be annoying to check both val and a custom field/column. Could we do a non-real valued Param subtype that just holds a constant?

There's some use cases where this would be handy for me as well

ConnectedSystems avatar Aug 22 '24 11:08 ConnectedSystems

Yes probably we just need that. Maybe AnyParam? It should be pretty easy to implement.

Maybe it would have been better to never try making Param a Number and just rely on stripparams always being run.

(Although I guess we want to be able to easily just get the Param objects and not AnyParam... may get complicated Idk

rafaqz avatar Aug 22 '24 11:08 rafaqz

I guess what I'm looking for is a @kwdef struct that has a convention for metadata (and other associated properties like bounding limits), is composable, and displays as a table. This would make understanding complex function inputs easier and the user could easily show what the state of inputs is in the repel or in print logs. I approached ModelParameters.jl thinking that this was its purpose but I was incorrect... it does seem however that ModelParameters.j supports everything I was looking for except the ability to define arbitrary static parameters.

alex-s-gardner avatar Aug 22 '24 17:08 alex-s-gardner

Okay, I'll try to make time to implement for @rafaqz 's review. It may be a short while though.

ConnectedSystems avatar Aug 23 '24 00:08 ConnectedSystems

I don't totally get it, doesn't this work for you?

@kwdef struct MyStruct
    a = Param(defaultval; prior=Normal(0, 1), bounds=(1, 2))
    b = Param(defaultval; prior=LogNormal(1, 2), bounds=(2, 3))
end

There is also FieldMetadata.jl if you want macro style, but it doesn't have Tables.jl interop

rafaqz avatar Aug 23 '24 10:08 rafaqz

As far as I can tell defaultval in your example must be numeric.

Is there a reason that defaultval should be restricted to numeric data? Could a method be included to accept any defaultval, like a tuple of strings or array of numbers?

alex-s-gardner avatar Aug 23 '24 16:08 alex-s-gardner

The reason is Param <: Number, which means you model will still run with Param embedded in it, and pass any type limitations on fields that need Number, or Real with RealParam.

If Param allows other values, the values Tuple/Vector you get from the Model will not be useful in an optimiser or a bayesian package, and will make your params vector type unstable. So it kind of defeatsone of the main purposes of the package.

Like I mention above adding AnyParam could be used for this and you could filter only Param or RealParam somehow to just get the numerical values. Then we get best of both. Someone just has to write it.

Also, FieldMetadata.jl (which I don't really use any more but other people do) is more flexible and you can put whatever you want wherever. But you don't get a Tables.jl compatible object, although its actually easy to make one in a few lines of code using metadata flattening in Flatten.jl.

rafaqz avatar Aug 23 '24 17:08 rafaqz

OK, I think everyone is in agreement.

  1. OP use case is not the intended use case for ModelParameters
  2. modifying Param to accept Any as it's value would defeat the intended purpose of ModelParameters
  3. ModelParameters has many of the features desired by the OP and could be "easily" modified to accommodate the requested features by adding an AnyParam type and methods

I reviewed the RealParam and ArrayParam PR, which I think is a template for adding AnyParam. This is over my head so I will not attempt a PR.

@ConnectedSystems has offered to make PR when they find the time.

alex-s-gardner avatar Aug 23 '24 18:08 alex-s-gardner

Its really not over your head ;)

rafaqz avatar Aug 23 '24 18:08 rafaqz

Yeah this should really just be copy and pasting some methods and types.

Fixing type instability issues with Flatten.jl on the other hand... :exploding_head:

bgroenks96 avatar Aug 28 '24 08:08 bgroenks96