gempy icon indicating copy to clipboard operation
gempy copied to clipboard

decouple computation from evaluation, use fit and transform scikit learn syntax

Open AndrewAnnex opened this issue 5 years ago • 0 comments

Is your feature request related to a problem? Please describe. Currently, it is difficult to cleanly use high resolution grids and dense input data simultaneously in gempy. It seems the current paradigm is to fit the model and evaluate the resulting regular grid/topography/sections simultaneously. However, it could be more memory efficient if model computation was separated from the evaluation of the model on various grids/topography/sections/etc.

This assumes that the kriging weights/theano results are entirely independent of the grids upon which they are evaluated which should be a true statement from prior discussions.

A side benefit could be that huge input datasets(10k or more input points) can be evaluated with the CPU, taking however much RAM and time is needed, and the resulting weights could be used on the GPU to evaluate the model on many grids

Describe the solution you'd like

Transition to use the scikit learn syntax for model "fit()" and "transform()", so that the fitting accepts input data and does not copy the surface point data needlessly to theano (gpu memory) when evaluating the model on grids/topo/etc with the "transform" operation.

This may call for a new semi-functional flavor of the gempy api, whereby the user can have a bit more control. psuedo code:

geo_data = gp.init_data(path_o, path_i)
geo_model = gp.create_model(any theano args from set_interpolation_data)
geo_model.fit_geology(geo_data) # replaces compute_model
regular_grid_result = geo_model.transform(some_grid)
section_result = geo_model.transform(some_section)

Describe alternatives you've considered I've been playing around with using the API as is, but the nesting of the objects many of which are wrapping dataframes make direct manipulation does not necessarily propogate changes everywhere needed (len_series_i, etc)

I haven't tried exporting the model weights and etc yet to just define a new model object, not sure if tht would work either.

AndrewAnnex avatar Jan 17 '20 00:01 AndrewAnnex