data-import
data-import copied to clipboard
refactor lookup tables
Now both definitions (simple and script) need the lookup tables. We need to find a better internal solution.
I'm finally continuing the development. I took a look at the branch with the partial import #49 and I'm not happy with the current state. The library need some refactoring to make partial import possible. Also the size of the branch is already too big to keep an overview. I decided to make small step and to begin with this issue here.
The lookup tables are currently too tightly coupled with the definitions. I'd like to make this a separate concept. Essentially the lookup table is a set of dictionaries. I like the name dictionary
in our context, because we want to lookup an value and get its meaning.
Also the DSL for the id lookup is not that great. definition(...).lookup_for(...)
, definition(...).row_imported(...)
and definition(...).identify_by(...)
is :shit:
I thought of two solutions to make the DSL more meaningful:
- Since the lookup is tied to the definition anymore, the
definition(...)
part doesn't make sense anymore. Instead the execution context provides something likelookup(...)
orid_dictionary
which returns a dictionary, which then provides methods likefind
,add
. - Or we could make the methods to update the dictionaries directly available in the DSL like
find_mapped_id
andadd_mapped_id
.
I prefer the first solution. If we provide a method for each use case in the execution context, its API will eventually explode. Also I find it difficult to find meaningful method names for the second solution.
@senny what do you think about it?
@stmichael I currently have limited time and can't dig back into the source of this lib.