nested lists

[uatach]

trafficstars

I'm dealing with some nested lists and would like to do something like:

>>> target = {
...        'f1': 'v',
...        'f2': [{
...            'f3': 'a',
...            'f4': 0,
...            'f5': [{
...                'f6': 1,
...                'f7': 2, ...
...            }, ...], ...
...        }, ...], ...
...    }
>>> glom(target, ...)
[{'f1': 'v', 'f2.f3': 'a', ..., 'f2.f5.f6': 1},
 {'f1': 'v', 'f2.f4': 0, ..., 'f2.f5.f6': 1},
 {'f1': 'v', 'f2.f3': 'a', ..., 'f2.f5.f7': 2}, ...]

I can get to the list of 'f2.f5.f6' kind of fields but how do I merge this list with parent values? Is this even possible?

[wdieter]

I think we have the same issue https://github.com/mahmoud/glom/issues/27 I'm able to access those items but they are returned as nested lists. In your case:

In [21]: spec = {'f2.f3': ("f2", ['f3']), 'f2.f5.f7': ('f2', ['f5'], [['f7']])}

In [22]: glom.glom(target, spec)
Out[22]: {'f2.f3': ['a'], 'f2.f5.f7': [[2]]}

but naturally {'f2.f3': 'a', 'f2.f5.f7': 2} would be better. Thats what I'm struggling with.

[uatach]

@wdieter thanks, I've managed something due to the comments on #27. Still, it doesn't seem quite right:

target = {
    'f1': 'v',
    'f2': [{
        'f31': 'a',
        'f41': 0,
        'f51': [{
            'f61': 10,
            'f71': 20,
        },],
    },{
        'f32': 'a',
        'f42': 1,
        'f52': [{
            'f62': 11,
            'f72': 21,
        },],
    },
    ],
}

glom({'data': target}, ('data', lambda d: [{'f1': d['f1'], 'f2': v} for v in d['f2']]))
#[{'f1': 'v', 'f2': {'f31': 'a', 'f41': 0, 'f51': [{'f61': 10, 'f71': 20}]}},
# {'f1': 'v', 'f2': {'f32': 'a', 'f42': 1, 'f52': [{'f62': 11, 'f72': 21}]}}]

It looks like I'll have to keep unpacking things at each step... @mahmoud @kurtbrose is there a better way?

[liquidgenius]

I have this exact question as well.

[pylang]

I would hope to have a string spec as follows to access nested lists:

target = {
    'f1': 'v',
    'f2': [{
        'f51': [{
            'f61': 10,
            'f71': 20,
        },],
    },...
...
}

glom(target, "f2[0].f51[0].f61")
# 10

Here you can quickly see when you're drilling into a dict key or list index. One tool that does spelunks nested containers in a similar way is Munch, a py 3 variant of a Bunch. Can we emulate that access style in parallel with the spec?

[mahmoud]

@pylang that's interesting. I think it's a little different than OP's ask. Regarding that syntax, I'll note that dict keys are also accessed with square brackets, so I'm not sure that you're accessing a list is universally obvious, not to mention that glom also traverses attribute accesses with the same . syntax in the string.

You can get list indexes in your example with glom(target, "f2.0.f51.0.f61")

For more explicit access, I'd say take a look at the T object.

[mahmoud]

For the rest of folks looking for upward merging, we're working on some features that'll help!

[pylang]

glom(target, "f2.0.f51.0.f61") does the job. Thanks.