nushell
update column where that column is not missing
Related problem
I found myself wanting to update an optional column of a table, changing the value if it is present and leaving it empty when it is absent, similar to how one might map over an optional value in a functional language. However, I've found that update complains when given a column that is not present for all rows, and if I default that column to null or use upsert, I'm not aware of a function to map over that possibly-null value. As a result, I've ended up doing ... | upsert col {|r|if $r.col? == null {null} else {$r.col | foo}} which is awfully verbose and not especially clear
Describe the solution you'd like
I would like a flag on update, to allow it to skip rows that are missing the chosen column, e.g.
... | update col --skip-missing {|r| $r.col | foo}
Describe alternatives you've considered
This might be easier if I had a way to map over an optional value (like a conceptual opposite of default), but I am not aware of any.
this way I could do:
... | upsert col {|r|$r.col? | map {foo}}
Additional context and details
No response
Do you have any sample data to play around with that demonstrates the problem?
> let data = [{name: zoey github: zoybean} {name: mitch twitter:mmmmm}]
> $data
─#─┬─name──┬─github──┬─twitter─
0 │ zoey │ zoybean │ ❎
1 │ mitch │ ❎ │ mmmmm
─#─┴─name──┴─github──┴─twitter─
ideal output:
> $data | update --skip-missing github {|r| $"https://github.com/($r.github)"}
─#─┬─name──┬───────────github───────────┬─twitter─
0 │ zoey │ https://github.com/zoybean │ ❎
1 │ mitch │ ❎ │ mmmmm
─#─┴─name──┴───────────github───────────┴─twitter─
"simple" hack, using upsert (replaces missing with null):
> $data | upsert github {|r| if ($r.github? == null) {null} else {$"https://github.com/($r.github)"}}
─#─┬─name──┬───────────github───────────┬─twitter─
0 │ zoey │ https://github.com/zoybean │ ❎
1 │ mitch │ │ mmmmm
─#─┴─name──┴───────────github───────────┴─twitter─
complex hack, redefining update myself:
> $data | each {|r| if ($r.github? == null) {$r} else {$r | update github $"https://github.com/($r.github)"}}
─#─┬─name──┬───────────github───────────┬─twitter─
0 │ zoey │ https://github.com/zoybean │ ❎
1 │ mitch │ ❎ │ mmmmm
─#─┴─name──┴───────────github───────────┴─twitter─
this leads me to this definition of update --skip-missing:
def "update --skip-missing" [path: cell-path f:any] {
each {|r| if (($r | get --ignore-errors $path) == null) {$r} else {$r | update $path $f}}
}
But this would always skip both null and missing, which might not be desired.
It could be helpful if is-empty would detect missing as well. But --skip-missing may be helpful too.