Functions to add progress bars to for loops and apply functions

[gaborcsardi]

E.g.

with_bar(
  for (index in sequence) block
)

and

lapply_with_bar(list, func)

Implementation is easy. The for loops need some non-standard evaluation.

[jefferis]

+1!

[gaborcsardi]

Thanks. I am not completely satisfied with the syntax and the lapply_with_bar name, but if I cannot come up with anything better, I'll go with these, soon.

[jefferis]

A few alternatives:

• lapply_progress
• plapply
• progressive_lapply
• lapply_bar

[gaborcsardi]

Alternative syntax ideas:

## This needs NSE to get `x`
progress > for (i in x) { }

## This knows `x` right away, no NSE needed
progress(x) > for (i in x) { }

## Similarly with *apply:
progress > lapply(x, function(i) { })

Maybe this is stupid, but how about

ooo == for (i in x) { }
ooo == lapply(x, function(i) {})

EDIT: this one also parses:

---O- for (i in x) { }
---O- lapply(x, function(i) {}

EDIT: with some hacks, even this seems possible:

for (i in progress(x)) { }
lapply(progress(x), function(i) { })

[gaborcsardi]

> will not work, because there seems to be no way to capture the unevaluated argument (except for completely redefining >, which I don't want).

So we need to go with sg clumsier, e.g.

progress %P% for (i in 1:10) { print(i) }
P%P% for (i in 1:10) { print(i) }

add %[=]% for (i in 1:10) { print(i) }

Or the magical := operator:

progress := for (i in 1:10) { print(i) }

But the %% operators are really clumsy and heavy, they attract attention. The := is lighter, but it is already exported by data.table.

So I guess it has to be %%, and I can override how it looks in the IDE. Actually exactly %% might not be too bad:

progress %% for (i in 1:10) { print(i) }
progress %% lapply(1:10, print)

[jefferis]

progress %% for (i in 1:10) { print(i) }
progress %% lapply(1:10, print)

These look good to me.

I haven't looked at the details at all, but would it be possible to use a pipe, perhaps like:

for (i in 1:10) %>% progress { print(i) }

# or reverse pipe

progress %<% for (i in 1:10) { print(i) }

[gaborcsardi]

for (i in 1:10) %>% progress { print(i) }

This is not valid R syntax, I am afraid. It does not parse.

progress %<% for (i in 1:10) { print(i) }

This is probably possible, but I would need to redefine %<%, which is not ideal.

[gaborcsardi]

Looks like %% does not work, actually. It has to be a conventional user defined %% operator then, e.g. %P% or %~~~%.

[jefferis]

Can you make a unary operator %progress% or %P% ?

%progress% for (i in 1:10) { print(i) }

[gaborcsardi]

That's not possible, unfortunately. I could use ? or !, but neither are very good.

[gaborcsardi]

Work is going on in the https://github.com/gaborcsardi/progress/tree/decorator branch now.

[gaborcsardi]

A relevant discussion: https://github.com/hadley/purrr/issues/149

I quite like the functional form:

progressify(lapply)(seq, fun)

and maybe this does not even require NSE. We just create a new function that calls both the original function and tick().

But this does not work for for loops. I am not sure how much of a problem that is. Maybe

for (i in progressify(seq)) { } 

could work. It requires some tricks, though. Maybe we can just wait until the iterator type for loop arrives to R, probably in 3.4.0.

[kendonB]

I assume y'all are aware of this: https://github.com/psolymos/pbapply ?

I am using it and it works great for apply functions. Still no solution for parallel functions or dplyr::mutate / purrr::map

[gaborcsardi]

@kendonB I have probably seen it, I am not sure to be honest.

[psolymos]

Just uploaded pbapply v1.3-0 to CRAN with parallel functionality.

[gaborcsardi]

Implemented in cli.