Configurable organization/view of pools

[philrz]

A community user that's been helping with the beta testing had the following suggestion:

When Brim displays the pools it would be nice to be able to rearrange them in a different order.  Then I thought, a directory structure would be even better than that.  If I had a 100+ pools it would be nice to be able to organize it all by some kind of structure.  This would probably be helpful on the CLI as well. 

Just yesterday @jameskerr was showing off a "tree view" he's been working on that allows for organizing items in the app into hierarchies, add/delete/rename, etc., so being able to re-use that component was the first thing that came to mind when hearing this. But it seems there's multiple angles to consider, so here's a dump of some of my initial thoughts.

As implied in the inquiry, it seems like an open question: "Where does the organization start?" Right now the set of Pools is flat in a Zed Lake, so depending on how users approach it, indeed I can see the potential to accumulate enough Pools that it becomes a challenge to browse them at both the CLI and app layers. This might make the case for allowing hierarchical organization at the Zed Lake layer and having that order be reflected "read only" in all clients. This would seem to imply an admin-defined organization of pools that's presumed to meet the needs of all users.

OTOH, users might appreciate the flexibility of being able to create their own personal view of how the pools are organized. What comes to mind is the "Quick Access" panel in Windows file explorer. This effectively consists of "pins" that point to commonly-accessed items in the large, hierarchically-organized filesystem. When browsing the filesystem, the user can right-click on any item and select "Pin to Quick Access". The entries already in Quick Access can be re-ordered or removed to suite the user's preferences. It looks like they can't be organized into a hierarchy, but we could still do it in ours if we wished. Of course, it may be seen as a drawback that every user is now responsible for maintaining this themselves.

I suppose we could get fancy and offer this at both layers. Once again, borrowing from the Windows example, there could be pool hierarchy/order defined within the Zed Lake itself, and if the pool count and admin-defined organization is tidy enough that it satisfies all users, they'd be free to stick with just this approach. Then if they outgrow it, something more visualization-layer like "Quick Access" could be made available to offer another layer of user-specific organization.

[jameskerr]

Here's another idea.

The sketch app I use will allow nesting layers by adding a "/" in the name.

For example, a flat list of pools might be

devs / james' pool
devs / mason's pool
admins / phil's pool

The app could then split on the forward-slash and create hierarchy from there.

[philrz]

Nice idea @jameskerr! Indeed, when I was discussing this with @mccanne this morning he thought of something similar, i.e., maybe keeping the Pools flat at the Zed Lake layer, but allowing slashes like that which could act as "hints" for higher layers that want to use them to represent hierarchy.

[mccanne]

This well help with scaling pools for example in the ETL project where there are thousands of tables and a pool for each table...

[mccanne]

Lets look at this for December.

[philrz]

Another community member just asked about this again in a new Slack thread.

is it reasonable to have a sort of visual grouping function for pools? just something that makes it easier to visually organize pools when you've got alot of them

[philrz]

I'm reopening this one. While it seemed like it was testing ok in the branch for #2572, when I went to verify the functionality post-merge at Brim commit 9759d70, I get the following error dump in the left panel on my first attempt to rename a pool to contain a slash.

Error: Data must contain an 'id' property or props.idAccessor must return a string
    at $5c74fef433be2b0a$export$e2da3477247342d1.accessId (/Users/phil/work/brim/node_modules/react-arborist/dist/index.js:1592:24)
    at visitSelfAndChildren (/Users/phil/work/brim/node_modules/react-arborist/dist/index.js:508:25)
    at /Users/phil/work/brim/node_modules/react-arborist/dist/index.js:540:16
    at Array.map (<anonymous>)
    at $0d7f39915c1a8ae9$export$882461b6382ed46c (/Users/phil/work/brim/node_modules/react-arborist/dist/index.js:539:26)
    at $5c74fef433be2b0a$export$e2da3477247342d1.update (/Users/phil/work/brim/node_modules/react-arborist/dist/index.js:1550:67)
    at /Users/phil/work/brim/node_modules/react-arborist/dist/index.js:2049:13
    at updateMemo (/Users/phil/work/brim/node_modules/react-dom/cjs/react-dom.development.js:17078:19)
    at Object.useMemo (/Users/phil/work/brim/node_modules/react-dom/cjs/react-dom.development.js:17733:16)
    at useMemo (/Users/phil/work/brim/node_modules/react/cjs/react.development.js:1645:21)

See also the attached video.

https://user-images.githubusercontent.com/5934157/199799478-80a7165c-5e54-4b41-ac96-f4b47c7485b3.mp4

[philrz]

Third time's a charm. 😉 While the fix in #2581 addressed the error dump that was being caused by the leading slashes, thorough testing now reveals that it introduced a new problem. As shown in the attached video at commit bc72354, now when I rename multiple pools to have the same /-separated base, they're not getting grouped together. The prior commit 9759d70 didn't have this problem.

https://user-images.githubusercontent.com/5934157/199842220-881e0c0d-f4ba-4a96-83ef-ee4d248c9fa4.mp4

[philrz]

Verified in Brim commit 9194bc8.

The following video shows the feature working much as @jameskerr envisions it in his first comment above. Pools remain flat at the Zed layer, but a user preference allows for a character that's interpreted as a separator for presenting the pools in a hierarchy. This character defaults to \, and if the setting is cleared entirely, the pools are shown in their true flat form. Right-click options like Rename and Delete performed on the upper-level groupings affect all the pools below them.

https://user-images.githubusercontent.com/5934157/200182152-944a60bf-d63e-48a6-9272-bd4d87c9f137.mp4

At least one user in the comments of this issue did note that hierarchy at the Zed layer itself might also be helpful, so we'll see how users react to what we've done thus far and revisit the subject if there's significant feedback that handling this exclusively at the presentation later is not enough.

Thanks @jameskerr!