Feature request / suggestion

[infinisean]

Juan,

First, thank you for releasing "Unbalanced". I just started using it and it is a wonderfully useful app and seems very well written. so thank you :)

After using it for a bit, trying to solve a problem (one I admittedly caused for myself, mostly by being impatient), I noticed Unbalanced does almost exactly what I was looking for, but not exactly (unless it does actually do what I need, and I am just not understanding how to make it happen, which is entirely possible too)....

My situation is that I had incorrectly set the directory split depth when I configured my "media" share, so now my movie folders are all split across multiple drives, i.e. the movie file can be on disk1, subtitles on disk2, additional "art" on disk3, etc. (rookie mistake, I know).

It seems like Unbalanced could fairly easily be made to correct this type of issue for all the other bozos like me out there, as it has almost all of the logic in place already.

The only thing to add would be to be able to select multiple folders (think movie folders, in my case), and have a target option of "align to primary drive" or something, such that in the example above, where the largest portion of the data in folder MOVIEABC is the movie file itself $MOVIE (say 5 GB or so), and the other ancillary files like art, subtitles, etc. comprise a very small portion (say 1-2 MB) of the total folder data.... so the mover would simply need to move all of the smaller files to the drive which contains the largest file. That seems like it would be fairly simple to do, from a logic perspective (i.e. calculating post-move data usage, per folder/drive), and also very light on actual data that needs to be moved to successfully re-align the data such that each folder exists on only one drive.

I think Unbalanced can already do this on a single-source-folder basis, but not on many source folders at once (please correct me if I am wrong on that).
If that is the case, would it be possible to incorporate the "many source folders at once" feature as I described above? I think that would add a great deal of value and usability to an already outstanding app, for all unraid users.

Please let me know your thoughts.... I'd be glad to help with beta testing such a feature, if you'd be interested in incorporating it.

Either way, thanks again for the great app!

--Sean

[jbrodriguez]

hi, thanks for the kudos !!

yes i think that for your use case, you would need to do a gather and select each "movie" folder individually towards the top drive

the logic doesn't seem too complicated, however it needs some considerations since, i think, it would involve re-evaluating free space after transfers

although i don't have bandwidth at the moment, i'm thinking about some pluggable algorithms that can be applied during calculations (not happening soon though 🙂)

[infinisean]

No worries, I totally understand the lack of bandwidth.... I'm slammed lately too. And While I don't have the same skill you clearly do, or the existing foundation of a working app to build on, maybe I can help contribute to make it slightly easier for you (whenever you do have time, that is) by brainstorming the logic/algorithms with you so all you have left to do is implement the logic in code... If that will help at all, I'll share what I've been rolling around in my head all day and you can feel free to use it, if you like, or not (won't hurt my feelings if you don't, but if you do, it would make me feel like I added something to a cool project :) Also, since I still need to solve my own problem, I'd be very interested in your feedback on my rough idea of a solution, as it might help me avoid some potential problems as I work on my own issue.

So, If you're still reading, here goes.... I think it might actually be easier than it might seem at first... (if I am way off base, or over simplifying things like a toddler, please be gentle... lol)

Since we can see the disk space used vs. free up front for all drives and folders, and the size of each file in all the selected source folders, I think the space calculations can be done all at once at the outset before a single file is moved. Which would allow you to show what the final space used/free would look like oin each drive after all the moves are complete, before the user pulls the trigger, like a "dry run".

For example: (using fake, round storage sizes for simplicity)

"Folder1 has BigAssMovie1=1500M on DriveA, SmallSubtitle2=2M on DriveB, SmallArt3=1M on DriveC" "Folder2 has BigAssMovie2=2000M on DriveC, SmallSubTitle3=3M on DriveA, SmallArt1=2M on DriveB" "Folder3 has BigAssMovie3=4000M on DriveB, SmallSubTitle1=4M on DriveC, SmallArt2=3M on DriveA"

So, current usage: DriveA: 1500 (BAM1) + 3 (SS3) + 3 (SA2) = 1506M Used of 1600M total ** DriveB: 4000 (BAM3) + 2 (SS2) + 2 (SA1) = 4004M Used of 4100M total DriveC: 2000 (BAM2) + 4 (SS1) + 1 (SA3) = 2005M Used of 2100M total

** For example, so a "mostly full" drive, as would likely be most common, or at least is the "trickiest" in terms of consideration for moving data around.... i.e. mostly empty drives are easy This is the simplest example possible, but you get the idea... To calculate up front, we can assume the biggest files (the movies) will stay put, as this gives us the least data to move, overall, and easiest calculations.

So, since BAM1 is currently on DriveA, we know we want to move SS1 (4M) and SA1 (2M) to DriveA eventually so All "1" files are on Drive "A" together, and so on for DriveB.... DriveN

If we add all desired "moves" to an array with their respective Sizes, we can loop through the Array, starting from the largest moves (which will still be relatively small, since the large files are staying put), to the smallest, and re-calculate the total storage used vs. free of each drive after each "dry run" file move operation.

As long as we don't exceed the free space of any target drive with any move, the operation should work from start to finish without issue. If any operation does run out of space (in the pre-planning phase), simply skip that move (say to target drive C) and instead, perform one or more moves away from drive C, until it has enough free space to handle the previously skipped move, then perform that move, and continue on down the line until all moves have been accounted for (in the dry run, still).

I think that would work and be fairly simple to calculate and display up front, before making any actual moves.

Please let me know your thoughts... this is just the first pass / rough draft in my head trying to work through this. And, assuming you won't have time to work on this any time soon, I'm going to have to write a quick and dirty version of this to fix my own self-imposed problem, so I am very interested to hear if you can see any glaring issues with my method that I am not seeing yet :)

Thanks again!

--Sean

On Fri, Feb 9, 2024 at 9:34 AM Juan B. Rodriguez @.***> wrote:

hi, thanks for the kudos !!

yes i think that for your use case, you would need to do a gather and select each "movie" folder individually towards the top drive

the logic doesn't seem too complicated, however it needs some considerations since, i think, it would involve re-evaluating free space after transfers

although i don't have bandwidth at the moment, i'm thinking about some pluggable algorithms that can be applied during calculations (not happening soon though 🙂)

— Reply to this email directly, view it on GitHub https://github.com/jbrodriguez/unbalance/issues/79#issuecomment-1936042782, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKYXWDKQH4ADZXKDHDYVZULYSYXYRAVCNFSM6AAAAABDAUYX5CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZWGA2DENZYGI . You are receiving this because you authored the thread.Message ID: @.***>

[jbrodriguez]

i think it should generally work, when you do it manually :)

simply skip that move (say to target drive C) and instead, perform one or more moves away from drive C

this part is a bit less clear to me from an programmatic point of view (as far as i can understand), it seems to require that the logic "look back" into transfers that were executed previously ...

without deep diving, i can't seem to find a way for a generic logic, let's say for example

• folder A is consolidated into drive 1 (through some logic)
• folder B is also consolidated into drive 1
• folder C is on drive 1, but there's not enough space left
• however is you move folder D out, there's enough space

there might be a generalization of this algorithm, something like what knapsack does, the trick is to figure that out :)

[jbrodriguez]

the plan is to have pluggable algorithms that handle the selection/collection and target phases, not sure when though