The processing speed starts out fast, and then gets slower and slower.

[BrutolocoW]

The processing speed starts out fast, and then gets slower and slower.

To restart the render speed, I have to close Flowframes and reopen it.

I'm interpolating a video with Vulkan with a float multiplier of 2.4, that converts a 25fps movie to 60fps in one step, when it finishes extracting frames and starts interpolating, the speed is about 30fps.

In the middle of the video, it usually goes for 15fps. And finally it gets stuck at 6fps. And so it remains until the end.

If after the video ends I add another new video, it starts at 6fps.

But if I close Flowframes and start again, it starts at 30fps.

I imagine this all has to do with concurrent processes going down on their own, but there seems to be no way for them to wake up on their own, or wake up again.

An option or feature that prevents processes from being closed might be the key.

[satvikpendem]

I'm running into this too. Did you find any other solution than closing and reopening?

[ghost]

Me too. It keeps getting slower and slower.

[jensdraht1999]

Me too. It's starting with high number of frames and then gets slower. I wonder if that is something to do with logging in pytorch. https://github.com/Lightning-AI/lightning/issues/581

[jensdraht1999]

@n00mkrad I found the error, why it is getting slow.

My Hardware: I5-10500H with 6 Cores / 12 Threads and Nvidia RTX 3060 Laptop with 6GB

The command that fixed it for me is: cd /D "pathtoFlowFrames\FlowframesData\pkgs\rife-cuda" & set CUDA_VISIBLE_DEVICES=0 & "pathtoFlowFrames\FlowFramesApp - 1.40\FlowframesData\pkgs\py-amp\python.exe" -u "pathtoFlowFrames\FlowframesData\pkgs\rife-cuda\rife.py" --input "pathtoFlowFrames\6-temp\frames" --output interp --model RIFE46 --multi 2 --wthreads 12 --rbuffer 400

The command that is generated by FlowFrames automatically: cd /D "pathtoFlowFrames\FlowframesData\pkgs\rife-cuda" & set CUDA_VISIBLE_DEVICES=0 & "pathtoFlowFrames\FlowFramesApp - 1.40\FlowframesData\pkgs\py-amp\python.exe" -u "pathtoFlowFrames\FlowframesData\pkgs\rife-cuda\rife.py" --input "pathtoFlowFrames\6-temp\frames" --output interp --model RIFE46 --multi 2 --wthreads 4 --rbuffer 200

Now the problematic line is "--wthreads 4", so it cannot feed the GPU fast enough and also generating pngs is costly also, which is faster, if you have more threads available.

So the code is the problem with: Code/Os/AiProcess.cs Line 202: string wthreads = $"--wthreads {2 * (int)interpFactor}";

You might add something like: int threadcount = Process.GetCurrentProcess().Threads.Count; string wthreads = $"--wthreads {threadcount}";

While I executed with wthreads above 8 threads, it never slowed down, cuda was used 90-95% always, just occasionally dropping to 85%, but then going up to 95% directly.

EDIT: Also tested jpg interpolated frames, it is not faster, because at that point the GPU is not ready to handle the files. But it uses less cpu resources, than png interpolated frames and also uses less space on your drive. So there is nothing we can do here. And I doubt giving such an option to people would be good. Also this might break transparent gif for people, so I would not care to look more into this.

[jensdraht1999]

@n00mkrad

OK, another two things found out, that make it faster:

1.) When generating so much images at once, Windows is using Windows Search indexer, because it thinks, that these are regular files. I tried to disable windows search indexer via cmd, but it does not work reliable. However another way to avoid this and this works perfectly and is faster(because cpu resources are not used for windows search service) is to have the files in the windows temp directory, which is not indexed by default.

2.) If you enter more wthreads than you actually have, then it is indeed faster by a slight margin, but cuda is running at 95% nearly all the time. I think, that the windows sceduler thinks, that it is important process and the cmd is literally running 70-90% all the time nothing eating other resources.

Edit: If you use jpg for interpolated frames, now it around 2-5% faster too. Just letting you know.

will need look for this too, if this changes something: https://superuser.com/questions/620724/changing-windows-process-priority-via-command-line

[jensdraht1999]

Okay compiled it. Good and bad news:

bad news: it still get slower with time even if you set python script prioty to above normal or high. The treads seem to sleeep, but only if you start it via FlowFrames it seems.

Good news: even when it is getting slower, it's much faster.

[jensdraht1999]

OK last good news, I have found the culprit code, but do not know how to fix it, I'm not a programmer.

I also interpolated a video 20 min via FlowFrames now and it did not slow down, there a few slowdowns and then it gets up high, with fp16 i'm about getting 68 frames per second with png.

My theory with -wthreads was wrong, however having more -wthreads makes everything faster without a doubt.

The culprit is following function in logger.cs:

            public static List<string> GetSessionLogLastLines(string filename, int linesCount = 5)
            {
                string log = GetSessionLog(filename);
                string[] lines = log.SplitIntoLines();
                return lines.Reverse().Take(linesCount).Reverse().ToList();
            }

, which I have commented out of all the code and it's totally stable 66-69 frames per second.

I assume following happens. The log file gets bigger. Now it takes the log file and then reverses it.

Why it is slow, because:

Reversing 100 Lines of text fast, but reversing 50000 lines of takes much much more time. And the bigger the file gets, reversing the entire files takes more cpu time and this cpu time cannot be spent to send commands to the gpu.

I think the author of the flowframes though about that problem, which is indicated by linescount 5, so he wanted to show the last 5 lines in reverse order, but somehow the code does not do, what it was intended to do.

BTW: Encoding a 20 Minute video which took me 30 mins, is now ending in 17 mins. There are no problems with the video. Autoencode was disable for this.

[satvikpendem]

I hope this is added by @n00mkrad but not sure it will be as this repo hasn't had a commit since March and his Patreon doesn't have any releases from then either.

[BrutolocoW]

OK last good news, I have found the culprit code, but do not know how to fix it, I'm not a programmer.

I also interpolated a video 20 min via FlowFrames now and it did not slow down, there a few slowdowns and then it gets up high, with fp16 i'm about getting 68 frames per second with png.

My theory with -wthreads was wrong, however having more -wthreads makes everything faster without a doubt.

The culprit is following function in logger.cs:

            public static List<string> GetSessionLogLastLines(string filename, int linesCount = 5)
            {
                string log = GetSessionLog(filename);
                string[] lines = log.SplitIntoLines();
                return lines.Reverse().Take(linesCount).Reverse().ToList();
            }

, which I have commented out of all the code and it's totally stable 66-69 frames per second.

I assume following happens. The log file gets bigger. Now it takes the log file and then reverses it.

Why it is slow, because:

Reversing 100 Lines of text fast, but reversing 50000 lines of takes much much more time. And the bigger the file gets, reversing the entire files takes more cpu time and this cpu time cannot be spent to send commands to the gpu.

I think the author of the flowframes though about that problem, which is indicated by linescount 5, so he wanted to show the last 5 lines in reverse order, but somehow the code does not do, what it was intended to do.

BTW: Encoding a 20 Minute video which took me 30 mins, is now ending in 17 mins. There are no problems with the video. Autoencode was disable for this.

Thanks for your comment.

I have read everything, but I don't know if I understand it right. I am even less knowledgeable on the subject.

So I have 2 question.

1. Finally, what would be the best workaround (from your analisys), for users, to make it faster until the nomkrad comes back and fixes it?
2. Is it necessary to disable autoencoding as well, since it you mention it at the end?

[jensdraht1999]

@BrutolocoW @satvikpendem

@n00mkrad might have not seen this. I am not exactly sure, because the only thing is to merge it and alone this should give FlowFrames some addiotal things.

I have made enhancements to FlowFrames. However I do not know, if you can actually use it, because I have never had used a PR myself, but I have provided a PR. So basically you should clone it and then use a free versions "Py-Amp" folder to use it.

However I do not know, how you can clone a PR. Never done it.

The PR in question is this:

https://github.com/n00mkrad/flowframes/pull/275

This is basically "FlowFrames 1.44"

[0rsa]

@jensdraht1999 thank you basically you can get your code using git clone https://github.com/jensdraht1999/flowframes-1.git . git checkout d0f4265dcc6ce9bd61da38783618fc228ee2ffba

Then I'm not sure to understand what you mean by "use a free versions "Py-Amp" folder to use it." Please can you elaborate on this?

[jensdraht1999]

@0rsa There is folder called "Py-amp". It's in FlowframesData -> pkgs -> py-amp. If you download the free version of 1.36 with embedded installation for ampere cards / RTX 30YZ cards, then it will be called "py-amp". Please compile flowframes via visual studio and then use the copy of the "py-amp" folder in your flowframes folder. So this would work.

[jensdraht1999]

OK last good news, I have found the culprit code, but do not know how to fix it, I'm not a programmer. I also interpolated a video 20 min via FlowFrames now and it did not slow down, there a few slowdowns and then it gets up high, with fp16 i'm about getting 68 frames per second with png. My theory with -wthreads was wrong, however having more -wthreads makes everything faster without a doubt. The culprit is following function in logger.cs:

            public static List<string> GetSessionLogLastLines(string filename, int linesCount = 5)
            {
                string log = GetSessionLog(filename);
                string[] lines = log.SplitIntoLines();
                return lines.Reverse().Take(linesCount).Reverse().ToList();
            }

, which I have commented out of all the code and it's totally stable 66-69 frames per second. I assume following happens. The log file gets bigger. Now it takes the log file and then reverses it. Why it is slow, because: Reversing 100 Lines of text fast, but reversing 50000 lines of takes much much more time. And the bigger the file gets, reversing the entire files takes more cpu time and this cpu time cannot be spent to send commands to the gpu. I think the author of the flowframes though about that problem, which is indicated by linescount 5, so he wanted to show the last 5 lines in reverse order, but somehow the code does not do, what it was intended to do. BTW: Encoding a 20 Minute video which took me 30 mins, is now ending in 17 mins. There are no problems with the video. Autoencode was disable for this.

Thanks for your comment.

I have read everything, but I don't know if I understand it right. I am even less knowledgeable on the subject.

So I have 2 question.

1. Finally, what would be the best workaround (from your analisys), for users, to make it faster until the nomkrad comes back and fixes it?
2. Is it necessary to disable autoencoding as well, since it you mention it at the end?

I totally forget to answer your questions.

1.) The best workaround is to download the like @0rsa explained and use it.

2.) Autoencode will lower the performance of fps out, since it will use ffmpeg in the background time to time, but it still is much faster, because most of the encoding is done. I recommended using it, leave it ON. However if you want to benchmark raw performance of your gpu, then you need to disable it, to see how much fps you can output.

In the settings there is a preset called "Intel/Nvidia" for high performance throughput. These are the best settings, I always use.

[0rsa]

@jensdraht1999 thank you again A last question :) I'm currently using the 1.40 release (full embedded), and I have also a py-amp folder in my local installation. Is it enough or should I start absolutely from a free version?

Update: Ok I think I'm good I did the reverse of what I understood: I copied the content of my current flowframes 1.40 "Pkgs" folder to the debug version compiled by VS (/Code/bin/Debug/FlowframesData/pkgs from the project folder you pulled from git). Then I use directly the Flowframes.exe located in /Code/bin/Debug of the pulled project. I will let you know as soon as I test. Thank you

Update 2:: I confirm it works much better! FPS and GPU usage remain constant now, from 2h05 with 1.40 to 52mn with this 1.44 Thank you very much @jensdraht1999

[jensdraht1999]

@0rsa Sorry for the late reply. It does actually not matter, if you use the free or paid or patreon version. Just the py-amp folder is enough.

You seem to have RTX 3080 or 4090?

And what kind of interpolation frames are you using? PNG or JPG? FP16 Mode or not?

[jensdraht1999]

Oh sorry I see your gpu on the flowframes gui title. It says RTX 3070 with 16GB.

However I wonder, why it is using so much vram?

[0rsa]

Oh sorry I see your gpu on the flowframes gui title. It says RTX 3070 with 16GB.

However I wonder, why it is using so much vram?

This is a 3070 with 8Gb so I don't know what is this number. Also CUDA is displaying 0.00% when the GPU is full used so I'm not sure about the reliability of these numbers.

About options I'm using default options

[jensdraht1999]

@0rsa I think it takes your total VRAM with internal GPU Ram. For me it is working okay.

For the cuda 0.00%, you must check, if this works at all.

Here a picture of my taskmanager on WIN10:

[0rsa]

I do not have Cuda in the list

[jensdraht1999]

@0rsa Okay then I know what is the problem. My brother has Windows 11 and also Nvida GPU and latest gpu drivers installed. It seems, that Windows 11 has no Performance counter for cuda or vram, so it shows those weird errors. The same with my brothers pc.

I think, we might look into this, but it seems at least the gpu is not overusing vram, which is also good.

[BrutolocoW]

OK last good news, I have found the culprit code, but do not know how to fix it, I'm not a programmer. I also interpolated a video 20 min via FlowFrames now and it did not slow down, there a few slowdowns and then it gets up high, with fp16 i'm about getting 68 frames per second with png. My theory with -wthreads was wrong, however having more -wthreads makes everything faster without a doubt. The culprit is following function in logger.cs:

            public static List<string> GetSessionLogLastLines(string filename, int linesCount = 5)
            {
                string log = GetSessionLog(filename);
                string[] lines = log.SplitIntoLines();
                return lines.Reverse().Take(linesCount).Reverse().ToList();
            }

, which I have commented out of all the code and it's totally stable 66-69 frames per second. I assume following happens. The log file gets bigger. Now it takes the log file and then reverses it. Why it is slow, because: Reversing 100 Lines of text fast, but reversing 50000 lines of takes much much more time. And the bigger the file gets, reversing the entire files takes more cpu time and this cpu time cannot be spent to send commands to the gpu. I think the author of the flowframes though about that problem, which is indicated by linescount 5, so he wanted to show the last 5 lines in reverse order, but somehow the code does not do, what it was intended to do. BTW: Encoding a 20 Minute video which took me 30 mins, is now ending in 17 mins. There are no problems with the video. Autoencode was disable for this.

Thanks for your comment. I have read everything, but I don't know if I understand it right. I am even less knowledgeable on the subject. So I have 2 question.

1. Finally, what would be the best workaround (from your analisys), for users, to make it faster until the nomkrad comes back and fixes it?
2. Is it necessary to disable autoencoding as well, since it you mention it at the end?

I totally forget to answer your questions.

1.) The best workaround is to download the like @0rsa explained and use it.

2.) Autoencode will lower the performance of fps out, since it will use ffmpeg in the background time to time, but it still is much faster, because most of the encoding is done. I recommended using it, leave it ON. However if you want to benchmark raw performance of your gpu, then you need to disable it, to see how much fps you can output.

In the settings there is a preset called "Intel/Nvidia" for high performance throughput. These are the best settings, I always use.

Hello,

I don't know what I'm doing wrong.

I downloaded https://github.com/jensdraht1999/flowframes-1.git as zip, decompressed, then installed Visual Studio and tried to compile it with Visual Studio (following this guide https://blog.gitnux.com/guides/how-to-compile-in-visual-studio/), always the same error. I tried mixing it with main code, and gives same error.

[0rsa]

@BrutolocoW you cannot download the zip directly from the repo. You have to select the right commit to have the latest version as I mentioned above. You can install git for windows to use the commands from my post in a command line.

[jensdraht1999]

@BrutolocoW I use Visual Studio. Just clone repisitory. Then compile. Afterwards take your FlowFrames Folder from your old installation and paste it into the bin directory where the compiled code is. Now it should work.

[jensdraht1999]

BTW, you need to clone: https://github.com/jensdraht1999/flowframes-1/tree/flowframes1.41, then it should work, otherwise it's the old main branch.

[BrutolocoW]

Hello, I'm sorry bothering again, I tried almost everything, and getting same error.

@BrutolocoW you cannot download the zip directly from the repo. You have to select the right commit to have the latest version as I mentioned above. You can install git for windows to use the commands from my post in a command line.

I installed git, open bash on a empty file, clone repository with git clone https://github.com/jensdraht1999/flowframes-1.git. Once finished, then I tried to compile with visual studio by opening sln file in /Flowframes/Code, adn then in the top bar, Build->Build solution, making sure is on Debug is on, tried Release too. Same error than before.

@BrutolocoW I use Visual Studio. Just clone repisitory. Then compile. Afterwards take your FlowFrames Folder from your old installation and paste it into the bin directory where the compiled code is. Now it should work.

Tried also with clone from visual Studio, and then build-> build solution, and getting same error on the above screen capture.

BTW, you need to clone: https://github.com/jensdraht1999/flowframes-1/tree/flowframes1.41, then it should work, otherwise it's the old main branch.

Tried to clone that link, from git bash and visual studio, and gives error on cloning.

PD: I managed to build it after downloading zip file for that link. Cloning gives error for that link, don't know why.

I did not found logic on copy old Flowframes folder to bin build files because it will overwrite build files, I made it the opposite, though makes more sense. I tried both ways, by not overwriting flowframes.exe I get new options in the UI, if I overwrite its like I'm just loading the old program, without any changes.

Flowframes gives this error when tried to make a video test interpolation (with all default options):

[jensdraht1999]

video test interpolation (with all default options):

If you have compiled via Visual studio: Copy the FlowFrames.exe from there to your FlowFrames Folder. Replace it. Then it should normally work fine.

[jensdraht1999]

[BrutolocoW]

video test interpolation (with all default options):

If you have compiled via Visual studio: Copy the FlowFrames.exe from there to your FlowFrames Folder. Replace it. Then it should normally work fine.

Hello,

I tried with only copying the new exe file, and not copying the rest, and then gives the same error I've shown above "Frame extraction failed". Seems like I can't make this work.

[jensdraht1999]

@BrutolocoW I think you might have broken your installation. I recommend following for you.

1.) Please download FlowFrames 1.40 with embeded python and also embeded dependencies.

2.) Check if it is working.

3.) If it is working, go to your Visual Studio Folder and get back the flowframes.exe, but rename it to something else like FlowFramesNew.exe and copy it to your FlowFrames 1.40 Folder beside the other .exe file, which is there.

4.) Test out, if it is working.

5.) Please also can you try to disable "Intel QSV Extraction" in the settings, if it is open. Because if you have no Intel, this might not work, also sometimes, driver releated it cannot recognize Intel QSV.

Because it seems there is a problem with the extraction of the images.

[BrutolocoW]

@BrutolocoW I think you might have broken your installation. I recommend following for you.

1.) Please download FlowFrames 1.40 with embeded python and also embeded dependencies.

2.) Check if it is working.

3.) If it is working, go to your Visual Studio Folder and get back the flowframes.exe, but rename it to something else like FlowFramesNew.exe and copy it to your FlowFrames 1.40 Folder beside the other .exe file, which is there.

4.) Test out, if it is working.

5.) Please also can you try to disable "Intel QSV Extraction" in the settings, if it is open. Because if you have no Intel, this might not work, also sometimes, driver releated it cannot recognize Intel QSV.

Because it seems there is a problem with the extraction of the images.

My standard flowframes installation 1.36 works, but doesn't with the new exe compiled. I might try with a new installation but I think the problem is my graphic card which is not RTX. It's a gtx 1060 6gb.

I've read something above about "py-amp" folder which is for rtx 3000, that might be the problem. I have to use the first option "compatibility with nvidia 16/20 and older" I've already tried disabling intel qsv, and chaning other new settings, it fails on extraction. I have an intel processor anyway.

Still don't know why I can't clone and build the program from git or visual studio. It only worked when I downloaded zip from this link https://github.com/jensdraht1999/flowframes-1/tree/flowframes1.41. Which I can't clone.

P.D: Confirmed, new installation of Flowframes 1.36 shows the same error message. It works with normal exe files and not with new compiled exe files.