keras-team
The data enhancement function apply_brightness_shift is very low efficiency, and I have an improvement to vastly speed up it
When you use big batchsize with multiGPU and enable keras brightness data enhancement at same time, you will find the cpu usage is very high and the gpu usage is very low.
I have an improvement to vastly speed up it. Before optimization, cpu usage is almost 100%, and gpu avg usage is 50%. After optimization, cpu avg usage is 40%, and gpu avg usage is 85%. By the way, my hardware are 4*TITAN XP and AMD 1950X(16C/32T). If your cpu has lower performence then it, maybe this optimization are more useful.
My optimization plan as follows.
Original code are:
def apply_brightness_shift(x, brightness): ... x = imgenhancer_Brightness = ImageEnhance.Brightness(x) x = imgenhancer_Brightness.enhance(brightness) ...
Replace them with:
def apply_brightness_shift(x, brightness): ... x = cv2.addWeighted(x, brightness, x, 0.0, 0.0) x = np.clip(x, 0, 255) ...
@pandasxx I'm not sure why you asked me to check. Did you mean to tag someone else?
@juharris I mean apply_brightness_shift is very low efficiency, when I use the original method in my project too much CPU resource was used. And maybe we can improve it.
That's great but why did you tag me specifically? I don't really work on this.
That's great but why did you tag me specifically? I don't really work on this.
Aha, sorry bro. But I don't know who work on this and could you give me some suggestions?
Sorry I'm not sure. You can look at files on GitHub related to what you want to change and click Blame to see who has changed them. Good luck. I will be unsubscribing from this issue now.
@rragundez Hey bro, I found apply_brightness_shift is very low efficiency, when I use the original method in my project too much CPU resource was used. And maybe we can improve it.
@pandasxx Hi, perhaps you should (1) Make a PR instead, not an issue. (2) When making the PR, explain why the old version suffers low memory usage efficiency and why your code works. (3) I've noticed that you're using cv2 in your code. Usually it's not a good idea to add opencv dependency because it's a heavy dependency. If opencv was not in official dependency list, this PR may not get merged.
Good luck~
@MoyanZitto Thanks bro. I will make a better method without cv2 dependency and submit a PR.