Tesseract-OCR-iOS
Tesseract-OCR-iOS copied to clipboard
gali8
Memory is not being freed
Hi, I have some code for OCR with custom settings, and after recognition the memory usage is about 100 MB which is not appropriate for me. But it seems like even this simple example doesn't work as needed:
let image = UIImage(named: "test.JPG")
let operationQueue = NSOperationQueue()
let operation = G8RecognitionOperation(language: "eng")
operation.tesseract.image = image
operation.recognitionCompleteBlock = {(tesseract: G8Tesseract!) -> Void in
println(tesseract.recognizedText)
G8Tesseract.clearCache()
}
operationQueue.addOperation(operation)
Before running this, my app uses 10.4 MB of memory. During the process of recognition, it uses about 97 MB, but after the recognition is done, the memory usage stays at 47 MB. I'm okay with high consumption during recognition but I want the memory to be released after. What can be the problem? G8Tesseract.clearCache() doesn't seem to make any effect. I'm using the latest TesseractIOS version from CocoaPods, 4.0.0. The image I use is a photo made with iPhone (3264x2448 px).
@icecoffin, it seems that you are trying to call 'G8Tesseract.clearCache()' while there is alive instance of tesseract in the memory. In such case no tess caches can be deallocated, cause they are retained by the tesseract instance. Try to call 'clearCache' after all the tesseract instances are completely deallocated. For example, with a delay as following:
dispatch_after(dispatch_time(DISPATCH_TIME_NOW, someDelayHere, dispatch_get_main_queue(), ^{
[G8Tesseract clearCache];
});
@ws233, thank you for your reply. Your advice did help, but the usage now looks like: Before recognition - 10 MB During recognition - 97 MB After recognition - 49 MB After clearing cache - 30 MB And the situation is even worse when I test on my real scenario (it's 150 MB during recognition and 80 MB after clearing cache). So it seems like some part is still not deallocated.
@icecoffin, we have got a lot of similar questions from different people about memory leaks in tesseract. You may search this repo by 'memory' keyword. In the most cases the issue was somewhere outside the tesseract. That doesn't mean the tesseract doesn't have memory leaks, but it seems so. So far I cannot say for sure, why you don't get the same footprint after the recognition as it was before. I don't have all of your code, but I suggest you to study the profiler logs to try to understand what's remain alive. If we get some extra details we can start investigating it deeper.
So far there is a known issue with the rotated images. In a few words, if the imageOrientation is left or right we have to redraw it before sending it to the tesseract. So in such case, we have 3 image copies at the same time in the memory. 1 is the original, 2nd is a rotated image, 3rd is an image in a special Pix format tesseract operate with. I have an idea to skip the second image and generate the 3rd one right from the first one, but I need to try it first. Perhaps, that will decrease the footprint of operating tesseract. But that won't fix an issue with the leaks of course. I will try your case as well very soon.
@ws233, sorry for the delay. I created a repository with sample code: https://github.com/icecoffin/TesseractTest-ObjC. The settings I use to configure the tesseract object are close to those I use in my real app (although the real app is on Swift, but the problem persists when using both Objective-C and Swift). Memory usage is the following: 6.8 MB on start 114 MB during recognition 58 MB after recognition and cleaning cache
If you have a chance, can you try my sample code to see if I'm doing anything wrong?
@icecoffin, as I expected it seems there is nothing to fix in tesseract. You use '[UIImage imageNamed:]' function, which caches the image being loaded. Proof from Apple doc:
imageNamed: Discussion: This method looks in the system caches for an image object with the specified name and returns that object if it exists. If a matching image object is not already in the cache, this method loads the image data from the specified file, caches it, and then returns the resulting object.
You may ensure that there is no leaks in tesseract, if you tap 'Start' button in your test project a few times. If there are leaks, you'll get increasing memory after every next recognition step, won't you? But I've tried your project and couldn't observe any increasing. After every run the app size falls down to the same 40Mb necessary for caching your test image by iOS.
The good point here is that those image caches are managed by the iOS, so there shouldn't be any leakes in the system. Furthermore I believe that if your app request so much memory, those caches will be cleaned automaticaly.
If you anyway want to clear the memory so it's in its startup state, I suggest you to use 'initWithData', 'imageWithData', 'imageWithContentsOfFile' or any other initializer instead of 'imageNamed'.
Finally, pls, let me know, if you could find any other leaks in Tesseract, which I've missed, and reopen a ticket so. But so far I think this issue can be closed.
@ws233, I know that iOS does caching when using imageNamed:. I actually don't use this method in my project, the image comes from camera (I probably should've pointed it out). But in fact there is no difference when I use another method - both imageNamed: and initWithContentsOfFile: give the same memory usage.
I also tried to take images from camera using UIImagePickerController. I'm not sure if there is any caching but I've got the same result - 60 MB memory usage after cleaning the cache :)
Anyway, thank you for your help. It feels kinda weird that iOS would use 50 MB to cache a single image. Can you tell me if 150-200 MB memory usage during recognition is a normal thing for Tesseract? I don't quite like the idea that many background apps may be removed the memory by iOS when the user starts the recognition process.
@icecoffin, let's just calculate. I assume the 4S camera with resolution 3264 x 2448. It's about 3000x2500, which is equal to 7,5MP. Every pixel is in 32 bits per pixel, which gives 4 bytes per pixel. So the total weight of the camera image in the memory is 7,5MP x 4Bpp = 30 MB. That's just we've got an image from the camera and upload it to the memory.
On the 2nd step the framework code checks, if the image has the orientation different from 'UIImageOrientationUp'. In this case, the framework rotates the image and redraws it so it's in UIImageOriantationUp. This gives us one more copy of the image and 30MB more. This step seems redundant, and I have a few ideas how to avoid it. But until I've not tried it and have not implemented it, we have to use this step. Otherwise, the framework will just crach, if it recieves UIImageOrientationUp image on input.
The 3rd step is preparing the binary data for the tesseract. In this step the framework just copies the image from step 2 byte by byte in the byte array. That gives us the 3rd copy of the image and 30MB more. Totally we have about 90MB at this step. It seems that this step is necessary and couldn't be eliminated, since upstream tesseract cannot understand UIImage format.
So it seems, that the numbers you've provided are very real. Perhaps, in your case there is just larger camera sensor.
Anyway, as I've already mentioned, the only way to decrease the app recognition footprint, which I see so far, is to eliminate the step 2 from the recognition process. But I'm a bit busy now, so I'll check this in near future. @icecoffin, feel free to help me with this if you are so interested in it. Thx!
PS: Perhaps, we even may free the original image and do not retain it anymore as soon as recognize function has been called, that will save one more image copy for us in the memory. But I'm not sure, it's a good idea. Let's first eliminate the 2nd step from above.
@icecoffin, pls, try #205 and let us know if this patch could decrease memory consumption during recognition in your case.
So the next possible step here is adding a 'shouldCacheInputImage' property. By default it's YES and do not change anything in current recognition procedure. But if it's NO, it forbids caching the input image in 'image' property. In such case if the user tries to get the input image, it will be generated directly from tesseract engine Pix data using 'imageFromPix' function. That's gonna work slow, but releases one more image from the memory. Actually retrieving current image from the tesseract seems to be a very rare operation, so perhaps we should forbid image caching by default.
Just a small code snippet to illustrate above:
{
// read an image and do not allow the system to cache it
UIImage *image = [UIImage imageWithData:[NSData dataFromFile:somePathHere]];
// we have 1 copy of the image at this point
// set it to tesseract
tesseract.image = image;
// Pix for the input image has been created so we actually have 2! image copies at this point
}
// at this point the image could be released cause we are out of the block where it was created,
// but it still retained by tesseract.image property
[tesseract recognize]; // tesseract only needs it Pix data here
@kevincon, what do you think? Do we need to decrease Tesseract footprint in the memory by forbidding to have an original input image in the memory?
Yeah that might be a good idea, but rather than trying to keep track of a shouldCacheInputImage property, I think it would just be easier for users if we reworked the API so that we have "purer" functions like recognizeWithImage and analyzeLayoutWithImage, etc. that don't store any state, and then we could completely remove the setImage function. What do you think?
@kevincon, I don't think your idea will succeed. Let me explain why with a code snippet.
// read an image and do not allow the system to cache it
// but anyway it's in the memory
UIImage *image = [UIImage imageWithData:[NSData dataFromFile:somePathHere]];
[tesseract recognizeWithImage:image]; // image is still retained here, cause it's sent to the function as a parameter, so it's alive while we are recognizing
// the image can be deallocated only here :(
Do you see any other ways to do it without setImage function?
Ah I see what you mean now. Yeah I think your shouldCacheInputImage property is a good idea to solve that problem.
