zeoslib
zeoslib copied to clipboard
marsupilami79
Oracle loblocator changes.
Fixes to UTF8 lob reading and writing. Threefold performance improvement for "None" and "OnAccess" cachelobmodes.
Note that I've fixed the TZOracleLobStream64.ReadPoll function. The existing code failed due to not properly handling the very finicky buffer sizes related to the LobPrefetch size (which defaults to 8k in Zeos.) However I've replaced that call with a non-polling version named ReadFull which is simpler and safer. I've left both versions in just in case there's a need.
Hello Mark,
again I am sorry for the delay. I wanted to have a running Oracle installation and automated tests before I apply the patch. The patch is applied and now I only will wait for the test results before I close the PR. THe changes should sow up on github by tomorrow.
Thank you and best regards,
Jan
Hello Mark,
so - Jenkins has done its job and we have test results for Oracle 21 and Oracle 11. Some of the tests fail now: TZTestDbcOracleCase.TestLongObjects TZTestDbcOracleCase.TestLargeBlob These tests fail for Oracle 21 as well as Oracle 11.
TZGenericTestDataSet.Test_LobCacheMode This test fails for Oracle 21 only...
Is there any chance for you to look into that?
I've found the issue (at least for the OCILobIsTemporary errors.) When using an uncached result set, getblob returns an interface. That interface is freed when it goes out of scope with refcount of zero. If the connection is closed before then (as it is in the tests) accessing the loblocator fails (Oracle has already freed the locator.) I didn't hit this in my tests as I don't use uncached result sets directly (cached result sets and datasets own the blobs and so they are freed before disconnect.) This issue can be solved in a couple ways. #1 is to just not raise that exception. #2 is to not have the blobs handle their own loblocators in which case the OracleResultSet keeps a list of all loblocators and frees them on close. I already have a version of this as this was my first pass attempt, but I don't love it as it was inelegant and loblocators had a longer lifetime than needed. I believe this issue only affects uncached result sets (not datasets) where a disconnect happens in the same scope as blob retrieval.
After further thought and a bunch of testing here are my thoughts:
We can't use this pull request as is. It will have problems with blob lifetime if an uncached recordset (unrelated to lobcachemode btw) is used (and not just the issues you found, there will be an issue with the same blob requested multiple times in different scopes.) I don't believe the technique I used (blobs managing their own loblocators) can be made to work.
I have another pull request which will give the same performance boost for cached recordsets and should avoid the issues above, but it will have the side effect of using more loblocators and associated memory for uncached resultsets. In my case, the performance improvement will be worth it, but uncached result sets will be affected unless it's possible to tell from inside TZOracleAbstractResultSet that is being used by a cached result set (I'm not sure that's possible.) [update... it looks like I can check rtForwardOnly for this!]
Again, sorry about this. I've now been able to setup the testing environment and so will try to avoid this kind of thing in the future. I've had no luck recreating the LobCacheMode error at all, but hopefully backing out this pull request will fix it too.
-Mark
I've pushed a new pull request with the fixes. A lot of the above can be ignored now as the cumulative changes should give the performance improvements without affecting forward-only uncached result sets. I did not do anything related to Test_LobCacheMode as I can't get it to fail on my system. Any additional info on that one is appreciated.