soda-core
soda-core copied to clipboard
sodadata
Fix YamlHelper for thread safety in multi-threaded environments.
Overview
This pull request addresses the thread safety issue with YamlHelper as reported in Issue #2064. The modifications ensure that YamlHelper can be safely used in multi-threaded environments, preventing potential data races and concurrency errors.
@m1n0 Can you please review the changes and provide feedback or approve the merge if everything looks in order. Thank you!
Quality Gate passed
Issues
0 New issues
0 Accepted issues
Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code
See analysis details on SonarCloud
![sonarqubecloud[bot] avatar](https://avatars.githubusercontent.com/in/12526?v=4 "Published by a pub.dev verified publisher"){kind=small}
Quality Gate passed
Issues
0 New issues
0 Accepted issues
Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code
See analysis details on SonarQube Cloud
hi @NiklasA, thanks for looking into this and thanks for your patience! I am thinking that opening a thread for this is a bit of an overkill. I looked into it myself and it seems to me that all that needs to happen is to have a new yaml instance instead of reusing the same one like this https://github.com/sodadata/soda-core/pull/2188 - my local testing confirms it working, so I merged it to unblock a release, please test it and get back to me. We can definitely revisit your proposal for the fix if it still does not fix the issue
Hi @m1n0 ,
thank you for your detailed response! We’ve tested your approach and compared it to our existing implementation. While your solution works well, we’ve observed that its performance is approximately 10% slower than our current approach (within our existing load tests).
For context, we are using the datacontract-cli as part of our data ingestion processes, where we need to verify a significant amount of data. This might represent an edge case, but in these scenarios, performance is critical for us.
We appreciate your input and are happy to discuss potential optimizations or alternative approaches that might bridge this gap. Let us know your thoughts!
Best regards Niklas