testcontainers-python icon indicating copy to clipboard operation
testcontainers-python copied to clipboard

feat(generic): Reintroducing the generic SQL module

Open Tranquility2 opened this issue 2 months ago • 7 comments

Related to #884 Trying to replace the old generic.py in core to a nicer version under the generic module.

SqlContainer its not as good (in being generic) as ServerContainer but it should allow us to deprecate core/testcontainers/core/generic.py with minimal effort for users, it could lead to a more Generic version like DBContainer in the future.

Update 1: Refactor to use SqlConnectWaitStrategy Update 2: Now SqlConnectWaitStrategy is required and the users can provide SqlContainer with any wait strategy. Update 3: Now utilizes all the latest improvements from WaitStrategy

Note: I think the added tests + documentation (provided in this PR) are by themselves a great improvement over the current generic.py

Tranquility2 avatar Oct 02 '25 21:10 Tranquility2

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 81.71%. Comparing base (f608df9) to head (aacc13a).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #892      +/-   ##
==========================================
+ Coverage   79.78%   81.71%   +1.93%     
==========================================
  Files          14       13       -1     
  Lines        1182     1154      -28     
  Branches      184      184              
==========================================
  Hits          943      943              
+ Misses        197      169      -28     
  Partials       42       42              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Oct 03 '25 03:10 codecov[bot]

can we add a new extra? like a db (or more accurately, sql) extra? this is assuming we dont want to aim for complete removal.

Even if we do that, i dont like the _connect method on the container - i think connecting is the business of 1) app code 2) sql waiting strategy - so all the db interface has to do is tell someone how to connect - and maybe this can just return the url?

This second point is made further complicated by now you can only use this waiting strategy on container that have the interface implemented. i think maybe this waiting strategy can be parametrized - with information on how to build the url. this removes the need for the container to build its own url (finally). just a thought.

alexanderankin avatar Oct 03 '25 13:10 alexanderankin

  • The new extra sound a bit like an overkill and could be a bit confusing for the users don't you think? My goal was to make the transition as seamless as possible but I agree it might be a good time to rethink some things. I just don't want to "branch out" too much, all in all its a very good point but I concerned it will be lost will all the other sql modules 😓
  • I totally understand your point regarding _connect , I had one iteration where all the logic was under ConnectWaitStrategy (with no _connect at all) and I assumed we want backward compatibility with DbContainer so it will be easier to move to (so we can remove generic.py), let me try and do another version without the connect and ConnectWaitStrategy more parametrized as you suggested.

Tranquility2 avatar Oct 03 '25 14:10 Tranquility2

New version is ready:

  1. no more _connect
  2. all extended logic moved to SqlConnectWaitStrategy (new file) making sql.py much cleaner and more generic
  3. smaller improvements to make the code simpler and easier to maintain

You can now say SqlConnectWaitStrategy is basically sqlalchemyConnectWaitStrategy 😂 but this can also be improved and made more generic.

Tranquility2 avatar Oct 03 '25 15:10 Tranquility2

I like the new SqlConnectWaitStrategy a lot. Indirectly the _connect method paired with wait_container_is_ready did just that in many database modules.

surister avatar Oct 03 '25 17:10 surister

Make one more critical improvement, now wait_strategy is a param for the SqlContainer, this

  1. keeps it very generic - all the others can use it with ease
  2. provides SqlConnectWaitStrategy if anyone needs its
  3. allows each module to provide a custom wait_strategy (that can be as simple as waiting for some log string from the server)

Tranquility2 avatar Oct 04 '25 09:10 Tranquility2

@alexanderankin now using WaitStrategy._poll and with_transient_exceptions from your latest improvements. This makes SqlConnectWaitStrategy much simpler cleaner, thanks 🙏

Tranquility2 avatar Oct 04 '25 10:10 Tranquility2