feat: add copying files to/from container
solves https://github.com/testcontainers/testcontainers-python/issues/665
i usually prefer using typing.Union[str, os.PathLike] - if you wanna use strings with this as well as paths
Codecov Report
Attention: Patch coverage is 32.00000% with 17 lines in your changes missing coverage. Please review.
Please upload report for BASE (
main@1e43923). Learn more about missing BASE report.
| Files | Patch % | Lines |
|---|---|---|
| core/testcontainers/core/container.py | 32.00% | 16 Missing and 1 partial :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #676 +/- ##
=======================================
Coverage ? 76.44%
=======================================
Files ? 12
Lines ? 624
Branches ? 96
=======================================
Hits ? 477
Misses ? 120
Partials ? 27
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
one other thing to consider is that there is this great interface "Transferable" in other language implementations-- does this PR make it harder or easier to add this to this implementation as well.
the interface (found here) is basically:
class Transferable(abc.ABC):
@abstractmethod
def transfer_to(content_stream, destination: typing.Union[str, os.PathLike]):
...
where content_stream is some bytesio or something pythonic for a place where you can write bytes to.
when i tried to do it, i accidentally interpreted the interface a bit too widely and instead of passing just the content output stream, i passed the whole container as that first argument.
https://github.com/testcontainers/testcontainers-python/commit/4b7db74176c0f00ca275bc39409ebb891a93cbb9 - misc functions, next commit doesnt make sense without reading first
https://github.com/testcontainers/testcontainers-python/commit/bbc44ec8e6d0ce35d40466f1d636106d75e0b197 - the commit where i add the transferable stuff
so im tiny bit hesitant to merge without a plan for adding transferable on top of this
do we need it to be 1:1 with java implementation though? any binary object now can be passed here using tempfile.NamedTemporaryObject:
with tempfile.NamedTemporaryObject() as f:
f.write(binary_data)
DockerContainer("nginx").with_copy_file_to_container(f.name, "/tmp/target.data").start()
we dont need to be 1-1 but what if instead of a file you had a string? i think it is beneficial to not force people to create temporary files.
refactored for Transferable although creating temp files cannot be avoided, now instead of asking users for 3 extra lines we add some complexity to codebase here. working example
input:
from pathlib import Path
from testcontainers.core.container import DockerContainer, Transferable
from testcontainers.core.network import Network
c = DockerContainer('nginx')
network = Network()
c \
.with_copy_file_to_container(Transferable(b'asd', Path('/tmp/mariusz.md1'))) \
.with_copy_file_to_container(Transferable(Path('/Users/mariusz/Documents/Programming/testcontainers-python/README.md'),Path('/tmp/mariusz.md2'))) \
.start()
c2 = DockerContainer('nginx')
print(c._container.exec_run('head /tmp/mariusz.md1'))
print(c._container.exec_run('head /tmp/mariusz.md2'))
output
ExecResult(exit_code=0, output=b'asd')
ExecResult(exit_code=0, output=b'[](https://python-poetry.org/)\n[](https://github.com/astral-sh/ruff)\n\n[](https://github.com/testcontainers/testcontainers-python/blob/main/LICENSE)\n[](https://pypi.python.org/pypi/testcontainers)\n[](https://codecov.io/gh/testcontainers/testcontainers-python)\n\n\n[](http://testcontainers-python.readthedocs.io/en/latest/?badge=latest)\n\n')
@mgorsk1 I have opened #677 to solve the same issue (I did not see you already had opened one 😓).
Yours have the Transferable feature. But I have the read_only feature.
And they are not implemented the same way :
- I am passing the paths in the
HostConfig.Bindto the docker_client.create (so the file exist before the container starts) - You are writing the files using the
docker.models.containers.Container.put_archivemethod (which works after the container has been started)
Are our two approaches complementary ? Reading again the Java documentation for copying files, there does not seem to be a difference, at the API level at least, between copying to a container before or after its startup.
I think copying before can already be solved by with_volume_mapping 😅
committed some code here - https://github.com/testcontainers/testcontainers-python/compare/main...feat/665-with-copy-file-to-container
rebased here https://github.com/testcontainers/testcontainers-python/compare/feat/665-with-copy-file-to-container_rebased
since this is not a high priority feature this can probably wait for 1) more api refinement - feedback welcomed on my suggestions above 2) more of my free time, as I'm not sure ill be able to spend too much more time on this this week (just spent my entire week's time budget on this today).