testcontainers
TypeError: object.__new__() takes exactly one argument (the type to instantiate)
Describe the bug
running testcontainers in a pytest using a with statement I get the following error
@wait_container_is_ready()
def _connect(self) -> MongoClient:
return MongoClient(self.get_connection_url())
E TypeError: object.new() takes exactly one argument (the type to instantiate)
Just wondering if anyone else has encountered this
can you post a little bit more details on how you got this output? some of your surrounding code maybe?
I'll update with test and example once I finish the urgent work for work. the fix was simple just needed the self.get_connection_url() added to the init rather than the constructor
@wait_container_is_ready() def _connect(self) -> MongoClient: mc = MongoClient() mc.init(self.get_connection_url()) return mc`
Sorry @alexanderankin I hadn't cleared cache and this is still happening. I'll write test case for this and the localstack bug.
if you have a stack trace for this that would also be helpful, I remember looking at this one and the mongodb client does not throw this error in the constructor, so its not obvious to me where it comes from.
if the bug is in the testcontainers library, presumably it is because we are not waiting for the container correctly or something like that. to be clear your code example is actually this, right:
@wait_container_is_ready()
def _connect(self) -> MongoClient:
mc = MongoClient()
mc.__init__(self.get_connection_url())
return mc`
i made it syntax highlight by using this in my markdown:
```python
# code goes here
```
Just cleaning down my local env. I’ll stick some stack traces and a test case in. I’d like to get rid of my private repo for my fixes :)
On 14 Feb 2024, at 11:52, David Ankin @.***> wrote:
if you have a stack trace for this that would also be helpful, I remember looking at this one and the mongodb client does not throw this error in the constructor, so its not obvious to me where it comes from.
if the bug is in the testcontainers library, presumably it is because we are not waiting for the container correctly or something like that. to be clear your code example is actually this, right:
@wait_container_is_ready() def _connect(self) -> MongoClient: mc = MongoClient() mc.init(self.get_connection_url()) return mc` i made it syntax highlight by using this in my markdown:
# code goes here— Reply to this email directly, view it on GitHub https://github.com/testcontainers/testcontainers-python/issues/401#issuecomment-1943620773, or unsubscribe https://github.com/notifications/unsubscribe-auth/BC3KZVOFHBAE2WMQVSR3PDLYTSQPFAVCNFSM6AAAAABAJJE3AWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBTGYZDANZXGM. You are receiving this because you modified the open/close state.
--
This email and any attachments are confidential, may contain privileged material and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Covatic. If you are not the intended recipient of this email, you must neither take any action based upon its content, nor copy nor show it to anyone.
Covatic is a limited company registered in England & Wales. Registration number: 10422855. Registered office: 412-413 Scott House, The Custard Factory, Gibb Street, Birmingham, B9 4AA.
OK with 4.0.0rc2 I still get
details
Container started: 731bdd5d5ae4
Waiting for container <Container: 731bdd5d5ae4> with image localstack/localstack:2.0.1 to be ready ...
test setup failed
@pytest.fixture(scope='session', autouse=True)
def localstack():
with LocalStackContainer(
image="localstack/localstack:2.0.1"
).with_bind_ports(
4566, 4566) as localstack:
resp = urllib.request.urlopen(f"{localstack.get_url()}/health")
services = json.loads(resp.read().decode())["services"]
# Check that all services are running
# assert all(value == "available" for value in services.values())
# Check that some of the services keys
assert all(test_service in services for test_service in ["dynamodb", "sns", "sqs"])
> s3 = localstack.get_client("s3")
E AttributeError: 'LocalStackContainer' object has no attribute 'get_client'
integration/test_pipelines.py:229: AttributeError
there is a fix for this already submitted. https://github.com/testcontainers/testcontainers-python/issues/372
running with 4.0.0.rc2 with patched localstack I get the following error with mongo
-------------------------------- live log setup --------------------------------
2024-02-14 14:25:59 [ INFO] Pulling image localstack/localstack:2.0.1 (container.py:60)
2024-02-14 14:26:00 [ INFO] Container started: ac4f215217f6 (container.py:66)
2024-02-14 14:26:01 [ INFO] Waiting for container <Container: ac4f215217f6> with image localstack/localstack:2.0.1 to be ready ... (waiting_utils.py:52)
2024-02-14 14:26:01 [ INFO] Waiting for container <Container: ac4f215217f6> with image localstack/localstack:2.0.1 to be ready ... (waiting_utils.py:52)
2024-02-14 14:26:02 [ INFO] Pulling image mongo:latest (container.py:60)
ERROR [ 92%]Pulling image localstack/localstack:2.0.1
Container started: ac4f215217f6
Waiting for container <Container: ac4f215217f6> with image localstack/localstack:2.0.1 to be ready ...
Waiting for container <Container: ac4f215217f6> with image localstack/localstack:2.0.1 to be ready ...
Pulling image mongo:latest
test setup failed
@pytest.fixture(scope='session', autouse=True)
def mongo_client():
> with MongoDbContainer("mongo:latest", username='admin', password='admin').with_bind_ports(
27017, 27020) as mongo_client1:
integration/test_pipelines.py:212:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../venv/lib/python3.10/site-packages/testcontainers/core/container.py:73: in __enter__
return self.start()
../venv/lib/python3.10/site-packages/testcontainers/core/generic.py:57: in start
super().start()
../venv/lib/python3.10/site-packages/testcontainers/core/container.py:62: in start
self._container = docker_client.run(
../venv/lib/python3.10/site-packages/testcontainers/core/docker_client.py:50: in run
container = self.client.containers.run(
../venv/lib/python3.10/site-packages/docker/models/containers.py:873: in run
container = self.create(image=image, command=command,
../venv/lib/python3.10/site-packages/docker/models/containers.py:931: in create
create_kwargs = _create_container_args(kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
kwargs = {'password': 'admin', 'username': 'admin'}
def _create_container_args(kwargs):
"""
Convert arguments to create() to arguments to create_container().
"""
# Copy over kwargs which can be copied directly
create_kwargs = {}
for key in copy.copy(kwargs):
if key in RUN_CREATE_KWARGS:
create_kwargs[key] = kwargs.pop(key)
host_config_kwargs = {}
for key in copy.copy(kwargs):
if key in RUN_HOST_CONFIG_KWARGS:
host_config_kwargs[key] = kwargs.pop(key)
# Process kwargs which are split over both create and host_config
ports = kwargs.pop('ports', {})
if ports:
host_config_kwargs['port_bindings'] = ports
volumes = kwargs.pop('volumes', {})
if volumes:
host_config_kwargs['binds'] = volumes
network = kwargs.pop('network', None)
networking_config = kwargs.pop('networking_config', None)
if network:
if networking_config:
# Sanity check: check if the network is defined in the
# networking config dict, otherwise switch to None
if network not in networking_config:
networking_config = None
create_kwargs['networking_config'] = NetworkingConfig(
networking_config
) if networking_config else {network: None}
host_config_kwargs['network_mode'] = network
# All kwargs should have been consumed by this point, so raise
# error if any are left
if kwargs:
> raise create_unexpected_kwargs_error('run', kwargs)
E TypeError: run() got unexpected keyword arguments 'password', 'username'
../venv/lib/python3.10/site-packages/docker/models/containers.py:1159: TypeError
If I remove the username and password arguments I am then left with the original error
-------------------------------- live log setup --------------------------------
2024-02-14 14:29:04 [ INFO] Pulling image localstack/localstack:2.0.1 (container.py:60)
2024-02-14 14:29:04 [ INFO] Container started: 298bdc4bd535 (container.py:66)
2024-02-14 14:29:05 [ INFO] Waiting for container <Container: 298bdc4bd535> with image localstack/localstack:2.0.1 to be ready ... (waiting_utils.py:52)
2024-02-14 14:29:06 [ INFO] Waiting for container <Container: 298bdc4bd535> with image localstack/localstack:2.0.1 to be ready ... (waiting_utils.py:52)
2024-02-14 14:29:06 [ INFO] Pulling image mongo:latest (container.py:60)
2024-02-14 14:29:06 [ INFO] Container started: b176a12044af (container.py:66)
2024-02-14 14:29:06 [ INFO] Waiting for container <Container: b176a12044af> with image mongo:latest to be ready ... (waiting_utils.py:52)
2024-02-14 14:29:06 [ INFO] Waiting for container <Container: b176a12044af> with image mongo:latest to be ready ... (waiting_utils.py:52)
ERROR [ 92%]Pulling image localstack/localstack:2.0.1
Container started: 298bdc4bd535
Waiting for container <Container: 298bdc4bd535> with image localstack/localstack:2.0.1 to be ready ...
Waiting for container <Container: 298bdc4bd535> with image localstack/localstack:2.0.1 to be ready ...
Pulling image mongo:latest
Container started: b176a12044af
Waiting for container <Container: b176a12044af> with image mongo:latest to be ready ...
Waiting for container <Container: b176a12044af> with image mongo:latest to be ready ...
test setup failed
@pytest.fixture(scope='session', autouse=True)
def mongo_client1):
> with MongoDbContainer("mongo:latest").with_bind_ports(
27017, 27020) as mongo_client1:
integration/test_pipelines.py:212:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../venv/lib/python3.10/site-packages/testcontainers/core/container.py:73: in __enter__
return self.start()
../venv/lib/python3.10/site-packages/testcontainers/core/generic.py:58: in start
self._connect()
../venv/lib/python3.10/site-packages/testcontainers/core/waiting_utils.py:60: in wrapper
return wrapped(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <testcontainers.mongodb.MongoDbContainer object at 0x29406b1c0>
@wait_container_is_ready()
def _connect(self):
from pymongo import MongoClient
> return MongoClient(self.get_connection_url())
E TypeError: object.__new__() takes exactly one argument (the type to instantiate)
../venv/lib/python3.10/site-packages/testcontainers/mongodb/__init__.py:78: TypeError
This will fail running as pytest. if you patch localstack with get_client() method then you will get a username and password error and then if you remove the username and password you will get the original error above
import json
import urllib
import pytest
from testcontainers.localstack import LocalStackContainer
from testcontainers.mongodb import MongoDbContainer
MONGO_LATEST = "mongo:latest"
S3_FOLDER = "test-bucket"
def test_client_and_brand(mongo_client, mongo_london, mongo_client2, mongo_client1, localstack):
assert localstack.get_url() is not None
assert mongo_client.get_connection_url() is not None
assert mongo_london.get_connection_url() is not None
assert mongo_client2.get_connection_url() is not None
assert mongo_client1.get_connection_url() is not None
@pytest.fixture(scope='session', autouse=True)
def mongo_client():
with MongoDbContainer(MONGO_LATEST, username='admin', password='admin').with_bind_ports(
27017, 27017) as mongo_client:
yield mongo_client
@pytest.fixture(scope='session', autouse=True)
def mongo_london():
with MongoDbContainer(MONGO_LATEST, username='admin', password='admin').with_bind_ports(
27017, 27018) as mongo_london:
yield mongo_london
@pytest.fixture(scope='session', autouse=True)
def mongo_client2():
with MongoDbContainer(MONGO_LATEST, username='admin', password='admin').with_bind_ports(
27017, 27019) as mongo_client2:
yield mongo_client2
@pytest.fixture(scope='session', autouse=True)
def mongo_client1():
with MongoDbContainer("mongo:latest").with_bind_ports(
27017, 27020) as mongo_client1:
yield mongo_client1
@pytest.fixture(scope='session', autouse=True)
def localstack():
with LocalStackContainer(
image="localstack/localstack:2.0.1"
).with_bind_ports(
4566, 4566) as localstack:
resp = urllib.request.urlopen(f"{localstack.get_url()}/health")
services = json.loads(resp.read().decode())["services"]
# Check that all services are running
# assert all(value == "available" for value in services.values())
# Check that some of the services keys
assert all(test_service in services for test_service in ["dynamodb", "sns", "sqs"])
s3 = localstack.get_client("s3")
s3.create_bucket(
Bucket=S3_FOLDER,
CreateBucketConfiguration={"LocationConstraint": "us-west-1"},
)
yield localstack
The fix for mongodb is
mongodb/testcontainers/mongodb/init.py
@wait_container_is_ready()
def _connect(self) -> MongoClient:
mc = MongoClient()
mc.__init__(self.get_connection_url())
return mc
@alexanderankin hope that covers it :). Now going to see if the docker issue has been fixed in RC2 as I have patched core as well
fix for testcontainers-localstack is in main branch but there has not been a release for the feature
@covatic-john we've released 4.0 :tada: can we then close this issue? Has everything been solved for you?
Does 4.0 include updates on the features, localstack[mongodb} or Localstack?
I’ll test this morning
On 7 Mar 2024, at 09:41, Bálint Bartha @.***> wrote:
@covatic-john https://github.com/covatic-john we've released 4.0 🎉 can we then close this issue? Has everything been solved for you?
— Reply to this email directly, view it on GitHub https://github.com/testcontainers/testcontainers-python/issues/401#issuecomment-1983109814, or unsubscribe https://github.com/notifications/unsubscribe-auth/BC3KZVM3NWHZAECMYA3XXK3YXAY5LAVCNFSM6AAAAABAJJE3AWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBTGEYDSOBRGQ. You are receiving this because you were mentioned.
--
This email and any attachments are confidential, may contain privileged material and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Covatic. If you are not the intended recipient of this email, you must neither take any action based upon its content, nor copy nor show it to anyone.
Covatic is a limited company registered in England & Wales. Registration number: 10422855. Registered office: 412-413 Scott House, The Custard Factory, Gibb Street, Birmingham, B9 4AA.
Does 4.0 include updates on the features, localstack[mongodb} or Localstack? I’ll test this morning
Thanks for the help! Not enough time to fix/test everything personally. :green_heart:
4.0.0 tested Still not fixed needs a release of testcontainers-mongodb pypi still has 0.0.1rc1. will try installing from main branch
It's "testcontainers[mongodb]" now
- There is an unreleased fix for mongo waiting - I can merge now and release it
- There is an open pr for the constructor fix you suggested
On Mon, Mar 11, 2024, 5:54 AM covatic-john @.***> wrote:
4.0.0 tested Still not fixed needs a release of testcontainers-mongodb
— Reply to this email directly, view it on GitHub https://github.com/testcontainers/testcontainers-python/issues/401#issuecomment-1988016466, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACECGJEH3P4ZEN7NEH2YYD3YXV5OVAVCNFSM6AAAAABAJJE3AWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBYGAYTMNBWGY . You are receiving this because you were mentioned.Message ID: @.***>
@alexanderankin sorry was busy last week with a P1 and migrations. merging and release would be fantastic 👍
Localstack giving me no issue now with new release :)
I messed up and forgot to check the PR title for the postgres pr, so now I need to fix the release pr. On the bright side if I'm able to do that, theres no other merges needed for the next release.
Let's give it a day or two to see if @totallyzen has any ideas about fixing or not, I think i can figure it out tonight (I'm est tz) maybe
On Mon, Mar 11, 2024, 6:30 AM covatic-john @.***> wrote:
@alexanderankin https://github.com/alexanderankin sorry was busy last week with a P1 and migrations. merging and release would be fantastic 👍
Localstack giving me no issue now with new release :)
— Reply to this email directly, view it on GitHub https://github.com/testcontainers/testcontainers-python/issues/401#issuecomment-1988095096, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACECGJCHVQDULVAUO22P3OLYXWBUHAVCNFSM6AAAAABAJJE3AWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBYGA4TKMBZGY . You are receiving this because you were mentioned.Message ID: @.***>
4.0.1 is out (maybe at the expense of our release-please pipeline)
can install testcontainers[mongodb] and see if anything is fixed if not then #448 is back on the table
nope still same error
integration/test_pipelines.py:53:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
integration/helpers.py:137: in setup_mongo_clients_aws_s3
london_db = mongo_london.get_connection_client().serendipity_dev
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <testcontainers.mongodb.MongoDbContainer object at 0x28c9cd0c0>
def get_connection_client(self) -> MongoClient:
> return MongoClient(self.get_connection_url())
E TypeError: object.__new__() takes exactly one argument (the type to instantiate)
../venv/lib/python3.10/site-packages/testcontainers/mongodb/__init__.py:88: TypeError
Tested this with the latest release still not working, this fixes the issue it's to do with how you are initialising the mongoclient
I know not pythonistic but it works :)
def get_connection_client(self) -> MongoClient:
mc = MongoClient()
mc.__init__(self.get_connection_url())
return mc
Merged another fix here https://github.com/testcontainers/testcontainers-python/pull/448, please check it out when it is released in the next package version.
When that happens, please provide:
- OS and version / build number
- Testcontainers version. Do not blindly trust the version you specify in your setup. Check it programmatically with a short snippet of
from importlib.metadata import version
print(version("testcontainers"))
