firedantic icon indicating copy to clipboard operation
firedantic copied to clipboard

Support for multiple firestore clients

Open mfisher29 opened this issue 4 months ago • 2 comments

This pull request intends to resolve issue #86.

I have followed the discussion there and created what I believe to be a nice solution based on Option 3.

Primary things to note about this solution:

  • The current way of configuring a client/db via the configure() method remains in tact.
  • Support for multiple clients builds on top of that and relies on storing of a ConfigItem in the CONFIGURATIONS dict.
  • Upon creation of a config, both a sync and async client will be set for that config
  • Code examples introduced below are included in a multiple_clients_examples directory, and can be removed before merge.
  • I have added ~20 more unit tests, but there may be some gaps. Happy to add more with your suggestions.

Other small things:

  • readme was updated with some more small details, and I recommend using pytest instead for development. It may be that I don't know poetry well enough to fix this, but I found that upon running poetry run invoke test my current changes were rolled back to the current firedantic version. I suppose this is because poetry is locked to version = "0.11.0" in the pyproject.toml and so I used pytest instead.
  • Aside from testing support of multiple clients, some test cases were added to check more types of operands, i.e. op.EQ and "==". All work as expected.
  • Apologies for the added spacing everywhere. I believe one of the formatters from the poetry setup did that. If it bothers anyone i can go back and remove it.

Now for the examples...

Example of client creation:

## OLD WAY:
def configure_client():
    # Firestore emulator must be running if using locally.
    if environ.get("FIRESTORE_EMULATOR_HOST"):
        client = Client(
            project="firedantic-test",
            credentials=Mock(spec=google.auth.credentials.Credentials),
        )
    else:
        client = Client()

    configure(client, prefix="firedantic-test-")
    print(client)


def configure_async_client():
    # Firestore emulator must be running if using locally.
    if environ.get("FIRESTORE_EMULATOR_HOST"):
        client = AsyncClient(
            project="firedantic-test",
            credentials=Mock(spec=google.auth.credentials.Credentials),
        )
    else:
        client = AsyncClient()

    configure(client, prefix="firedantic-test-")
    print(client)


## NEW WAY:
def configure_multiple_clients():
    config = Configuration()

    # name = (default)
    config.create(
        prefix="firedantic-test-",
        project="firedantic-test",
        credentials=Mock(spec=google.auth.credentials.Credentials),
    )

    # name = billing
    config.create(
        name="billing",
        prefix="test-billing-",
        project="test-billing",
        credentials=Mock(spec=google.auth.credentials.Credentials),
    )

    config.get_client()  ## will pull the default client
    config.get_client("billing")  ## will pull the billing client
    config.get_async_client("billing")  ## will pull the billing async client

Under the hood of config.create():

  • (seeconfiguration.py for more details) Created suggested Configuration class for added support of multiple configurations/clients.
  • The general idea is to save each configuration to the CONFGURATIONS Dict. When using the singular configure method, it will be saved there as (default).
  • New single clients can also be configured with the new multi client method. Multi is not required to use multi.
class Configuration:
    def __init__(self):
        self.configurations: Dict[str, ConfigItem] = {}

    def create(
        self,
        name: str = "(default)",
        prefix: str = "",
        project: str = "",
        credentials: Credentials = None,
    ) -> None:
        self.configurations[name] = ConfigItem(
            prefix=prefix,
            project=project,
            credentials=credentials,
            client=Client(
                project=project,
                credentials=credentials,
            ),
            async_client=AsyncClient(
                project=project,
                credentials=credentials,
            ),
        )
        # add to global CONFIGURATIONS
        global CONFIGURATIONS
        CONFIGURATIONS[name] = self.configurations[name]

For saving/finding entries:

## With single client
def old_way():
    # Firestore emulator must be running if using locally.
    configure_client()

    # Now you can use the model to save it to Firestore
    owner = Owner(first_name="John", last_name="Doe")
    company = Company(company_id="1234567-8", owner=owner)
    company.save()

    # Prints out the firestore ID of the Company model
    print(f"\nFirestore ID: {company.id}")

    # Reloads model data from the database
    company.reload()


## Now with multiple clients/dbs:
def new_way():
    config = Configuration()

    # 1. Create first config with config_name = "(default)"
    config.create(
        prefix="firedantic-test-",
        project="firedantic-test",
        credentials=Mock(spec=google.auth.credentials.Credentials),
    )

    # Now you can use the model to save it to Firestore
    owner = Owner(first_name="Alice", last_name="Begone")
    company = Company(company_id="1234567-9", owner=owner)
    company.save()  # will use 'default' as config name

    # Reloads model data from the database
    company.reload()  # with no name supplied, config refers to "(default)"

    # 2. Create the second config with config_name = "billing"
    config_name = "billing"
    config.create(
        name=config_name,
        prefix="test-billing-",
        project="test-billing",
        credentials=Mock(spec=google.auth.credentials.Credentials),
    )

    # Now you can use the model to save it to Firestore
    account = BillingAccount(name="ABC Billing", billing_id="801048", owner="MFisher")
    bc = BillingCompany(company_id="1234567-8", billing_account=account)

    bc.save(config_name)

    # Reloads model data from the database
    bc.reload(config_name)  # with config name supplied, config refers to "billing"

    # 3. Finding data
    # Can retrieve info from either database/client
    # When config is not specified, it will default to '(default)' config
    # The models do not know which config you intended to use them for, and they
    # could be used for a multitude of configurations at once.
    print(Company.find({"owner.first_name": "Alice"}))

    print(BillingCompany.find({"company_id": "1234567-8"}, config=config_name))

    print(
        BillingCompany.find(
            {"billing_account.billing_id": "801048"}, config=config_name
        )
    )

A simplified example for the find() method: Note here, the multi client method is using the (default) config, as no config was specified.

print("\n---- Running OLD way ----")
configure_client()

companies1 = Company.find({"owner.first_name": "John"})
companies2 = Company.find({"owner.first_name": {op.EQ: "John"}})
companies3 = Company.find({"owner.first_name": {"==": "John"}})
assert companies1 != []
assert companies1 == companies2 == companies3


print("\n---- Running NEW way ----")
configure_multiple_clients()

companies1 = Company.find({"owner.first_name": "Alice"})
companies2 = Company.find({"owner.first_name": {op.EQ: "Alice"}})
companies3 = Company.find({"owner.first_name": {"==": "Alice"}})
assert companies1 != []
assert companies1 == companies2 == companies3

mfisher29 avatar Aug 27 '25 03:08 mfisher29

A big thanks for the PR!

I'm really sorry to say we're pretty busy for the next month, so going to frankly say we're most likely not able to have a deeper look into the PR within the next month.

joakimnordling avatar Sep 03 '25 11:09 joakimnordling

Hi @joakimnordling , Thanks so much for the detailed review and suggestions. I've been MIA on this since your message but I should have time next week to give it another go. Will keep you posted. M

mfisher29 avatar Nov 13 '25 14:11 mfisher29