Build container from schema

[torotil]

I’ve discovered this project while looking for Python dependency injection solutions. So far it seems to be the most promising one.

One thing I’m looking for is to specify dependencies in a serializable format. Something that could be read or written to easily to a YAML file. Is this possilbe with this package? If not: What would be the most likely way to implement it?

[rmk135]

Hi @torotil , take a look at https://python-dependency-injector.ets-labs.org/providers/configuration.html#loading-from-a-yaml-file

[torotil]

Hi @torotil , take a look at https://python-dependency-injector.ets-labs.org/providers/configuration.html#loading-from-a-yaml-file

Thanks. Seems I have put the question in a misleading way. What I meant with “container config” is the specs of how to instantiate services (providers), not some config values. Is there a way to put those into YAML as well?

[rmk135]

Got it. There is no way to do it out of the box. Take a look at DynamicContainer example here: https://python-dependency-injector.ets-labs.org/containers/dynamic.html

[rmk135]

@torotil I thought about your question and don't see anything that prevents adding this feature in the framework. I'll convert this into into a feature. Let me know if you'd like to participate as early adopter.

[mysticfall]

I also have the same requirements as desribed by the issue and I'd be more than happy to be able to test the feature as an early adopter when it becomes available.

By the way, I also considered using DynamicContainer but I decided against it once I learned that it does not support wiring like its declarative counterpart.

I think it'd be ideal if there's a way to construct a container from Configuration, and possibly support doing so in a cascading manner (e.g. constructing a container from a given value of a Configuration, then allow the new container to construct its child containers in the same manner from a chosen subtree of its parent configuration tree, and so on).

[rmk135]

@mysticfall , got it, sounds good. The feature is in the backlog. Will begin working on it as done with earlier backlog items.

[rmk135]

Started working on this feature. Here is a sample schema for single container example:

version: "1"

providers:
  config:
    provider: Configuration

  logging:
    provider: Resource
    provides: logging.config.fileConfig
    kwargs:
      fname: ./example/logging.ini

  database_client:
    provider: Singleton
    provides: sqlite3.connect
    args:
      - config.database.dsn

  s3_client:
    provider: Singleton
    provides: boto3.client
    kwargs:
      service_name: s3
      aws_access_key_id: config.aws.access_key_id
      aws_secret_access_key: config.aws.secret_access_key

  user_service:
    provider: Factory
    provides: example.services.UserService
    kwargs:
      db: database_client

  auth_service:
    provider: Factory
    provides: example.services.AuthService
    kwargs:
      db: database_client
      token_ttl: config.auth.token_ttl.as_int()

  photo_service:
    provider: Factory
    provides: example.services.PhotoService
    kwargs:
      db: database_client
      s3: s3_client

Appreciate you feedback @torotil @mysticfall

[torotil]

Awesome, thanks! I’ll try to look into it soon.

[mysticfall]

I think it looks very useful already and thanks much for the great work! :)

On a side note, I originally asked for a feature that would allow me to describe a nested service structure. The example above doesn't seem to cover such a case yet, but I also realised that it may not be a good idea to mix service configurations and user preferences/settings.

To clarify, this was the configuration that I initially considered to implement using this feature:

• https://github.com/mysticfall/alleycat/blob/master/config.json

As you see, it's used to construct and configure a nested structure of components, so that an InputBinding named menu contains an Input of type key_press and so on.

Currently, I configured an InputBindingFactory and InputFactory using DI like this:

• https://github.com/mysticfall/alleycat/blob/master/alleycat/input/context.py

And I handled the nested instantiation of services by a custom code (i.e. from_config methods on various types) which I initially thought might be replaced by this proposed feature. But now I'm leaning toward keeping the custom code as it is, but using the new feature to separate service configuration from user preferences.

[rmk135]

@torotil Thanks

[rmk135]

@mysticfall thanks for the feedback and sharing the use case. I don't have a sample like you need right now, but I'll make sure this use case is supported. I have a sample schema with nested containers. That's not exactly what you're looking for. Could be useful though:

version: "1"

providers:

  core:
    provider: Container
    providers:
      config:
        provider: Configuration

      logging:
        provider: Resource
        provides: logging.config.fileConfig
        kwargs:
          fname: ./example/logging.ini

  gateways:
    provider: Container
    providers:
      database_client:
        provider: Singleton
        provides: sqlite3.connect
        args:
          - core.config.database.dsn

      s3_client:
        provider: Singleton
        provides: boto3.client
        kwargs:
          service_name: s3
          aws_access_key_id: core.config.aws.access_key_id
          aws_secret_access_key: core.config.aws.secret_access_key

  services:
    provider: Container
    providers:
      user:
        provider: Factory
        provides: example.services.UserService
        kwargs:
          db: gateways.database_client

      auth:
        provider: Factory
        provides: example.services.AuthService
        kwargs:
          db: gateways.database_client
          token_ttl: core.config.auth.token_ttl.as_int()

      photo:
        provider: Factory
        provides: example.services.PhotoService
        kwargs:
          db: gateways.database_client
          s3: gateways.s3_client

[rmk135]

@mysticfall I tried to build your json with current yaml syntax. That's what I got:

version: "1"

providers:

 input:
   provider: Container
   providers:

     general:
       provider: Container
       providers:
         menu:
           provider: Factory
           provides: sample.module.TriggerClass
           kwargs:
             name: "Show Menu"
             description: "Toggle the main menu."
             input:
               provider: Factory
               provides: sample.module.KeyPress
               kwargs:
                 keycode: "ESCKEY"

     view:
       provider: Container
       providers:

         rotate:
           provider: Factory
           provides: sample.module.Axis2D
           kwargs:
             name: "Look Around"
             description: "Rotate the current view."
             input:
               provider: Dict
               kwargs:
                 x:
                   provider: Factory
                   provides: sample.module.MouseAxis
                   kwargs:
                     axis: "x"
                     sensitivity: 0.4
                 y:
                   provider: Factory
                   provides: sample.module.MouseAxis
                   kwargs:
                     axis: "y"
                     sensitivity: 0.4

         move:
           provider: Factory
           provides: sample.module.Axis2D
           kwargs:
             name: "Move"
             description: "Move the camera."
             input:
               provider: Dict
               kwargs:
                 x:
                   provider: Factory
                   provides: sample.module.KeyAxis
                   kwargs:
                     positive_key: "AKEY"
                     negative_key: "DKEY"
                     sensitivity: 1
                 y:
                   provider: Factory
                   provides: sample.module.KeyAxis
                   kwargs:
                     positive_key: "WKEY"
                     negative_key: "SKEY"
                     window_size: 1
                     window_shift: 0.05
                     sensitivity: 1

How does it look to you?

[mysticfall]

Yes, it looks great to me. Although, I'm not determined if I should rewrite my current implementation with it since it could be a bit more suitable to be user settings (c.f. service configuration) and supports schema validation (via JSON Schema) also, I think the proposed feature is good enough to cover my use case.

Thanks again for the great work!

[rmk135]

@mysticfall , thanks for the feedback. I keep working on it and I have some updates.

1. I plan to support 3 formats: raw dicts, yaml and json. Container will have 3 corresponding methods:

container.from_schema({})
container.from_yaml_schema('schema.yml')
container.from_json_schema('schema.json')

2. I simplified the format a little bit:

version: "1"

container:

  input:
    general:
      menu:
        provider: Factory
        provides: sample.module.TriggerClass
        kwargs:
          name: "Show Menu"
          description: "Toggle the main menu."
          input:
            provider: Factory
            provides: sample.module.KeyPress
            kwargs:
              keycode: "ESCKEY"

      view:
        rotate:
          provider: Factory
          provides: sample.module.Axis2D
          kwargs:
            name: "Look Around"
            description: "Rotate the current view."
            input:
              provider: Dict
              kwargs:
                x:
                  provider: Factory
                  provides: sample.module.MouseAxis
                  kwargs:
                    axis: "x"
                    sensitivity: 0.4
                y:
                  provider: Factory
                  provides: sample.module.MouseAxis
                  kwargs:
                    axis: "y"
                    sensitivity: 0.4

        move:
          provider: Factory
          provides: sample.module.Axis2D
          kwargs:
            name: "Move"
            description: "Move the camera."
            input:
              provider: Dict
              kwargs:
                x:
                  provider: Factory
                  provides: sample.module.KeyAxis
                  kwargs:
                    positive_key: "AKEY"
                    negative_key: "DKEY"
                    sensitivity: 1
                y:
                  provider: Factory
                  provides: sample.module.KeyAxis
                  kwargs:
                    positive_key: "WKEY"
                    negative_key: "SKEY"
                    window_size: 1
                    window_shift: 0.05
                    sensitivity: 1

I understand that it's questionable if you should migrate to this format from what you have. Seems like your schema has a bit different role. I'm not sure if that simplifies your implementation, but may be you could do such transformation: Your schema format -> DI dict schema -> container.from_schema().

Thanks a lot again for participating the design discussion. Really appreciate it!

[robmoore]

I've attempted unsuccessfully to populate the container first with some providers and then call from_schema with the intention of referencing providers outside of the schema from within the schema. The goal is to use the schema configuration for only a minority of the objects. Is this a use case that you are planning to support?

[rmk135]

Hey @robmoore , this wasn't on my mind before, but this makes sense, so I'll make a note. Many thanks again for sharing your experience.

[rowan-maclachlan]

Hello there. I'm just curious about this feature. I could imagine a use-case myself for specifying a container and providers via yaml or json.