Some use cases:

1. override one value

assign a gpu for a runner

BENTOML_CONFIG_OPTIONS='runners.pytorch_mnist.resources."nvidia.com/gpu"=1' bentoml serve pytorch_mnist_demo --production

equal to

runners:
    pytorch_mnist:
        resources:
            nvidia.com/gpu: 1

2. override multiple values (divide with space or new line)

assign two specified gpus to a runner

BENTOML_CONFIG_OPTIONS='runners.pytorch_mnist.resources."nvidia.com/gpu"=[0,2]' bentoml serve pytorch_mnist_demo --production

or

BENTOML_CONFIG_OPTIONS='runners.pytorch_mnist.resources."nvidia.com/gpu"[0]=0 runners.pytorch_mnist.resources."nvidia.com/gpu"[1]=2' bentoml serve pytorch_mnist_demo --production

equal to

runners:
    pytorch_mnist:
        resources:
            nvidia.com/gpu:
                - 0
                - 2

or

runners:
    pytorch_mnist:
        resources:
            nvidia.com/gpu: [0, 2]

Codecov Report

Merging #3018 (1ba4603) into main (a69764e) will decrease coverage by 1.91%. The diff coverage is 26.78%.

@@            Coverage Diff             @@
##             main    #3018      +/-   ##
==========================================
- Coverage   62.89%   60.98%   -1.92%     
==========================================
  Files         145      132      -13     
  Lines       11828    11538     -290     
  Branches        0     1913    +1913     
==========================================
- Hits         7439     7036     -403     
+ Misses       4389     4133     -256     
- Partials        0      369     +369     

Impacted Files	Coverage Δ
bentoml/_internal/utils/__init__.py	65.84% <11.11%> (-6.21%)	:arrow_down:
bentoml/_internal/utils/unflatten.py	21.84% <21.84%> (ø)
bentoml/_internal/configuration/containers.py	69.87% <53.84%> (-7.27%)	:arrow_down:
bentoml/_internal/configuration/__init__.py	70.88% <60.00%> (-6.04%)	:arrow_down:
bentoml/_internal/utils/buildx.py	24.48% <0.00%> (-23.84%)	:arrow_down:
bentoml/_internal/service/inference_api.py	56.25% <0.00%> (-13.75%)	:arrow_down:
bentoml/_internal/utils/uri.py	78.94% <0.00%> (-10.53%)	:arrow_down:
bentoml/_internal/server/http/access.py	85.50% <0.00%> (-10.27%)	:arrow_down:
bentoml/_internal/utils/pkg.py	81.81% <0.00%> (-10.19%)	:arrow_down:
bentoml/_internal/utils/cattr.py	71.42% <0.00%> (-9.53%)	:arrow_down:
... and 90 more

I'm thinking to pass it as a config-options

bentoml serve --production --config-options 'runners.pytorch_mnist.resources."nvidia.com/gpu"=[0,2]' --config-options "api_server.http.metrics.enabled=false"