[Feature Suggestion] Deployment Parameters: Allow unused paramaters in deployment

[mfeyx]

Is your feature request related to a problem? Please describe

I want to deploy resources based on a params.json file. If unused parameters are present the deployment currently fails:

InvalidTemplate - Deployment template validation failed: 'The template parameters 'rg_tags' in the parameters file are not valid; they are not present in the original template and can therefore not be provided at deployment time. The only supported parameters for this template are 'name, tags'.

main.bicep

param name string
param tags object

resource storage 'Microsoft.Storage/storageAccounts@2021-06-01' = {
  name: name
  location: resourceGroup().location
  kind: 'StorageV2'
  sku: {
    name: 'Standard_ZRS'
  }
  properties: {
    accessTier: 'Cool'
  }

  tags: tags
}

params.json

{
  "id": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "title": "Parameters",
  "description": "An Azure deployment parameter file",
  "type": "object",
  "parameters": {
    "name": {
      "value": "mystoragename"
    },
    "rg_tags": {
      "value": {
        "businessowner": "tba",
        "project": "tba"
      }
    },
    "tags": {
      "value": {
        "application": "XYZ",
        "version": "0.0.1"
      }
    }
  }
}

Describe the solution you'd like

Basically, I want to deploy resources without having to delete parameters in the parameters file. In other words, I want to provide as many parameters in my configuration as I want, but use only a few of them in a specific deployment. Hence, I want to be able to define parameters even though they might not be used in a deployment. When I deploy a resource group, for instance, I will use my rg_tags values from the parameters, but not the normal tags that might be used in other deployments.

Additional context

This is the command I use:

az deployment group what-if --resource-group "MY_RESOURCE_GROUP" -f "main.bicep" --parameters "params.json"

Hope this is the right place for the suggestion. I already opened an issue here, but this place seems to be a better fit.

Kind regards :)

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @armleads-azure.

Issue Details

---

Is your feature request related to a problem? Please describe

I want to deploy resources based on a params.json file. If unused parameters are present the deployment currently fails:

InvalidTemplate - Deployment template validation failed: 'The template parameters 'rg_tags' in the parameters file are not valid; they are not present in the original template and can therefore not be provided at deployment time. The only supported parameters for this template are 'name, tags'.

main.bicep

param name string
param tags object

resource storage 'Microsoft.Storage/storageAccounts@2021-06-01' = {
  name: name
  location: resourceGroup().location
  kind: 'StorageV2'
  sku: {
    name: 'Standard_ZRS'
  }
  properties: {
    accessTier: 'Cool'
  }

  tags: tags
}

params.json

{
  "id": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "title": "Parameters",
  "description": "An Azure deployment parameter file",
  "type": "object",
  "parameters": {
    "name": {
      "value": "mystoragename"
    },
    "rg_tags": {
      "value": {
        "businessowner": "tba",
        "project": "tba"
      }
    },
    "tags": {
      "value": {
        "application": "XYZ",
        "version": "0.0.1"
      }
    }
  }
}

Describe the solution you'd like

Basically, I want to deploy resources without having to delete parameters in the parameters file. In other words, I want to provide as many parameters in my configuration as I want, but use only a few of them in a specific deployment. Hence, I want to be able to define parameters even though they might not be used in a deployment. When I deploy a resource group, for instance, I will use my rg_tags values from the parameters, but not the normal tags that might be used in other deployments.

Additional context

This is the command I use:

az deployment group what-if --resource-group "MY_RESOURCE_GROUP" -f "main.bicep" --parameters "params.json"

Hope this is the right place for the suggestion. I already opened an issue here, but this place seems to be a better fit.

Kind regards :)

Author:	mfeyx
Assignees:	-
Labels:	Service Attention, ARM
Milestone:	-

[yonzhan]

route to service team

[j-oliver]

Is there any progress here? Providing a parameters file with unused variables shouldn't be a problem. I right now have to parse out the correct parameters of my parameters.json file just to not run into a

Invalid Template: Deployment template validation failed: 'The template parameters 'a,b,c' in the parameters file are not valid; they are not present in the original template and can therefore not be provided at deployment time. The only supported parameters for this template are 'a,b'

error. In this case I would expect the parameter c to just be ignored.

[cata008]

It would be very useful to have this feature implemented as it would allow to have a centralized parameter file with all the values in there. That could be passed in a yml pipeline and sent to the deployment file where only the needed params would be loaded and the rest ignored.

[ajarvinen]

This would be very usefull for me also. I have a deployment that uses three different bicep templates, which are called with a single parameter file. The templates have some of the same parameters, but few different also.

To work around this limitation I have created dummy parameters for the bicep templates. This means everytime I want to add a new parameter for any of the templates, I also need to add the Dummy params for the other two.

Could the az deployment command just have a flag --ignore-unused-parameters=true ?

[mfeyx]

Hi,

maybe this is not the best solution to this request but I have written a script for the bicep deployment.

Folder Structure

.
├── RG1
│   ├── main.bicep
│   └── modules
├── RG2
│   ├── main.bicep
│   ├── modules
│   ├── params.json
│   └── params.secret.json
├── config.json
├── deploy.py
└── params.json

config.json

{
  "location": "northeurope",
  "subscriptionId": "000e0000-e00b-00d0-a000-000000000000"
}

params.json / params.secret.json

⚠️ put the params.secret.json in .gitignore

{
  "id": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "title": "Parameters",
  "description": "Azure deployment parameter file",
  "type": "object",
  "parameters": {
    "tags": {
      "value": {
        "key": "value"
      }
    }
  }
}

deploy.py

The script...

• takes the params.json from the root folder;
• then checks for a params.json file in the resource group folder, if available, both were merged: {**global, **project};
• then the main.bicep file is checked for params. The deploy will run if all params are found (a params.deployment.json file is created and deleted afterwards. It will only contain the params from main.bicep)

deployment commands

python deploy.py --help

script file

import os
import re
import json
import time
import argparse
import hashlib

from string import Template

# ---------------------------------------------------------------------------- #
#                                UTIL FUNCTIONS                                #
# ---------------------------------------------------------------------------- #


def walk_folder(folder: str) -> tuple:
    top_dir = list(os.walk(folder))
    root_dir = top_dir[0]
    return root_dir


def _print(msg: str or list) -> None:
    if not type(msg) == list:
        msg = [msg]
    for m in msg:
        print(m)


def xprint(msg: str | list, code=1) -> None:
    _print(msg)
    exit(code)


def nprint(msg: str or list, new_line="\n") -> None:
    if new_line:
        print(new_line)
    _print(msg)


def padding(val):
    val = str(val)
    if len(val) == 2:
        return val
    return f" {val}"


# ---------------------------------------------------------------------------- #
#                                DEFAULT VALUES                                #
# ---------------------------------------------------------------------------- #
# script default values
SEP = os.sep
ENV_ARG = "env"
ENCODING = "utf-8"
DEFAULT_VALUE = {"value": None}


# configuration files used for deploy.py
CONFIG_JSON = "config.json"
CONFIG_SECRET_JSON = "config.secret.json"

# configuration for bicep files
PARAMS_JSON = "params.json"
PARAMS_SECRET_JSON = "params.secret.json"

# tmp file
# will be generated by this script, deleted afterwards
# only used for deployment
PARAMS_DEPLOYMENT = "params.deploy.json"

# possible env values
DEV = "dev"
PROD = "prd"


# ---------------------------------------------------------------------------- #
#                               STRING TEMPLATES                               #
# ---------------------------------------------------------------------------- #

param_json_template = Template("""{
  "id": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "title": "Parameters",
  "description": "An Azure deployment parameter file",
  "type": "object",
  "parameters": $parameters
}""".strip())

command_template = Template("""
az deployment sub create --name $name --subscription $subscription_id -l $location -f $bicep_file
""".strip())

info_template = Template("""
DEPLOYMENT COMMAND
---------------------------
$command
""")


# ---------------------------------------------------------------------------- #
#                                ARGUMENT PARSER                               #
# ---------------------------------------------------------------------------- #

parser = argparse.ArgumentParser(
    description="Deploy Infrastructure to Azure Cloud.")

parser.add_argument("-e", f"--{ENV_ARG}",
                    help="Choose Deployment Environment", default=None)

parser.add_argument("-f", "--file",
                    help="Name of bicep file", default="main.bicep")

parser.add_argument("-g", "--group",
                    help="Resource Group folder for deployment, relative path", default=None)

parser.add_argument("-s", "--skip-preview", help="Run Deployment without Preview",
                    action=argparse.BooleanOptionalAction, default=False)

args = vars(parser.parse_args())
# print(args)


# ---------------------------------------------------------------------------- #
#                               CHECK FOR PROJECT                              #
# ---------------------------------------------------------------------------- #

global_root, global_folders, global_files = list(walk_folder('.'))
ROOT = args.get("group")
if not ROOT:
    try:
        project_folders = {}

        i = 0
        print("RESOURCE GROUPS")
        print("===============")
        for project in global_folders:
            i += 1
            print(f"{padding(i)} --> {project}")
            project_folders[f"{i}"] = project
        while not ROOT:
            project = str(input("\nSelect Resource Group Number: "))
            ROOT = project_folders.get(project)
    except KeyboardInterrupt:
        xprint(["None", "Abort Deployment."])

ROOT_PATH = f".{SEP}{ROOT}{SEP}"
print(f"Running Deployment for: {ROOT}")


# ---------------------------------------------------------------------------- #
#                              HANDLE CONFIG FILES                             #
# ---------------------------------------------------------------------------- #

project_root, project_folders, project_files = list(walk_folder(ROOT))

if not re.search(r"bicep", "|".join(project_files)):
    xprint("No Bicep File found")

config_global_json = f"{CONFIG_JSON}"
global_config = {}
if config_global_json in global_files:
    with open(config_global_json, "r") as f:
        global_config = json.load(f)

config_json = f"{ROOT_PATH}{CONFIG_JSON}"
project_config = {}
if CONFIG_JSON in project_files:
    with open(config_json, "r", encoding=ENCODING) as f:
        project_config = json.load(f)

# project overrides global
config = {**global_config, **project_config}


# ---------------------------------------------------------------------------- #
#                               INPUT VALIDATION                               #
# ---------------------------------------------------------------------------- #

# mandatory files: config.json, *.bicep file with resources
# mandatory fields in config.json: subscriptionId, location

# ------------------------------- CONFIGURATION ------------------------------ #

if not config:
    xprint(["No `config.json` found, or config is emtpy!",
           "Fields: [subscriptionId, location] are mandatory!"])

# ------------------------------ SUBSCRIPTION ID ----------------------------- #

subscription_id = config.get("subscriptionId")
if not subscription_id:
    xprint([
        "Missing Subscription ID.",
        "Make sure `config.json` exists with `subscriptionId` field."
    ])


# --------------------------------- LOCATION --------------------------------- #

location = config.get("location")
if not location:
    xprint("No Location specified. Add `location` to your `config.json` file.")

# -------------------------------- BICEP FILE -------------------------------- #

bicep_file = args.get('file')
if not bicep_file in project_files:
    xprint(f"Bicep file `{bicep_file}` not found in { project_files }")


# ---------------------------------------------------------------------------- #
#                        BUILDING DEPLOYMENT PARAMETERS                        #
# ---------------------------------------------------------------------------- #

bicep_file = f"{ROOT_PATH}{bicep_file}"
with open(bicep_file, "r") as b:
    # line := param <name> type = <default value>
    params = [line.strip().split(" ")[1] for line in b.readlines()
              if line.startswith("param") and len(line.strip().split(" ")) == 3]

deployment_json = None
if params:
    print(f"Params in Bicep: {params}")
    print("--> Building Deployment Parameters")
    deploy_parameters = {}

    # global params
    for json_file in [PARAMS_JSON, PARAMS_SECRET_JSON]:
        if json_file in global_files:
            params_json = json_file
            with open(params_json, "r") as f:
                params_content = json.load(f)

            parameters = params_content.get("parameters")
            for param in params:
                p = parameters.get(param)
                if p:
                    deploy_parameters[param] = p

    # project params
    for json_file in [PARAMS_JSON, PARAMS_SECRET_JSON]:
        if json_file in project_files:
            params_json = f"{ROOT_PATH}{json_file}"
            with open(params_json, "r") as f:
                params_content = json.load(f)

            parameters = params_content.get("parameters")
            for param in params:
                p = parameters.get(param)
                if p:
                    deploy_parameters[param] = p

    env_value = args.get(ENV_ARG)
    if env_value and (ENV_ARG in params):
        print("found env: {}".format(env_value))
        env_value = env_value.lower()
        is_production = re.search(r"pr.?d.*", env_value)
        env = PROD if is_production else DEV
        deploy_parameters[ENV_ARG] = {"value": env}

    # ---------------------------- VALIDATE PARAMS --------------------------- #

    missing_params = [
        param for param in params
        if param not in deploy_parameters.keys()
    ]

    if missing_params:
        if "env" in missing_params:
            env_value = input(
                "Choose Deployment Environment -> [d]ev, [p]rod: ")
            is_production = str(env_value).lower().startswith("p")
            env = PROD if is_production else DEV
            deploy_parameters[ENV_ARG] = {"value": env}
        else:
            xprint(f"--> Missing Parameters: {missing_params}")

    # ------------- WRITE DEPLOYMENT PARAMS FILE AFTER VALIDATION ------------ #

    nprint("Writing Deployment Config")

    deploy_parameters_str = json.dumps(deploy_parameters)
    parameters_deployment = param_json_template.safe_substitute(
        parameters=deploy_parameters_str)

    deployment_json = f"{ROOT_PATH}{PARAMS_DEPLOYMENT}"
    with open(deployment_json, "w", encoding=ENCODING) as f:
        f.write(parameters_deployment)


# ---------------------------------------------------------------------------- #
#                               BUILD THE COMMAND                              #
# ---------------------------------------------------------------------------- #

print("Building Deployment Command")

# base command
command = command_template.safe_substitute({
    "name": hashlib.md5(bytes(ROOT, encoding="utf-8")).hexdigest(),
    "subscription_id": subscription_id,
    "location": location,
    "bicep_file": bicep_file,
})

# add parameters argument to command
if deployment_json and os.path.isfile(deployment_json):
    command += f" -p {deployment_json}"

# skip preview or not?
skip_preview = args.get("skip_preview")
if not skip_preview:
    command += " --confirm-with-what-if"


# ---------------------------------------------------------------------------- #
#                                RUN THE COMMAND                               #
# ---------------------------------------------------------------------------- #

info = info_template.safe_substitute(command=command)
print(info)

time.sleep(1)

# ? Run Forest -> RUN!
os.system(command)
if os.path.isfile(deployment_json):
    os.remove(deployment_json)

Hope it helps! Cheers

[rijais]

Is there an update on this?

[Shafeeqts89]

It would be very useful to have this feature implemented. please consider such useful improvisations.