Need to catch/raise errors on Bicep compilation issues (or use newer ARM TTK and Bicep like XPlat version of extension)

[spoelstraethan]

I've noticed this issue off and on, but hadn't been keeping a close enough eye on the log output until recently.

Scenario: I have a Bicep file with a ternary construct (in a var or in-line) for determining whether to include a properties: value dynamically. With the arm-ttk-extension there is an error in the bicep build stage and consequently there are some missing .json versions of Bicep files, including the main.bicep, but this doesn't cause the extension to throw an error.

I noticed again while explaining a pipeline with the ARM TTK extension present that the number of Tests tracked on the Tests tab seemed low, so I checked the ARM TTK output and noticed some errors (that weren't fatal) and then noticed the number of {bicepmodule}.json files tested didn't match up with all the modules I have. It also complains about the #disable-next-line skips I have in place.

I got our admins to add the new arm-ttk-xplat and it runs successfully, though was failing on different errors (until I successfully suppressed them with #disable-next-line).

It would be great if there was either a hard fail or better visibility into the fact that certain Bicep constructs will result in ARM TTK NOT testing the entirety of the files where they are present or any files linking to them.

[sam-cogan]

Thanks for the PR. Just for my clarity it sounds like there are two issues here:

1. If the Bicep compiliation of the template fails, we ignore the failed file and show no idication in the test results. We should catch this and show a failure
2. There is a specific issue with your templates that is causing bicep to fail, which seems to be fixed in the cross platform version ( Is this correct?

Item 1 is a very valid point and I will look at how we can solve this. For item 2 I would reccomend switching to the cross platform version as I will stop making changes to the old version soon.

[spoelstraethan]

Correct, # 1 is the "defect" where there isn't a "SucceededWithIssues" warning or failOnBicepConversionError flag or anything like that to indicate something went wrong, it just shows green on the status.

Part # 2 is two-fold, the Bicep conversion in this extension complains about #disable-next-line Bicep directives, but is also fails to handle a multi-line ternary/union that has been valid Bicep for a while and is a workaround for handling adding conditional properties to a set of base common properties. https://github.com/Azure/bicep/issues/387#issuecomment-1028699186

I'm definitely switching over to the XPlat version as quickly as possible, because we are shooting for new code to be Bicep as much as possible instead of JSON ARM templates.

[sam-cogan]

Ok great. I suspect the ternary issue is down to the TTK being older, I will get that updated. I'lll test out the other issue.