coreos-assembler `go-autorest v9.1.0` : Update module to `v14.2.0` to address authorization bypass in jwt-go

The go-autorest dependency in Mantle is currently using v9.1.0 which uses jwt-go the vulnerable dependency. COSA build fails at any version after v9.10.0 since there is a change in package structure. Specifically Building Ore fails. The switch from jwt (insecure) to jwt(secure) for go-autorest occurs in the v14.2.0 documented over here .

The function used in the the current v9.1.0 called GetClientSetup() used in coreos-assembler/mantle/platform/api/azure/api.go is now replaced by several functions GetEnvironmentSettings(), NewAuthorizerFromFile(), GetSettingsFromFile().

To reproduce the fail in building COSA:

cd coreos-assembler/mantle
go get github.com/Azure/[email protected]
go mod tidy
go mod vendor 
make

Results in

./build cmd/ore
Building ore
# github.com/Azure/azure-sdk-for-go/arm/storage
vendor/github.com/Azure/azure-sdk-for-go/arm/storage/accounts.go:55:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/storage/accounts.go:144:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/storage/accounts.go:193:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/storage/accounts.go:219:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/storage/accounts.go:292:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/storage/accounts.go:487:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/storage/accounts.go:564:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/storage/accounts.go:648:29: undefined: validation.NewErrorWithValidationError
# github.com/Azure/azure-sdk-for-go/arm/resources/resources
vendor/github.com/Azure/azure-sdk-for-go/arm/resources/resources/deploymentoperations.go:61:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/resources/resources/deploymentoperations.go:138:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/resources/resources/deployments.go:62:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/resources/resources/deployments.go:137:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/resources/resources/deployments.go:223:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/resources/resources/deployments.go:272:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/resources/resources/deployments.go:314:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/resources/resources/deployments.go:361:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/resources/resources/deployments.go:391:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/resources/resources/deployments.go:466:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/resources/resources/deployments.go:466:29: too many errors
\# github.com/Azure/azure-sdk-for-go/arm/compute
vendor/github.com/Azure/azure-sdk-for-go/arm/compute/images.go:58:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/compute/images.go:107:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/compute/images.go:173:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/compute/usage.go:52:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/compute/virtualmachineextensions.go:102:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/compute/virtualmachineextensions.go:171:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/compute/virtualmachines.go:60:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/compute/virtualmachines.go:109:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/compute/virtualmachines.go:177:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/compute/virtualmachines.go:219:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/compute/virtualmachines.go:219:29: too many errors
# github.com/Azure/azure-sdk-for-go/arm/network
vendor/github.com/Azure/azure-sdk-for-go/arm/network/applicationgateways.go:105:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/network/applicationgateways.go:135:29: undefined: validation.NewErrorWithValidationError
vendor/github.com/Azure/azure-sdk-for-go/arm/network/applicationgateways.go:184:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/network/applicationgateways.go:251:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/network/applicationgateways.go:548:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/network/applicationgateways.go:615:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/network/expressroutecircuitauthorizations.go:105:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/network/expressroutecircuitauthorizations.go:174:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/network/expressroutecircuitpeerings.go:105:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/network/expressroutecircuitpeerings.go:173:9: undefined: azure.DoPollForAsynchronous
vendor/github.com/Azure/azure-sdk-for-go/arm/network/expressroutecircuitpeerings.go:173:9: too many errors
make: *** [Makefile:12: ore] Error 2

I am working on changing the function calls to update them from v9.1.0 to v14.2.0. Let me know if anyone has questions, issues, concerns, advice.

Jun 15 '22 18:06 RishabhSaini

@bgilbert you've touched the Azure code most recently (and generally are knowledgeable about mantle things), could you weigh in here?

Jun 16 '22 15:06 miabbott

I'm not an SME on the Azure authentication code, but SGTM.

Jun 16 '22 16:06 bgilbert

The https://github.com/Azure/go-autorest repo mentions:

NOTE: The modules in this repo will go out of support by March 31, 2023.
Additional information can be found
[here](https://azure.microsoft.com/updates/support-for-azure-sdk-libraries-that-do-not-conform-to-our-current-azure-sdk-guidelines-will-be-retired-as-of-31-march-2023/).

It's possible that updating the SDK (go get -u github.com/Azure/azure-sdk-for-go) will just drop the go-autorest dependency entirely.

Jun 16 '22 19:06 dustymabe

So should I try updating the Azure-SDK-for-go and then work on fixing any issues that come up in mantle/platform/Azure/api.go?

Jun 16 '22 19:06 RishabhSaini

Oh I see. We consume it directly (not indirectly) in

https://github.com/coreos/coreos-assembler/blob/1d83765e132b65915f674a296b6dda6df325bcf2/mantle/platform/api/azure/api.go#L31

So we'll have to find a replacement for that.

Jun 16 '22 19:06 dustymabe

OK. This page says:

This article applies to the legacy version of the Azure SDK for Go. For authenticating to the latest modules use the Azure Identity package.

Which is already a part of azure-sdk-for-go, so we just need to adapt our code to use that.

Jun 16 '22 19:06 dustymabe

After updating azure-sdk-for-go to the latest version, the go-autorestdependency can be updated to v14.2.0 with no errors and is only used indirectly. However, updating those have broken azure-vhd-utils and Mantle is giving this error upon make command:

./build cmd/ore
Building ore
# github.com/Microsoft/azure-vhd-utils/upload
vendor/github.com/Microsoft/azure-vhd-utils/upload/upload.go:89:35: cxt.BlobServiceClient.PutPage undefined (type storage.BlobStorageClient has no field or method PutPage)
vendor/github.com/Microsoft/azure-vhd-utils/upload/upload.go:93:15: undefined: storage.PageWriteTypeUpdate
# github.com/Microsoft/azure-vhd-utils/upload/metadata
vendor/github.com/Microsoft/azure-vhd-utils/upload/metadata/metaData.go:95:33: blobClient.GetBlobMetadata undefined (type storage.BlobStorageClient has no field or method GetBlobMetadata)
make: *** [Makefile:12: ore] Error 2

Upon further investigation, I have found that the file storage_mit.go at line #26 contains this: // derived from https://github.com/Microsoft/azure-vhd-utils/blob/8fcb4e03cb4c0f928aa835c21708182dbb23fc83/vhdUploadCmdHandler.go Does this line prevent any updates possible to azure-vhd-utils?

According to this page, storage is deprecated and replaced by azblob

Jun 17 '22 15:06 RishabhSaini