Azure
[BUG] Nondescript error with 'github.com/vbauerster/getparty' / 'msazure.visualstudio.com/msazure/msk8s/downloadsdk.git'
Describe the bug A clear and concise description of what the bug is.
To Reproduce Steps to reproduce the behavior:
- Try on Windows Admin Center to make a cluster, and it failed with
Applying platform configurations failed. Error: Processing data from remote server dellr730-01.ad.splchicago.com failed with the following error message: The WSMan provider host process did not return a proper response. A provider in the host process may have behaved improperly. For more information, see the about_Remote_Troubleshooting Help topic.so I did a bit more digging and learned that I can do it in powershell. And I now know the error is because the process actually kills powershell, not just simply failing which explains why Windows Admin Center sees some sort of generic powershell error. - Try to do it on powershell via https://learn.microsoft.com/en-us/azure/aks/hybrid/kubernetes-walkthrough-powershell
- After I run the command in step 3 there is an error and I did a lot of googling and I do not see anything about this error - There is actually an issue on this github which I believe is unrelated #347 - but that has a more specific error message and I tried the solution to no avail.
- See error
Expected behavior I expect something more to happen. I expect at least an error message
Screenshots If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):
- OS: [e.g. iOS]
- OS Name: Microsoft Windows Server 2019 Standard
- OS Version: 10.0.17763 N/A Build 17763
- This is a 2 node failover cluster
- AKS-HCI Version: I do not know how to get this, I just learned about AKS Hybrid a few days ago so it is either the latest, whatever was out a few days ago, or what would come with the operating system because I never went out and sought out a specific version nor know what I have.
- Kubernetes Version - IDK
Collect log files I collected the logs, it is very large and I am unsure if there is any PII inside it. Below is the console logs
PS C:\Windows\system32> Set-AksHciConfig -imageDir c:\clusterstorage\volume1\Images -workingDir c:\ClusterStorage\Volume
1\ImageStore -cloudConfigLocation c:\clusterstorage\volume1\Config -vnet $vnet -cloudservicecidr "192.168.10.34/24"
True
DownloadSdk Tests Execution Started 12.11.2023 16:49:11
Test-KVASettings
Kva - Downloading image Linux_k8s_1-26-6_1.0.21.11030
Connection to https://msk8s.api.cdp.microsoft.com ... Succeeded
Connection to http://msk8s.b.tlu.dl.delivery.mp.microsoft.com ... Succeeded
Host is able to reach list of URLs requirements
Test Succeeded
Details: Host is able to reach list of URLs requirements
Recommendation:
Test execution time: 271.7319 milliseconds
DownloadSdk Tests Execution Ended 12.11.2023 16:49:11
=====================================================
All DownloadSdk Validation tests are successful
=====================================================
Check the test report(downloadsdk_validation_report.html) in current directory
Kva Tests Execution Started 12.11.2023 16:49:11
===============================================================================
Test (1 of 1): "Validate KVA". Category: KVA
panic: totalWritten != stat.Size()
goroutine 147 [running]:
github.com/vbauerster/getparty.Session.concatenateParts({{0x1c000182700, 0xf3}, {0x1c000182700, 0xf3}, {0x1c00007a5b9, 0x67}, {0x0, 0x0}, {0x1c0001a2cd0, 0x5}, ...}, ...)
/home/vsts/go/pkg/mod/github.com/vbauerster/[email protected]/session.go:146 +0x15a5
github.com/vbauerster/getparty.(*Cmd).Run(0x1c000601da0, {0x1c00015e200, 0x5, 0x8}, {0x0, 0x0}, {0x0, 0x0})
/home/vsts/go/pkg/mod/github.com/vbauerster/[email protected]/getparty.go:318 +0x1d85
msazure.visualstudio.com/msazure/msk8s/downloadsdk.git/sdk/pkg/http.DownloadFile({0x7fff0a0edba8?, 0x1c00009f320}, {0x1c000182700, 0xf3}, {0x1c00007a4d0, 0x67}, 0xa, {0x0, 0x0}, 0x1c00000a100)
/home/vsts/work/1/s/sdk/pkg/http/http.go:85 +0x72f
msazure.visualstudio.com/msazure/msk8s/downloadsdk.git/sdk/pkg/sfsclient.(*SFSClient).GetVerifiedFiles(0x1c0000a81b0, {0x7fff0a0edba8, 0x1c00009f320}, {{{0x1c00001c820, 0x19}, {0x7fff0a01c27a, 0x7}, {0x1c0005cc874, 0xc}}}, {0x1c000022100, ...}, ...)
/home/vsts/work/1/s/sdk/pkg/sfsclient/sfsclient.go:187 +0x85b
msazure.visualstudio.com/msazure/msk8s/downloadsdk.git/sdk/pkg/download/provider/sfs.(*sfsProvider).getReleaseInternal(0x1c0000a81c8, {0x7fff0a0edba8, 0x1c00009f320}, {0x1c00001c820?, 0x0?}, {0x1c0005cc874?, 0x0?}, {0x1c000022100, 0x38}, 0xa, ...)
/home/vsts/work/1/s/sdk/pkg/download/provider/sfs/release.go:49 +0x454
msazure.visualstudio.com/msazure/msk8s/downloadsdk.git/sdk/pkg/download/provider/sfs.(*sfsProvider).GetRelease(0x0?, {0x7fff0a0edba8?, 0x1c00009f320?}, {0x1c00001c820?, 0x7fff0a2d6b80?}, {0x1c0005cc874?, 0xa?}, {0x0?, 0x0?}, {0x0, ...}, ...)
/home/vsts/work/1/s/sdk/pkg/download/provider/sfs/release.go:16 +0x96
msazure.visualstudio.com/msazure/msk8s/downloadsdk.git/sdk/pkg/download.(*downloadClient).getRelease(0x1c000308cc0, {0x7fff0a0edba8, 0x1c00009f320}, {{0x1c00001c820, 0x19}, {0x1c0005cc874, 0xc}, {0x0, 0x0}, {0x0, ...}, ...})
/home/vsts/work/1/s/sdk/pkg/download/release.go:73 +0x2b0
msazure.visualstudio.com/msazure/msk8s/downloadsdk.git/sdk/pkg/download.(*downloadClient).GetRelease(0x1c000519eb0?, {0x7fff0a0edba8?, 0x1c00009f320?}, {{0x1c00001c820, 0x19}, {0x1c0005cc874, 0xc}, {0x0, 0x0}, {0x0, ...}, ...})
/home/vsts/work/1/s/sdk/pkg/download/release.go:56 +0x13d
main.GetRelease.func2()
/home/vsts/work/1/s/sdk/main.go:175 +0x97
created by main.GetRelease
/home/vsts/work/1/s/sdk/main.go:174 +0x5ec
C:\Windows\system32>powershellSet-AksHciConfig -offlineDownload $true -mode full -stagingShare c:\akshciimages -imageDir c:\clusterstorage\volume1\Images -workingDir c:\ClusterStorage\Volume1\ImageStore -cloudConfigLocation c:\clusterstorage\volume1\Config -vnet $vnet -cloudservicecidr "192.168.10.34/24" -verbose
'powershellSet-AksHciConfig' is not recognized as an internal or external command,
operable program or batch file.
C:\Windows\system32>
I seem to have partially resolved this.
I noticed that the paths I was using, while specifically called out in the documentation, do not work. You have to put things in C:\clusterstorage\volume1\akshci\{images/imagestore/config}
Now that I did I can reproduce #363
But to me this issue is still a separate bug unless documentation was added to say that custom paths (or inseed, the path listed in the documentation) do not work now.
This one has come back with a vengeance. Idk if I even figured out the problem. I actually just delved into getting openshift today (which is like 2-5x as expensive) so I won't be looking into this more but between this and the other bug I reproduced on separate hardware/setup I'm not entirely sure it's possible to stand up any aks hybrid at the moment. Perhaps you can share your reference deployment. I mean I might be willing to give it a second look but not likely.
@markamber - this is a log from WAC? I'm just wondering about this:
If setup over WAC fails for you connect to a host and use Powershell: https://learn.microsoft.com/en-us/azure/aks/hybrid/kubernetes-walkthrough-powershell That's still my preferred way to setup until https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-hybrid-preview-overview is fully released.
@markamber - this is a log from WAC? I'm just wondering about this:
If setup over WAC fails for you connect to a host and use Powershell: https://learn.microsoft.com/en-us/azure/aks/hybrid/kubernetes-walkthrough-powershell
That's still my preferred way to setup until https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-hybrid-preview-overview is fully released.
No as I mentioned in my post I did start with WAC but moved to the powershell per the document I linked. As you can see there I just randomly typed powershell (because the command literally crashed powershell, so I was trying to re-enter) and then probably clicked the right mouse button with a string of text that had a new line character and included it in my post for no particular reason.
Hmm, I just spent 3 days trying to figure this out. In my case, the issue was my CSV name. I was saving to "C:\ClusterStorage\T3-Data.01\AksHCI\[path name]", which was resulting in the above error. When I renamed the CSV to "C:\ClusterStorage\T3Data", It started working. So it seems like the installer doesn't like either a hyphen or a period in the CSV name.
I might have spoken too soon. I got further in the install then got another error. Now when I re-Run the install i'm now back getting this error again even with the different CSV name...
I just checked the logs from Get-AksHciLogs and it seems I keep running into issues downloading files. Maybe related to https://github.com/Azure/aksArc/issues/355 . It could be something to do with my firewall... not sure.. I'm going to try the offline method.