File to output capture to Blob storage using Blob SAS URL "Failed to validate blob url" and "Failed to output network traffic"

[avwsolutions]

Describe the bug Maybe doing things wrong here, but the capture isn't uploaded to the SAS URL. Seems to be complaining. Followed the documentation. Seems to be a little bit limited in description.

Messages are "Failed to validate blob url" and "Failed to output network traffic"

Related to net/url: invalid control character in URL"

Tried may things below the anonymised logs.

ts=2024-03-24T11:28:37.948Z level=error caller=outputlocation/blob.go:55 msg="Failed to validate blob url" goversion=go1.21.8 os=linux arch=amd64 numcores=2 hostname=aks-agentpool-35551448-vmss000000 podname=my-first-capture-99wqd-wxpx5 error="parse \"https://1234cap.blob.core.windows.net/captures?sp=racwdli&st=2024-03-24T11:25:58Z&se=2024-03-24T19:25:58Z&spr=https&sv=2022-11-02&sr=c&sig=0vksBBdje4XlXxxjOJdztOZN%2FTfiMWf16D53VxyzPHs%3D\\n\": net/url: invalid control character in URL"
ts=2024-03-24T11:28:37.948Z level=error caller=captureworkload/main.go:57 msg="Failed to output network traffic" goversion=go1.21.8 os=linux arch=amd64 numcores=2 hostname=aks-agentpool-35551448-vmss000000 podname=my-first-capture-99wqd-wxpx5 error="location \"BlobUpload\" output error: parse \"https://1234cap.blob.core.windows.net/captures?sp=racwdli&st=2024-03-24T11:25:58Z&se=2024-03-24T19:25:58Z&spr=https&sv=2022-11-02&sr=c&sig=0vksBBdje4XlXxxjOJdztOZN%!F(MISSING)TfiMgf16D53VxyzPHs%!D(MISSING)\\n\": net/url: invalid control character in URL\n"

To Reproduce Steps to reproduce the behavior:

1. Created a default storage container
2. Created a new private blob container called captures
3. Generated a SAS on that container with required write,read,list permissions (also tried full)
4. Copied the Blob SAS URL value to my local txt file
5. Create secret.

kubectl create secret generic capture-blob-storage --from-file=blob-upload-url=./blob-upload-url.txt

6. Create first capture

# Getting the first available node
if [[ -z $1 ]]; then 
  target=`kubectl get nodes -o 'jsonpath={.items[0].metadata.name}'`
else
  target=$1
fi

cat <<EOF | kubectl create -f -
apiVersion: retina.sh/v1alpha1
kind: Capture
metadata:
  name: my-first-capture
spec:
  captureConfiguration:
    captureOption:
      duration: 30s
    captureTarget:
      nodeSelector:
        matchLabels:
          kubernetes.io/hostname: ${target}
  outputConfiguration:
    hostPath: "/tmp/retina"
    blobUpload: capture-blob-storage
EOF

7. Show the logs of the pod that executed the job.

ts=2024-03-24T11:28:37.948Z level=error caller=outputlocation/blob.go:55 msg="Failed to validate blob url" goversion=go1.21.8 os=linux arch=amd64 numcores=2 hostname=aks-agentpool-35551448-vmss000000 podname=my-first-capture-99wqd-wxpx5 error="parse \"https://1234cap.blob.core.windows.net/captures?sp=racwdli&st=2024-03-24T11:25:58Z&se=2024-03-24T19:25:58Z&spr=https&sv=2022-11-02&sr=c&sig=0vksBBdje4XlXxxjOJdztOZN%2FTfiMWf16D53VxyzPHs%3D\\n\": net/url: invalid control character in URL"
ts=2024-03-24T11:28:37.948Z level=error caller=captureworkload/main.go:57 msg="Failed to output network traffic" goversion=go1.21.8 os=linux arch=amd64 numcores=2 hostname=aks-agentpool-35551448-vmss000000 podname=my-first-capture-99wqd-wxpx5 error="location \"BlobUpload\" output error: parse \"https://1234cap.blob.core.windows.net/captures?sp=racwdli&st=2024-03-24T11:25:58Z&se=2024-03-24T19:25:58Z&spr=https&sv=2022-11-02&sr=c&sig=0vksBBdje4XlXxxjOJdztOZN%!F(MISSING)TfiMgf16D53VxyzPHs%!D(MISSING)\\n\": net/url: invalid control character in URL\n"

Expected behavior Upload and store my capture file.

Screenshots If applicable, add screenshots to help explain your problem.

Platform (please complete the following information):

• OS: AKSUbuntu-2204gen2containerd-202403.13.0
• Kubernetes Version: 1.29.0
• Host: AKS (Default dev pattern)
• Retina Version: Latest main build 24/3)

Additional context Add any other context about the problem here.

[avwsolutions]

See differences which can matter. I have created a container and expects that blobs are added there?

Code

blob_test.go https://retina.blob.core.windows.net/container/blob?sp=r&st=2023-02-17T19:13:30Z&se=2023-02-18T03:13:30Z&spr=https&sv=2021-06-08&sr=c&sig=NtSxlRK5Vs4kVs1dIOfr%2FMdLKBVTA4t3uJ0gqLZ9exk%3D

blob-upload-url ( I have set) https://1234cap.blob.core.windows.net/captures?sp=racwdli&st=2024-03-24T11:25:58Z&se=2024-03-24T19:25:58Z&spr=https&sv=2022-11-02&sr=c&sig=0vksBBdje4XlXxxjOJdztOZN%2FTfiMWf16D53VxyzPHs%3D

[vakalapa]

@mainred can you help triage this ?

[avwsolutions]

@mainred @rbtr Also tried to use the blob container using the CLI ( 0.0.11 without succes.). Btw also seems that I have to add '--name flag', but this isn't part of the documentation example.

./kubectl-retina capture create  --name example --node-selectors "kubernetes.io/os=linux"  –debug --blob-upload https://test.blob.core.windows.net/captures?sp=rw&st=2024-05-28T19:46:42Z&se=2024-05-29T03:46:42Z&skoid=5df443ed-258d-4282-960c-xxxxxxxxxxxxxktid=84f1e4ea-8554-43e1-8709-f0b8589ea118&skt=2024-05-28T19:46:42Z&ske=2024-05-29T03:46:42Z&sks=b&skv=2022-11-02&spr=https&sv=2022-11-02&sr=c&sig=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx%3D

ts=2024-05-28T20:02:25.274Z level=info caller=outputlocation/blob.go:46 msg="Upload capture file to blob." location=BlobUpload ts=2024-05-28T20:02:25.880Z level=error caller=outputlocation/blob.go:81 msg="Failed to upload file to storage account" location=BlobUpload error="PUT https://test.blob.core.windows.net/captures/example-aks-agentpool-32992052-vmss000000-20240528080120UTC.tar.gz\n--------------------------------------------------------------------------------\nRESPONSE 401: 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.\nERROR CODE: NoAuthenticationInformation\n--------------------------------------------------------------------------------\n<Error><Code>NoAuthenticationInformation</Code><Message>Server failed to authenticate the request. Please refer to the information in the www-authenticate header.\nRequestId:7d325446-a01e-000b-2339-b16b90000000\nTime:2024-05-28T20:02:25.8755372Z</Message></Error>\n--------------------------------------------------------------------------------\n" ts=2024-05-28T20:02:25.883Z level=error caller=captureworkload/main.go:68 msg="Failed to output network traffic" error="location "BlobUpload" output error: PUT https://test.blob.core.windows.net/captures/example-aks-agentpool-32992052-vmss000000-20240528080120UTC.tar.gz\n--------------------------------------------------------------------------------\nRESPONSE 401: 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.\nERROR CODE: NoAuthenticationInformation\n--------------------------------------------------------------------------------\n<Error><Code>NoAuthenticationInformation</Code><Message>Server failed to authenticate the request. Please refer to the information in the www-authenticate header.\nRequestId:7d325446-a01e-000b-2339-b16b90000000\nTime:2024-05-28T20:02:25.8755372Z</Message></Error>\n--------------------------------------------------------------------------------\n\n"

[rbtr]

Opened #410 for the --name flag docs. @vakalapa I think we need to reassign this for investigation

[mainred]

Thank you @avwsolutions for reporting this issue. Let me take a look.

[mainred]

The issue is caused by the blob URL validation failure, which needs to be corrected.

[mainred]

@rbtr I can take https://github.com/microsoft/retina/issues/410

[mainred]

@avwsolutions sorry for my late response and fix. The PR linked to this issue should fix the issue.

[rbtr]

Thanks @mainred 🙂

[avwsolutions]

I will test coming days, so i can use this in my blog article.