Our vic-machine-server output violates json RFC

[mhagen-vmware]

trafficstars

When we query the server for path under target we receive back:

{"auth":{"client":{"certificate_authorities":[]},"server":{"certificate":{"pem":"-----BEGIN CERTIFICATE-----\nMIIDUjCCAjqgAwIBAgIQbhsuhiCU7ff0xmv0JQv6hDANBgkqhkiG9w0BAQsFADAs\nMRAwDgYDVQQKEwdkZWZhdWx0MRgwFgYDVQQDEw92Y2guZXhhbXBsZS5jb20wHhcN\nMTgwMjA1MDE0NDU0WhcNMTkwMjA2MDE0NDU0WjAsMRAwDgYDVQQKEwdkZWZhdWx0\nMRgwFgYDVQQDEw92Y2guZXhhbXBsZS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IB\nDwAwggEKAoIBAQDSTAbDzv4+dGwwYU/GISzS3z31fHXR5LMl9C5kIgrzHufbzG0g\nlmJZsZq2S8s2bzsaZM69cwqvPPk61FLsUwaZ5rYl3iTvdruHX1Dd5T+ocIOaS9MN\n8lZMTaSvtycS8MiZuVdIBK/iUCMZTaz7Erb6BhxZxpMMGgzibKCMZ6djnAmkGMIJ\nZDvgK3qaRFUwnLPrJwgZf0Z0EFVBYeKmrmLKnozslK88lyuXtTN1aRpZxP7Klvoy\n/Xjn6FfMttq2fMJ7oqbchGqLK3uwHD/Tdzxbu0B6r9OalCfuTkuBR1dGX07vWpF+\n3mcn4k9tR6Zu63EtFvUNLcw4wa0BZoBEYO2fAgMBAAGjcDBuMA4GA1UdDwEB/wQE\nAwIDqDATBgNVHSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB0GA1UdDgQW\nBBTBUED7+9o8GhXnj1qeDLRepYfyUzAaBgNVHREEEzARgg92Y2guZXhhbXBsZS5j\nb20wDQYJKoZIhvcNAQELBQADggEBAGGjOmHUpze12uo0Gz7IowCV9j9Gid5Nt960\nVUu2L+c2EAswyWMXYokxLHGrlkcMywN865koenIbd7N2ewgFJ9PCdiKIPFc2jC3a\niWD8SrxLDjT9ka5wRBoanlfRdvrOPVL+mUnR1m9NQxKJ/m0+uEp6ojWs9Gg2SCKJ\n6VglKk30fIrfUmaGNdSvaNNL+FTLoU7xVmXHs8JqzlfuZCWkTxhw8F7DGCJayJrZ\nP8Sq8Vk9fk/bvGXf14qeIRkSRv0dtq9AIkoQru6ROu0wsr2ACoauiOPk7onWR/b1\nlvHA0pTDT4BjgKV9hr10w672FhaBTrr3IkRanRmHioU45DW1+n0=\n-----END CERTIFICATE-----\n"}}},"compute":{"cpu":{"shares":{"level":"normal"}},"memory":{"shares":{"level":"normal"}},"resource":{"id":"pool1636"}},"container":{},"endpoint":{"cpu":{"sockets":1},"memory":{"units":"MiB","value":2048},"operations_credentials":{"user":"droneci"}},"name":"VCH-16067-3103-api-test-dc","network":{"bridge":{"ip_range":"172.16.0.0/12","port_group":{"id":"HaNetwork-VCH-16067-3103-bridge"}},"client":{"nameservers":[],"port_group":{"id":"HaNetwork-VM Network"}},"container":[],"management":{"nameservers":[],"port_group":{"id":"HaNetwork-VM Network"}},"public":{"nameservers":[],"port_group":{"id":"HaNetwork-VM Network"}}},"registry":{"certificate_authorities":[],"insecure":[],"whitelist":[]},"runtime":{"power_state":"poweredOn","upgrade_status":"Up to date"},"storage":{"base_image_size":{"units":"KB","value":8000000},"image_stores":["ds://datastore1/VCH-16067-3103-api-test-dc"],"volume_stores":[]},"version":"v1.3.0-rc1-16067-9433aba1"}

This causes the following error in newer versions of jq: parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 21, column 1

Per this issue, we need to fix our formatting but for now I have used an older version of jq that works: https://github.com/stedolan/jq/issues/1049

This is going to be a problem for our CI as well as any customers that might be trying to machine parse our API output ( not officially supported yet)

[zjs]

@mhagen-vmware:

What version of jq were you seeing this error with? I can parse the example output provided in this ticket just fine with jq-1.5-1-a5b5cbe, which appears to be the latest release.

parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 21, column 1

This error itself is a bit odd; the example input is only 1 line long.

Do you have a pointer to where this was seen in CI? I'd like to try to reproduce.

[gitjeff05]

That json has \n characters in it which are most likely being interpreted by and expanded by echo -- or something -- into a literal newline. jq does not like that. How are you passing it to jq?

Try printf "%s" "$var" | jq .

or

echo -E "$var"

See comment in issue 1049.