datadog-api-client-go icon indicating copy to clipboard operation
datadog-api-client-go copied to clipboard

`datadogV2.GetMonthlyCostAttribution`: fail to parse http response to `MonthlyCostAttributionResponse` due to different in time format

Open nvatuan opened this issue 9 months ago • 5 comments

Describe the bug

I was following the example from the official usage-metering docs on get-monthly-cost-attribution and I cannot seem to retrieve the Values field in the Response object.

MonthlyCostAttributionResponse field Attributes.Values is nil after successfully fetched and unmarshalled.

To Reproduce Steps to reproduce the behavior:

  1. The program to replicate the issue:
`main.go`
package main

import (
	"context"
	"fmt"
	"os"
	"time"

	"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
	"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
	"github.com/sirupsen/logrus"
)

var log = logrus.New()

func main() {
	// https://docs.datadoghq.com/api/latest/usage-metering/#get-monthly-cost-attribution example
	ctx := datadog.NewDefaultContext(context.Background())
	configuration := datadog.NewConfiguration()
	apiClient := datadog.NewAPIClient(configuration)
	api := datadogV2.NewUsageMeteringApi(apiClient)

	// go back 1 month and 20 days to ensure last month. Eg:
	// April 13th -> Get February data
	// April 20th -> Get March data
	timeToExtractMonth := time.Now().AddDate(0, -1, -20)
	log.Infof("Getting monthly cost attribution for %v...", timeToExtractMonth)

	resp, r, err := api.GetMonthlyCostAttribution(
		ctx,
		timeToExtractMonth,
		"*",
	)

	log.Infof("resp status: %v", r.StatusCode) // => 200

	if err != nil {
		fmt.Fprintf(os.Stderr, "Error when calling `UsageMeteringApi.GetMonthlyCostAttribution`: %v\n", err)
		fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
	}

	data, ok := resp.GetDataOk() // => ok
	if !ok {
		log.Errorf("get data not ok")
		panic("get data not ok")
	}
	if len(*data) == 0 {
		log.Errorf("no data")
		panic("no data")
	}
	log.Infof("ok can get data (data len = %v)", len(*data)) // => ok
	//
	attrib, ok := (*data)[0].GetAttributesOk() // => ok
	if !ok {
		log.Errorf("get attrib not ok")
		panic("get attrib not ok")
	}
	log.Infof("ok can get attrib") // => ok
	//
	values, ok := attrib.GetValuesOk() // => NOT OK, values is nil
	log.Infof("values: %v", values)    // => nil
	if !ok {
		log.Errorf("get values not ok")
		panic("get values not ok")
	}
	log.Infof("ok can get values")
	//
	valuesMap, ok := (*values).(map[string]interface{})
	if !ok {
		log.Errorf("values is not a map, resp: %v", resp)
		panic("values is not a map")
	}
	log.Infof("values len: %v", len(valuesMap))
}
  1. Runs with appropriate env:
DD_API_KEY="<datadog-api-key>" DD_APP_KEY="<datadog-app-key>" go run main.go
  1. Logs:
INFO[0000] Getting monthly cost attribution for 2025-02-12... 
INFO[0001] resp status: 200                             
INFO[0001] ok can get data (data len = 2)               
INFO[0001] ok can get attrib                            
INFO[0001] values: <nil>                                
ERRO[0001] get values not ok                            
panic: get values not ok

Expected behavior

The HTTP Response was successful, I confirmed the return data contains complete data and also data for the Values field. There were no error in err, thus, I expected var monthlyCostAttributionData.Attributes.Values to not be nil after parsing.

Environment and Versions (please complete the following information):

Additional context The Values data can be found in monthlyCostAttributionData.Attributes.UnparsedObject, so I suspect parsing has failed at some point, but I have debugged up until the json.Unmarshal call and I couldn't find anything suspicous.

nvatuan avatar Apr 03 '25 19:04 nvatuan

Continue debugging:

  • The error seems to be from here:

https://github.com/DataDog/datadog-api-client-go/blob/86c68fbf3c53af986295c4d592b8ce6c7960f5d2/api/datadogV2/model_monthly_cost_attribution_attributes.go#L300-L302

  • Adding comments to debug:
// UnmarshalJSON deserializes the given payload.
func (o *MonthlyCostAttributionAttributes) UnmarshalJSON(bytes []byte) (err error) {
	println("datadoglib: unmarshalling is called#### 1") // debug
	all := struct {
		Month           *time.Time          `json:"month,omitempty"`
		OrgName         *string             `json:"org_name,omitempty"`
		PublicId        *string             `json:"public_id,omitempty"`
		TagConfigSource *string             `json:"tag_config_source,omitempty"`
		Tags            map[string][]string `json:"tags,omitempty"`
		UpdatedAt       *string             `json:"updated_at,omitempty"`
		Values          interface{}         `json:"values,omitempty"`
	}{}
	if err = datadog.Unmarshal(bytes, &all); err != nil {
		println("datadoglib: unmarshalling is called#### 3") // debug
		println("datadoglib: err", err.Error()) // debug
		return datadog.Unmarshal(bytes, &o.UnparsedObject)
	}
	println("datadoglib: unmarshalling is called#### 2") // debug
   ...
  • Log:
datadoglib: unmarshalling is called#### 1
datadoglib: unmarshalling is called#### 3
datadoglib: err parsing time "2025-02-01T00:00:00+0000" as "2006-01-02T15:04:05Z07:00": cannot parse "+0000" as "Z07:00"

We can locate the error is from datadog trying to unmarshal into a local struct, since the API specifies Month to be Datetime in ISO-8601 format, json unmarshal in go (I assumed) implicitly RFC3339 so this error was thrown.

https://github.com/DataDog/datadog-api-client-go/blob/86c68fbf3c53af986295c4d592b8ce6c7960f5d2/api/datadogV2/model_monthly_cost_attribution_attributes.go#L292

nvatuan avatar Apr 04 '25 07:04 nvatuan

Now the problem is much clearer, I updated the Issue while no one is on it yet.

nvatuan avatar Apr 04 '25 08:04 nvatuan

I may suggest a PR with the fix, I will read on how to contribute

nvatuan avatar Apr 04 '25 09:04 nvatuan

Hi, I believe this is a backend issue, I contacted the team in charge. We'll come back to you.

therve avatar Apr 11 '25 08:04 therve

Thanks for your contribution!

This issue has been automatically marked as stale because it has not had activity in the last 30 days. Note that the issue will not be automatically closed, but this notification will remind us to investigate why there's been inactivity. Thank you for participating in the Datadog open source community.

If you would like this issue to remain open:

  1. Verify that you can still reproduce the issue in the latest version of this project.

  2. Comment that the issue is still reproducible and include updated details requested in the issue template.

github-actions[bot] avatar May 12 '25 00:05 github-actions[bot]