Support Custom fields for csv export

[lmello]

Is your feature request related to a problem? Please describe. Today csv export has hardcoded the fields it export with the exception of the labels.

As a user I want to be able to specify different fields to include in the export. for example, I would like to include the provider id on the csv export.

This is a request from our slack channel. the user worked around the issue by querying directly the API.

Describe the solution you'd like User configures an environment variable EXPORT_CSV_EXTRA_FIELDS=properties.providerid

The csv export would include the data for the configured extra fields in addition to the current exported fields.

Describe alternatives you've considered Using the API directly yelds the right result when aggregated by namespace,pod,container. But this places the burden on the user to write the export script, run and monitor it, while the csv export is highly integrated with opencost and is run automatically everytime the opencost pod restarts or once daily.

Additional context https://cloud-native.slack.com/archives/C03D56FPD4G/p1701178161992719

[mattray]

Thanks for the follow up, this isn't likely to be addressed by Kubecost since they don't use it but if someone provides a patch we'll get it merged

[santosh-sahoo334]

+1 @lmello Thanks for creating this issue. This will make life lot more easier!! @mattray Can you please help me link to the backend module that drives the CSV export and environment variable ? I can spend some time to provide a patch. Thank you!

[AjayTripathy]

Supportive here! This may be a good place to start.

https://github.com/opencost/opencost/blob/2f3f98c43b32889069563a696e6582dfbb58d022/pkg/costmodel/csv_export.go#L37

[santosh-sahoo334]

@AjayTripathy Thanks. Will work on this and update.

[Bharadwajshivam28]

Hey if anyone is not working then can i work on it?

[Bharadwajshivam28]

I need some steps to get started

[mattray]

@Bharadwajshivam28 AFAIK no one is working on it. @r2k1 might know differently

[Bharadwajshivam28]

@Bharadwajshivam28 AFAIK no one is working on it. @r2k1 might know differently

I can work on it if no one is ..can you provide me some more details and steps to get started

[r2k1]

Should be relatively straightforward to implement.

Take a look at https://github.com/opencost/opencost/blob/develop/pkg/costmodel/csv_export.go#L292

I would begin with considering how user will configure it. Are there default fields? Do you want to make column order configurable? Do you want an ability to disable default columns etc.

[r2k1]

Looks like there is an option to configure label aggregation. Something similar can be done for other options.

[r2k1]

I would probably add something like:

EXPORT_CSV_COLUMNS=Date,ControllerKind,ControllerName,TotalCost

And if it's not specified, set the currently exported columns.

I think it will be the most flexible options covering most of the use-cases.

[Bharadwajshivam28]

Thanks @r2k1 I will soon update it and raise PR

[Bharadwajshivam28]

Hey @r2k1 What I think that we can use environment variable to get columns from users and if its not specified then we can use the default columns

I was thinking like we can get columns from environment variable and if its not given then we can use the Pre-defined columns which are currently in the csv_export.go file

I will raise the PR by making the changes and then I will modify the changes if needed