[BUG] fields argument in Dataset.get_data makes non-selected columns NaN but all columns still present in DataFrame

[caelan-schneider]

Describe the bug Non-selected fields become NaN but all fields are still present in the dataframe - this is different from the example output in the docs.

To Reproduce Run the code from the "Fields Selection" section of this page:

from gs_quant.data import Dataset
from datetime import date

weather_ds = Dataset('WEATHER')
data_frame = weather_ds.get_data(date(2016, 1, 1), date(2016, 1, 2), city=["Boston"], fields=['maxTemperature', 'minTemperature'])

print(data_frame)

Expected behavior Returns:

     city        date  maxTemperature  minTemperature
0  Boston  2016-01-01            41.0            33.0
1  Boston  2016-01-02            40.0            31.0

Actual behavior Returns:

              city  maxTemperature  minTemperature  dewPoint  windSpeed   
date                                                                      
2016-01-01  Boston            41.0            33.0       NaN        NaN  \
2016-01-02  Boston            40.0            31.0       NaN        NaN   

            precipitation  snowfall  pressure  updateTime  
date                                                       
2016-01-01            NaN       NaN       NaN         NaN  
2016-01-02            NaN       NaN       NaN         NaN  

Systems setup:

• OS: Windows 10
• Python version: 3.11
• GS-Quant version: 1.0.44

[l11ca]

If you set the standard_fields=True when calling get_data, you won't get the extra columns any more:

weather_ds = Dataset('WEATHER') data_frame = weather_ds.get_data(date(2016, 1, 1), date(2016, 1, 2), city=["Boston"], fields=['maxTemperature', 'minTemperature'], standard_fields=True)