bdh request does not return data frame because of a single bad date in Bloomberg data

[philaris]

Hi all, and thank you for this nice API.

There is an issue with the following historical request for ticker 'RBS LN Equity' and field LATEST_ANNOUNCEMENT_DATE:

Rblpapi::blpConnect()
result <- Rblpapi::bdh(
  securities = c('RBS LN Equity'),
  fields = c('LATEST_ANNOUNCEMENT_DT'),
  start.date = as.Date('2003-12-01'),
  end.date = as.Date('2018-11-30'),
  options = NULL,
  overrides = c('FUND_PER' = 'Q'),
  verbose = TRUE
)

I get the following error:

Day of month value is out of range 1..31

<boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::gregorian::bad_day_of_month> > in bdh_Impl(con, securities, fields, start.date, end.date, options, overrides, verbose, identity, int.as.double): Day of month value is out of range 1..31>

Data in the LATEST_ANNOUNCEMENT_DT field is received as a real number, where for example 2018-10-26 is represented by the real number 20181026.000000.

The reason for the error is the following entry in the data from Bloomberg as seen from the verbose output of the above request:

fieldData = {
  date = 2015-12-31
  LATEST_ANNOUNCEMENT_DT = 0.000000
}

All other entries for LATEST_ANNOUNCEMENT_DT in the timeseries are fine (stored as real number values), e.g.:

fieldData = {
  date = 2018-09-30
  LATEST_ANNOUNCEMENT_DT = 20181026.000000
}

I traced the exception through HistoricalDataResponseToDF in src/bdh.cpp:

            populateDfRow(res[colindex], i, e, rtypes[colindex]);

and then populateDfRow in src/blpapi_utils.cpp:

  case RblpapiT::Date:
    // handle the case of BBG passing down dates as double in YYYYMMDD format
    REAL(ans)[row_index] = e.datatype()==BLPAPI_DATATYPE_FLOAT32 || e.datatype()==BLPAPI_DATATYPE_FLOAT64 ?
      bbgDateToRDate(e.getValueAsFloat64()) :
      bbgDateToRDate(e.getValueAsDatetime());
    break;

and bbgDateToRDate in src/blpapi_utils.cpp:

const int bbgDateToRDate(const Datetime& bbg_date) {
  if(bbg_date.hasParts(DatetimeParts::TIME)) {
    throw std::logic_error("Attempt to convert a Datetime with time parts set to an R Date.");
  }
  const boost::gregorian::date r_epoch(1970,1,1);
  boost::gregorian::date bbg_boost_date(bbg_date.year(),bbg_date.month(),bbg_date.day());
  boost::gregorian::date_period dp(r_epoch,bbg_boost_date);
  return static_cast<int>(dp.length().days());
}

const int bbgDateToRDate(const double yyyymmdd_date) {
  if(yyyymmdd_date < 0) {
    throw std::logic_error("Attempt to convert a negative double value to an R Date.");
  }
  if(trunc(yyyymmdd_date)!=yyyymmdd_date) {
    throw std::logic_error("Attempt to convert a double value with time parts set to an R Date.");
  }

  const boost::gregorian::date r_epoch(1970,1,1);
  const int year = static_cast<int>(yyyymmdd_date/1.0e4);
  const int month = static_cast<int>(yyyymmdd_date/1.0e2) % 100;
  const int day = static_cast<int>(yyyymmdd_date) % 100;
  boost::gregorian::date bbg_boost_date(year,month,day);
  boost::gregorian::date_period dp(r_epoch,bbg_boost_date);
  return static_cast<int>(dp.length().days());
}

The exception is thrown in the constructor call for variable bbg_boost_date in the above code. With a single problematic value as above, the Rblpapi::bdh call returns no R data frame. My question is whether this behavior is too strict.

I am wondering if a better behavior would be to ignore the exceptions that are propagated through populateDfRow and have an NA value in the resulting R data frame.

A simple (but drastic) solution that implements this last behavior and does not change the existing code too much could be to create a version of populateDfRow (e.g., called populateDfRowNoExcept) that catches all exceptions and use that in HistoricalDataResponseToDF of src/bdh.cpp.

What do you think?

Best regards, Panagiotis CHEILARIS