Add verbose flag to log messages

[johnlaing]

As suggested in Whit's comment here: https://github.com/Rblp/Rblpapi/issues/73#issuecomment-144064295

[eddelbuettel]

--verbose or --debug or --dump or ... ?

What is it supposed to do? Print to stdout? Or stderr? In a standard format?

[armstrtw]

bbg actually has a nice message formatter. So, I think we can just pass an ostream to it as an arg. like Rout...

[johnlaing]

Bloomberg provides a << operator for messages, so we can use that. This will be extremely useful for debugging.

Here is an example:

> bdh("VNF SJ Equity", "PX_LAST", as.Date("2015-09-21"), include.non.trading.days=TRUE, verbose=TRUE)
ServiceOpened = { serviceName = "//blp/refdata" }
HistoricalDataResponse = { securityData = { security = "VNF SJ Equity" eidData[] = { } sequenceNumber = 0 fieldExceptions[] = { } fieldData[] = { fieldData = { date = 2015-09-21 } fieldData = { date = 2015-09-22 
} fieldData = { date = 2015-09-23 } fieldData = { date = 2015-09-24 } fieldData = { date = 2015-09-25 } fieldData = { date = 2015-09-26 } fieldData = { date = 2015-09-27 } fieldData = { date = 2015-09-28 } fieldD
ata = { date = 2015-09-29 } } } }
        date
1 2015-09-21
2 2015-09-22
3 2015-09-23
4 2015-09-24
5 2015-09-25
6 2015-09-26
7 2015-09-27
8 2015-09-28
9 2015-09-29

[eddelbuettel]

Yes, I have used that too. Not parseable, so I'd file that under limited usefulness :)

Shall we at least prefix it with something so that one could grep/sed/awk/... it away?

[johnlaing]

I don't understand. By default, verbose=FALSE and there is no message, so nobody needs to do anything to make it go away. I find it extremely useful.

[armstrtw]

I thought his point was that the curly bracket formatted messages are not great for parsing in large quantities...

of course if you do have large quantities of messages, then you probably don't want it showing up on the command line, so perhaps we need an additional option to sink it to a file.

I was going to suggest letting verbose be either T/F or a string (as in file.out). However, I don't think that plays nicely w/ Rcpp as the arguments are not allowed to be multiple types (correct me if I'm making a bad assumption). so an additional arg could be use defaulting to NULL in which case the output would show on the command line, otherwise, sink to the file location provided (in the additional argument)...

[johnlaing]

Using print function instead of << get us newlines. This is better:

> ans <- bdh("VNF SJ Equity", "PX_LAST", as.Date("2015-09-21"), include.non.trading.days=TRUE, verbose=TRUE)
SessionConnectionUp = { server = "app:8194" }
SessionStarted = { }
ServiceOpened = { serviceName = "//blp/refdata" }
HistoricalDataResponse = {
    securityData = {
        security = "VNF SJ Equity"
        eidData[] = {
        }
        sequenceNumber = 0
        fieldExceptions[] = {
        }
        fieldData[] = {
            fieldData = {
                date = 2015-09-21
            }
            fieldData = {
                date = 2015-09-22
            }
            fieldData = {
                date = 2015-09-23
            }
            fieldData = {
                date = 2015-09-24
            }
            fieldData = {
                date = 2015-09-25
            }
            fieldData = {
                date = 2015-09-26
            }
            fieldData = {
                date = 2015-09-27
            }
            fieldData = {
                date = 2015-09-28
            }
            fieldData = {
                date = 2015-09-29
            }
        }
    }
}

I think there are lots of things we could do for fancy logging... but at the moment my only goal is to provide a little bit of transparency into what Bloomberg is sending back to us.

[armstrtw]

Yes! This is the one that takes the stream as an argument. nice. msg.print(Rcpp::Rcout);

[eddelbuettel]

@johnlaing I was referring to what Whit referred to: Once you turn it on, and get more than trivial amounts for more that just a handful of datapoints, you want to parse/filter.

@armstrtw Right. And we could even have an option to

• turn logging on
• turn logging to file on if filename was given (we just pass the file stream object to msg.print)

[armstrtw]

Sure, we can use getOption for this. is that what you mean? and just leave the verbose flag as a boolean?

[eddelbuettel]

I have done tricks as well where , logging=c(NA, "") could be differentiated:

• NA means current behaviour
• "" turns it on, use Rcpp::Rcout
• "path/and/file.txt" opens that file and passes the stream object around

Good? Or overkill? Also still open for best name for the new option. And we could of course support settings of a global option but don't have to either ...

[armstrtw]

prefix it w/ "blp"...? blpLogging ?

Yes, it could get complicated.
I'm not quite following on the NA. What is the current behaviour? Also, this will come from R, so it could be: options(blpLogging=textConnection("/tmp/Rblp.log"))

[eddelbuettel]

Current behaviour is not logging. So I am thinking about

bdh <- function(securities, fields, start.date, end.date=NULL,
    include.non.trading.days=FALSE,                ## old, eventually part of options()?
    options=NULL, overrides=NULL,                  ## pass through to Bloomberg
    identity=NULL,                                 ## unchanged
    logging=c(NA, ""),                             ## NEW 
    con=defaultConnection()) {

    logging <- match.arg(logging)        ## assuming that works with NA, not sure

    # rest omitted
}

[johnlaing]

I don't think match.arg is the right choice for this. I would vote for logging=getOption(logging, NULL) with:

• NULL (default), NA, or FALSE: no logging
• TRUE or "": console (Rcout)
• character value: file of that name
• any other value: error

[eddelbuettel]

Just played with it, and indeed: "file.txt" does not pass the proposed match.arg() with an empty string.

If you two of you prefer a new global option, can we make it start with blp.... like the other ones? I'd be fine with just logging=NULL and later testing for is.character ...