Suppressing Stan optimizer printing in Python

[StuartGordonReid]

Hi,

I recently started using Prophet in Python, and whilst it is working (quite well), it is printing out messages which appear to be generated by PySTAN and suppressing output from stdout, stderr, and stdin doesn't seem to stop it. Could you add an option to suppress output from PySTAN. I've attached screenshots of the output and the function I am trying to use to suppress the output which is not working:

Kind regards, Stuart Reid

[bletham]

I worked on this a bit for #170 and didn't make it very far. Help here is greatly appreciated :-) We may need an upstream change.

[randlet]

Here's a decorator that DOES suppress output:

# from https://stackoverflow.com/questions/11130156/suppress-stdout-stderr-print-from-python-functions
class suppress_stdout_stderr(object):
    '''
    A context manager for doing a "deep suppression" of stdout and stderr in
    Python, i.e. will suppress all print, even if the print originates in a
    compiled C/Fortran sub-function.
       This will not suppress raised exceptions, since exceptions are printed
    to stderr just before a script exits, and after the context manager has
    exited (at least, I think that is why it lets exceptions through).

    '''
    def __init__(self):
        # Open a pair of null files
        self.null_fds = [os.open(os.devnull, os.O_RDWR) for x in range(2)]
        # Save the actual stdout (1) and stderr (2) file descriptors.
        self.save_fds = (os.dup(1), os.dup(2))

    def __enter__(self):
        # Assign the null pointers to stdout and stderr.
        os.dup2(self.null_fds[0], 1)
        os.dup2(self.null_fds[1], 2)

    def __exit__(self, *_):
        # Re-assign the real stdout/stderr back to (1) and (2)
        os.dup2(self.save_fds[0], 1)
        os.dup2(self.save_fds[1], 2)
        # Close the null files
        os.close(self.null_fds[0])
        os.close(self.null_fds[1])

# used like
 with suppress_stdout_stderr():
     p = Propet(*kwargs).fit(training_data)

[jimanvlad]

@randlet Awesome. Has this been pulled?

[randlet]

Not yet, but I would be happy to work up a PR this week if @bletham is ok with using a context manager to (optionally) suppress output.

[randlet]

Actually, I don't know how the copyright/FB contributors agreement would work for this. I didn't write the code and AFAIK Stack Overflow snippets (prior to 2016) are licensed with CC Share Alike license. @bletham do you know if it's possible for you to accept PR's for code that has a CC-SA license?

[bletham]

@randlet thanks for posting this, this is really great and should be immediately useful to people.

I'm not sure about the license thing. I think I'm inclined to just leave this issue open so that people can find and use the decorator, and then wait a bit longer on the Stan team for an upstream fix. They're definitely aware of the issue (https://github.com/stan-dev/rstan/issues/49) and say as of April that it will be suppressed soon, then it'll just be a matter of adding the verbose arg to prophet.

[randlet]

That approach makes good sense to me. Thanks!

[randlet]

Just wanted to post an update to this. We are running Prophet in a long running process doing lots of forecasting and found we were running out of file descriptors pretty quickly causing issues on the server. I finally traced it back to the decorator posted above which leaks 2 file descriptors every time it is used.

I have modified it to the following with success:

class suppress_stdout_stderr(object):
    '''
    A context manager for doing a "deep suppression" of stdout and stderr in
    Python, i.e. will suppress all print, even if the print originates in a
    compiled C/Fortran sub-function.
       This will not suppress raised exceptions, since exceptions are printed
    to stderr just before a script exits, and after the context manager has
    exited (at least, I think that is why it lets exceptions through).

    '''
    def __init__(self):
        # Open a pair of null files
        self.null_fds = [os.open(os.devnull, os.O_RDWR) for x in range(2)]
        # Save the actual stdout (1) and stderr (2) file descriptors.
        self.save_fds = [os.dup(1), os.dup(2)]

    def __enter__(self):
        # Assign the null pointers to stdout and stderr.
        os.dup2(self.null_fds[0], 1)
        os.dup2(self.null_fds[1], 2)

    def __exit__(self, *_):
        # Re-assign the real stdout/stderr back to (1) and (2)
        os.dup2(self.save_fds[0], 1)
        os.dup2(self.save_fds[1], 2)
        # Close the null files
        for fd in self.null_fds + self.save_fds:
            os.close(fd)

[eeilon79]

Hi. I've got here from issue #219, yet this solution definitely does not work with Prophet.fit logger.INFO messages like "Disabling yearly seasonality..." and "Disabling daily seasonality...". I'm quite at a loss of how to get rid of them, actually. Nothing works.

[eeilon79]

Nevermind. Turns out that if you specify "daily_seasonality=False" etc. the messages disappear. Although why on earth an unspecified parameter should generate a console output, Facebook knows.

[bletham]

Those messages can be disabled by setting the logging level for fbprophet to something above INFO:

import logging
logging.getLogger('fbprophet').setLevel(logging.WARNING)

The reason for them is to avoid a scenario where a new user fits a time series with default values, seasonality is disabled e.g. due to too short a time series, but the user doesn't know this and is left wondering why the seasonality doesn't show up in the plot like in the documentation. With the info message they will know that it was on purpose and not a bug.

[eeilon79]

Hello Ben, Thank you for your answer. Sorry about the earlier disgruntled post and about the delay in my response. I've tried logging before, probably with the wrong syntax. However, now it works. Thanks.

[smontanaro]

Any progress on this? I just started to experiment with Prophet in an existing application which spits out HTML to stdout. All this business about log joing probabilities and Hessians is wreaking havoc with that. I finally had to wrap this sort of business around my prophet calls:

stdout = os.dup(1)
stderr = os.dup(2)
null = os.open(os.devnull, os.O_RDWR)
os.dup2(null, 1)
os.dup2(null, 2)
try:
    ... prophet calls here ...
finally:
    os.dup2(stdout, 1)
    os.dup2(stderr, 2)

I will eventually turn that into a decorator. Still would be nice to have an official way to do that suppression, one which didn't require so many system calls... (I'm guessing Stan has no clean way to tell it to shut up?)

I'm using 0.2.1 (package from Conda forge). Maybe a later version has this problem solved?

[bletham]

The upstream issue in Stan is still open, there is nothing in the later version of prophet to fix this. I was hoping for an upstream fix to avoid having to mess around with redirecting stderr, but we may have to just do it for the next version. In the meantime a decorator like that is the way to go.

[drcrook1]

Is there a link to the docs on how to do this? I'm running thousands of simulations and this is killing me...

[eeilon79]

Hi drcrook1,

I'm not sure what did the job for me, but it was either logging or suppress_stdout_stderr above. I had a class which had: a. suppress_stdout_stderr as a sub class b. a fit function that called for suppress_stdout_stderr: The relevant parts of the code are:

import logging
from fbprophet import Prophet

logging.getLogger('fbprophet').setLevel(logging.WARNING)

model = Prophet()
with self.suppress_stdout_stderr():
    model.fit(fit_data)

Hopes it helps.

[eeilon79]

Errr.. There was a question by shaidams64 which I don't see now, but I would answer anyway: suppress_stdout_stderr wasn't a sub class of a fit class. I had a parent class which managed the entire simulation which had suppress_stdout_stderr as a sub_class and also had a fit function that used that class in the way I've quoted above. The model.fit class which was used on the fit function was imported from another library. The fit function and suppress_stdout_stderr class are both children of the same parent class.

[eromoe]

I found that : 1 . suppress_stdout_stderr works if you run it in main process . 2. Didn't work when I used joblib or multiprocessing , it would still print the log, which seem slow down training .

[wuxiyu]

@eromoe hi, it fixes using this. This will help you silence logger

[mattc-eostar]

import logging
from fbprophet import Prophet

logging.getLogger('fbprophet').setLevel(logging.WARNING)

model = Prophet()
with self.suppress_stdout_stderr():
    model.fit(fit_data)

@eromoe, I have this successfully blocking the output across multiple processes.

[ShreyasJothish]

Using the wrapper code, I was able to suppress INFO logs but is there a way to stop WARNING logs too?

I tried setting the log level to ERROR but still WARNING logs appear.

logging.getLogger('fbprophet').setLevel(logging.ERROR)

[sammo]

@ShreyasJothish this should do it.

# Turn off pystan warnings
warnings.simplefilter("ignore", DeprecationWarning)
warnings.simplefilter("ignore", FutureWarning)

# Turn off fbprophet stdout logger
logging.getLogger('fbprophet').setLevel(logging.ERROR)

[RonanFelipe]

I have tried the solution from @bletham and the one from @sammo but nothing, still having those Iteration messages.

[mattc-eostar]

@RonanFelipe Use the suppression class given above from @randlet https://github.com/facebook/prophet/issues/223#issuecomment-326455744

Then use:

import logging
from fbprophet import Prophet

logging.getLogger('fbprophet').setLevel(logging.WARNING)

model = Prophet()
with self.suppress_stdout_stderr():
    model.fit(fit_data)

[RonanFelipe]

Thanks @mattc-eostar, it worked.

[CrashLaker]

Running the code proposed by @randlet solved it for me thank you so much.

However running this on AWS lambda sometimes gives me this error multiple times: OSError: [Errno 24] Too many open files: '/dev/null'

Can anyone help? python3.7 pystan==2.18 fbprophet==0.5

[randlet]

@CrashLaker did you see my revised code which addresses running out of file descriptors here: https://github.com/facebook/prophet/issues/223#issuecomment-326455744

[decloedt]

Is there a workaround to suppress stan output in R?

[sahilee26]

@ShreyasJothish this should do it.

# Turn off pystan warnings
warnings.simplefilter("ignore", DeprecationWarning)
warnings.simplefilter("ignore", FutureWarning)

# Turn off fbprophet stdout logger
logging.getLogger('fbprophet').setLevel(logging.ERROR)

thanks it worked

[chinmaydas96]

It didn't work for me while I am using the cross_validation while model fitting using dask, Does anyone know a workaround for that thing?

I was using this like this.

with self.suppress_stdout_stderr(): df_cv = cross_validation(m, horizon='7 days', period='7 days', initial='500 days', parallel='dask')