can't use tune after ignore when train dlm

[sun137653577]

hi Samuel, Sorry for bothering again, I want to use ignore and tune method in myDLM, while it is error when run following code:

from pydlm import dlm, trend, seasonality, dynamic, autoReg, longSeason y=[1,2,3,4,5,6,7,8,9,10] myDLM=dlm(np.array(y)) myDLM=myDLM+trend(1,discount=0.9)+seasonality(period=4) myDLM.fit() (temp_obser, temp_var)=myDLM.predict(date=9)

myDLM.ignore(2) myTuner=modelTuner(method='gradient_descent',loss='mse') bankDLM=myTuner.tune(myDLM, maxit=100) tuned_discounts=myTuner.getDiscounts() myDLM.fit()

TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

If we comment 'myDLM.ignore(2)', then the codes will run successfully. I wonder whether tune method can't accept missing data?

Another question, where is it appropriate to place myDLM.ignore(2), before myDLM.fit() or after myDLM.fit()？To be on the safe side, i add fit both before and after myDLM.ignore(2). thanks.

[wwrechard]

Hi Sun,

Thanks for your email. I believe there is a bug, because when computing the mean squared error I was doing 'y - predicted' instead of ignoring the None values. Thanks for pointing it out. I will fix it and link you to the commit.

Thanks! Xiangyu

On Mon, Jun 19, 2017 at 11:18 PM, sun137653577 [email protected] wrote:

hi Samuel, Sorry for bothering again, I want to use ignore and tune method in myDLM, while it is error when run following code:

from pydlm import dlm, trend, seasonality, dynamic, autoReg, longSeason y=[1,2,3,4,5,6,7,8,9,10] myDLM=dlm(np.array(y)) myDLM=myDLM+trend(1,discount=0.9)+seasonality(period=4) myDLM.fit() (temp_obser, temp_var)=myDLM.predict(date=9)

myDLM.ignore(2) myTuner=modelTuner(method='gradient_descent',loss='mse') bankDLM=myTuner.tune(myDLM, maxit=100) tuned_discounts=myTuner.getDiscounts() myDLM.fit()

TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

If we comment 'myDLM.ignore(2)', then the codes will run successfully. I wonder whether tune method can't accept missing data?

Another question, where is it appropriate to place myDLM.ignore(2), before myDLM.fit() or after myDLM.fit()？To be on the safe side, i add fit both before and after myDLM.ignore(2). thanks.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/wwrechard/pydlm/issues/13, or mute the thread https://github.com/notifications/unsubscribe-auth/AIDNcpQX9Bb8Q5_Xk7VaJaajBo3vday1ks5sF2RJgaJpZM4N_IXB .

-- Xiangyu (Samuel) Wang Google, MTV Email: [email protected] [email protected] [email protected]

[wwrechard]

You should place .ignore() after .fit()

Another unrelated issue is that after you tuned the model, i.e.,

bankDLM=myTuner.tune(myDLM, maxit=100)

you should use 'bankDLM' as the tuned DLM, i.e.,

bankDLM.fit()

'myDLM' will remain unchanged as before (mainly for comparison).

On Mon, Jun 19, 2017 at 11:25 PM, Samuel [email protected] wrote:

Hi Sun,

Thanks for your email. I believe there is a bug, because when computing the mean squared error I was doing 'y - predicted' instead of ignoring the None values. Thanks for pointing it out. I will fix it and link you to the commit.

Thanks! Xiangyu

On Mon, Jun 19, 2017 at 11:18 PM, sun137653577 [email protected] wrote:

hi Samuel, Sorry for bothering again, I want to use ignore and tune method in myDLM, while it is error when run following code:

from pydlm import dlm, trend, seasonality, dynamic, autoReg, longSeason y=[1,2,3,4,5,6,7,8,9,10] myDLM=dlm(np.array(y)) myDLM=myDLM+trend(1,discount=0.9)+seasonality(period=4) myDLM.fit() (temp_obser, temp_var)=myDLM.predict(date=9)

myDLM.ignore(2) myTuner=modelTuner(method='gradient_descent',loss='mse') bankDLM=myTuner.tune(myDLM, maxit=100) tuned_discounts=myTuner.getDiscounts() myDLM.fit()

TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

If we comment 'myDLM.ignore(2)', then the codes will run successfully. I wonder whether tune method can't accept missing data?

Another question, where is it appropriate to place myDLM.ignore(2), before myDLM.fit() or after myDLM.fit()？To be on the safe side, i add fit both before and after myDLM.ignore(2). thanks.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/wwrechard/pydlm/issues/13, or mute the thread https://github.com/notifications/unsubscribe-auth/AIDNcpQX9Bb8Q5_Xk7VaJaajBo3vday1ks5sF2RJgaJpZM4N_IXB .

-- Xiangyu (Samuel) Wang Google, MTV Email: [email protected] [email protected] [email protected]

-- Xiangyu (Samuel) Wang Google, MTV Email: [email protected] [email protected] [email protected]

[sun137653577]

thank you. this is my typo, actually, bankDLM is my actual model. if we place .ignore() after .fit(), there should be another .fit() after .ignore(), the format should be myDLM.fit() myDLM.ignore() myDLM.fit() Per my understanding, .ignore is like data preprocess, thus we need to refit the model, am I right?

I have tested ignore place for both cases in my actual model, myDLM.fit() myDLM.ignore(2) myDLM.fit() and myDLM.ignore(2) myDLM.fit() there will get two different results, but the results are the same in y=[1,2,3,4,5,6,7,8,9,10], i don't understand the methodology for .ignore, do you think is it possible to get such different results in some situations? thanks.

[wwrechard]

I think previously it is not allowed to put .ignore() before a .fit(), because I though usually you want to ignore a data only when you fit the model and see the plots of the results. But then someone complained to me so I changed the code so that you can place .ignore() before ,fit()

The latter two cases are expect to give you the SAME result. What .ignore(t) does is that it reverts the model to an uninitialized state (a clean model, like a model just built) and then replaces the data at time t with None (i.e., missing data). So any fitting step before myDLM.ignore(2) is useless and has no impact on the results after .ignore() is called. So what happened in the first situation

myDLM.fit() // fit a model (no impact on the third line) myDLM.ignore(2) // if fitted then undo fit. Replace data[2] with 'None' myDLM.fit() // fit model

is equivalent to the second situation.

On Mon, Jun 19, 2017 at 11:58 PM, sun137653577 [email protected] wrote:

thank you. this is my typo, actually, bankDLM is my actual model. if we place .ignore() after .fit(), there should be another .fit() after .ignore(), the format should be myDLM.fit() myDLM.ignore() myDLM.fit() Per my understanding, .ignore is like data preprocess, thus we need to refit the model, am I right?

I have tested ignore place for both cases in my actual model, myDLM.fit() myDLM.ignore(2) myDLM.fit() and myDLM.ignore(2) myDLM.fit() there will get two different results, but the results are the same in y=[1,2,3,4,5,6,7,8,9,10], i don't understand the methodology for .ignore, do you think is it possible to get such different results in some situations? thanks.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/wwrechard/pydlm/issues/13#issuecomment-309662722, or mute the thread https://github.com/notifications/unsubscribe-auth/AIDNcic6KULkzCgQbcWSUacv7KnRxangks5sF22ngaJpZM4N_IXB .

-- Xiangyu (Samuel) Wang Google, MTV Email: [email protected] [email protected] [email protected]

[sun137653577]

I will double check my codes. And, can we ignore a date list in .igore()?