bashtage
Replace `assert` statements used for runtime validation
In many files, assert __ is not None statements are repeatedly used outside of unit tests (e.g., in asset_pricing/covariance.py, iv/absorbing.py, system/model.py, etc.). Running Python in optimization mode (with -O or -OO) will fail to check these assert statements. Beyond this, they're simply less descriptive as generic AssertionErrors compared to, say, TypeErrors with a comprehensive error message (which can explain why we need these assert __is not None checks in the first place). I propose we replace all non-test assert statements with TypeErrors/ValueErrors is appropriate across the codebase; I've already begun drafting a PR. Let me know if, for whatever reason, this is an intentional design choice.
These exist purely for use by mypy and are not essential for correct operation of the code.
Basically you have a variable that is say Optional[ndarray] and is initiated to None. By the time you hit some point in code, I know it has to be an ndarray. When I assert that this variable is not none. This tells my pie that it needs to rule out the None type from the variable type.
But I opened this issue because I myself was using linearmodels and ran into an error with assert endog is not None while constructing an IV-2SLS model. I'll try and find the exact file if you want, but it did get to the point where, without poking around the API unnecessarily (just using normal IV-2SLS stuff), I received that AssertionError instead of it being impossible due to previous checks in the code. If I'd run it with python -O, for instance, then the model would've broken.
That sounds like a bug. Clearly my assertion to get typing correct is wrong, or the code would eventually fail somewhere else. Without looking at code, I think endog is always required.
I ran into the error a few days ago, so I can't tell off the top of my head, but I'll try and reproduce that issue with an IV-2SLS model without an endog. All I know is that I accidentally forgot to include an endog and instead of some other sort of error, I ran into an AssertionError with that exact message (assert endog is not None). I'm about to sleep but I'll take a look tomorrow; you can too, if you like :)
I took a brief look at iv/model.py, and I think without an endog I managed to create the model (not fit it or anything, just create a model instance from a formula) and call the predict method on the _IVModelBase class. There were no prior checks for endog to not be None, so I got the AssertionError.
Might be a bug. Should add tests.
On Thu, 17 Apr 2025, 07:23 Luis M. B. Varona, @.***> wrote:
I took a brief look at iv/model.py, and I think without an endog I managed to create the model (not fit it or anything, just create a model instance from a formula) and call the predict method on the _IVModelBase class. There were no prior checks for endog to not be None, so I got the AssertionError.
— Reply to this email directly, view it on GitHub https://github.com/bashtage/linearmodels/issues/638#issuecomment-2811892917, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKTSRMKOCNDLV22IF36CT32Z5CGTAVCNFSM6AAAAAB3J7MTLWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMJRHA4TEOJRG4 . You are receiving this because you commented.Message ID: @.***> Luis-Varona left a comment (bashtage/linearmodels#638) https://github.com/bashtage/linearmodels/issues/638#issuecomment-2811892917
I took a brief look at iv/model.py, and I think without an endog I managed to create the model (not fit it or anything, just create a model instance from a formula) and call the predict method on the _IVModelBase class. There were no prior checks for endog to not be None, so I got the AssertionError.
— Reply to this email directly, view it on GitHub https://github.com/bashtage/linearmodels/issues/638#issuecomment-2811892917, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKTSRMKOCNDLV22IF36CT32Z5CGTAVCNFSM6AAAAAB3J7MTLWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMJRHA4TEOJRG4 . You are receiving this because you commented.Message ID: @.***>
Alright, you'll add tests or you're asking me to? (For context, the assert statement was in the predict method of _IVModelBase. I managed to create an _IVModelBase instance without an endog, but then I called predict and the error was thrown.)
Prediction from new exog is a good question. I suppose the only official way is to use the correct X and W and any ole Y and Z to create a new model, and then to make the predictions. Sort of a bug/feature. I think statsmodels supports a class method to make predictions from new data, which would probably be a good approach here.
@bashtage Just following up to see if you wanted to do anything about this matter. 🙂
Prediction from new exog is a good question. I suppose the only official way is to use the correct X and W and any ole Y and Z to create a new model, and then to make the predictions. Sort of a bug/feature. I think statsmodels supports a class method to make predictions from new data, which would probably be a good approach here.
Hi @bashtage, just following up again!
I took a brief look at
iv/model.py, and I think without an endog I managed to create the model (not fit it or anything, just create a model instance from a formula) and call the predict method on the _IVModelBase class. There were no prior checks for endog to not be None, so I got the AssertionError.
I don't have the code on hand, but I did say this a few months ago. I can definitely reproduce a working example and put the script here sometime later.
Hi @bashtage, that makes a lot of sense… I've been trying to no avail to reproduce the issue, and I was very confused ¯_(ツ)_/¯ Good to know it's already fixed!