Test suite for combination of settings

[lukesonnet]

Build a test suite which tests the different instances of KRLS and KRLogit. These will be fast and simple tests that just make sure that they pass, not necessarily to check that the answers are all correct. This should test the interaction of:

Top priority:

• Loss function: 'leastsquares', 'logistic'
• Truncation: T/F
• Lambdasearch: fixed, optimize, grid
• whichkernel: gaussian, poly2
• standarderrors: regular, robust, clustered

Lower priority:

• bsearch

[lukesonnet]

I've created a test file within the package that tests all of the configurations of krls. It essentially searches over all possible combinations of these parameters:

• Loss function: 'leastsquares', 'logistic'
• Truncation: T/F
• Lambdasearch: fixed, optimize, grid
• whichkernel: gaussian, poly2
• standarderrors: regular, robust, clustered
• weights: yes/no

2x2x3x2x3x2 = 144 possible combinations.

For some combinations, nothing works, not even finding a solution for the coefficients. For example, trying to use weights without truncation in KRLS is currently not supported at all. In those cases, the test suite checks that there is an error thrown in the fitting process, ie. during krls.

For other combinations, fitting the coefficients is possible, but we've thrown errors when getting derivatives. For example, when trying to get pwmfx with non-gaussian kernels. In those cases, the test suite checks that fitting coefficients, the krls function, passes without error but that it errors out when using summary.krls2.

In other cases, the fitting works and the pwmfx work. In that case, the test suite confirms that no errors are thrown in either krls or summary.krls2.

[lukesonnet]

I added two testing services, Travis and Appveyor. These check new commits to make sure that the package can still compiles and they run the test suite as well. Travis does this in a *nix environment while Appveryor does it in a Windows environment. You can see the current status of the package on those platforms on our main page: https://github.com/lukesonnet/KRLS

[chadhazlett]

Great

On Jul 29, 2017 13:26, "Luke Sonnet" [email protected] wrote:

I added two testing services, Travis and Appveyor. These check new commits to make sure that the package can still compiles and they run the test suite as well. Travis does this in a *nix environment while Appveryor does it in a Windows environment. You can see the current status of the package on those platforms on our main page: https://github.com/lukesonnet/KRLS

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/lukesonnet/KRLS/issues/7#issuecomment-318845777, or mute the thread https://github.com/notifications/unsubscribe-auth/AFVCyBRNyJseqs1_zx3d8tjP6myCpxcgks5sS2tCgaJpZM4OnRk1 .