Jme test api

[pavly-gerges]

This is an initial implementation for the testable api, targeting issue #1649, which suggests building a better test chooser backend for jme3-examples, where one can pickup which test cases to run inside a package.

This is a video demonstrating testables in action :

FAQ:

How can i make my own test ?: A: Your test should extends JmeTestApp or JmeTestState, in either cases, you will override launch() method and add what you want to test there, whether a JmeApplication.attachState(this); for AppStates or setting up AppSettings and start() or setting up a JmeSurfaceView for android.

Does this approach provide a recursive way of opening a package ? A; Yep, all you need to do, is to give the package, which can be done by 2 methods : METHOD-A :

 TestableExecutor.execute("jme3test", "--No-Data--", signatures);

METHOD-B :

 TestableExecutor.execute(Main.class.getPackage().getName(), "--No-Data--", signatures);

Can i test multiple packages that aren't on the same tree, at the same time ? A: Yep, just use :

TestableExecutor.execute(new String[] {"jme3test.app", "jme3test.animation"}, "--No-Data--", signatures);

Can i use multiple signatures in the same @Test annoation ? A: Yep, use this :

@Annotations.Test(signatures = {Launcher.SIG_WATER_FILTERS, Launcher.SIG_ALL})

But, no @Test repetition...it complicates simple stuff and not clean anyway.

What will happen if i have non-class files on my package A: Don't worry, the utility skips those.

What will happen if i have other classes not implementing the Testable ? A: Those are skipped too.

Can i use this on android ? A: Theoretically, you can, but i haven't found the time to test that yet.

What will happen if i extended JmeTestApp and didn't add a @Test with a signature ? A: The utility will exclude this test, because how should it know 'is it ready to be executed?', if you want to run all Testables under a package then use a unified signature in all those classes, for example, add this to all the testables you want to run Launcher.SIG_ALL while you can still use other signatures to filter your tests for other run configurations :

@Annotations.Test(signatures = {Launcher.SIG_WATER_FILTERS, Launcher.SIG_ALL})

I think that is it, for now, if that is what you are seeking, then i hope it will be helpful, please give a quick review.

EDIT: An example:

1. The Testable class :

/*
 * Copyright (c) 2009-2021 jMonkeyEngine
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are
 * met:
 *
 * * Redistributions of source code must retain the above copyright
 *   notice, this list of conditions and the following disclaimer.
 *
 * * Redistributions in binary form must reproduce the above copyright
 *   notice, this list of conditions and the following disclaimer in the
 *   documentation and/or other materials provided with the distribution.
 *
 * * Neither the name of 'jMonkeyEngine' nor the names of its contributors
 *   may be used to endorse or promote products derived from this software
 *   without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
 * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
 * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

package jme3test.app.test;

import com.jme3.system.AppSettings;
import com.jme3.testable.impl.JmeAppTest;
import com.jme3.scene.Geometry;
import com.jme3.scene.shape.Box;
import com.jme3.system.Annotations;

import jme3test.Launcher;

/**
 * Test a bare-bones application, without SimpleApplication.
 */
@Annotations.TestableTags({Launcher.SIG_ALL})
public class BareBonesAppTest extends JmeAppTest<AppSettings> {

    private Geometry boxGeom;

    public static void main(String[] args){
        AppSettings settings = new AppSettings(true);
        settings.setRenderer(AppSettings.LWJGL_OPENGL2);
        settings.setAudioRenderer(AppSettings.LWJGL_OPENAL);
        new BareBonesAppTest().launch(settings);
    }


    @Override
    public void simpleUpdate(float tpf) {
        super.simpleUpdate(tpf);
        boxGeom.rotate(tpf * 2, tpf * 4, tpf * 3);
    }

    @Override
    public void simpleInitApp() {

        // create a box
        boxGeom = new Geometry("Box", new Box(2, 2, 2));

        // load some default material
        boxGeom.setMaterial(assetManager.loadMaterial("Interface/Logo/Logo.j3m"));

        // attach box to display in primary viewport
        rootNode.attachChild(boxGeom);
    }


    @Override
    public void launch(AppSettings userData) {
        super.launch(userData);
        setSettings(userData);
        setShowSettings(false);
        start();
    }
}

2. The Launcher (main class) :

package jme3test;

import com.jme3.system.AppSettings;
import com.jme3.util.TestableExecutor;

import java.lang.reflect.InvocationTargetException;

/**
 * Tests the new jme test api.
 *
 * @author pavl_g.
 */
public class Launcher {
    public static final String SIG_ALL = "ALL";
    public static final String SIG_WATER_FILTERS = "WATER-FILTER";
    public static final String PBR = "PBR";

    public static final String[] signatures = new String[] {
            SIG_ALL
    };

    public static void main(String[] args) throws InstantiationException, IllegalAccessException, ClassNotFoundException, NoSuchMethodException, InvocationTargetException {
        final AppSettings settings = new AppSettings(true);
        settings.setRenderer(AppSettings.LWJGL_OPENGL2);
        settings.setAudioRenderer(AppSettings.LWJGL_OPENAL);
        TestableExecutor.getInstance().launch(new String[] {"jme3test"}, settings, signatures);
    }
}

EDITED: (After moving to generics). EDITED2: (After replacing signatures with tags).

[stephengold]

Since this PR is not ready to be integrated, could you mark it as a draft, please?

[pavly-gerges]

Since this PR is not ready to be integrated, could you mark it as a draft, please?

Yeah sure, but i am keeping it open so you don't ignore it, i just need to be sure of the general pattern of this PR, is that sufficient for improving jme3 examples ??

[stephengold]

At the moment, I'm busy with other tasks, so I haven't looked closely at this PR.

[capdevon]

Hi Pavl, I appreciate your effort, but it doesn't seem like a core feature to me. If approved, I think it should be in the jme3-examples module. In this way, you do not have to wait for the publication of a new version of the engine, to correct any errors in the tests. In any case, I still have many doubts about the usefulness of this architecture. I would keep the examples simple without non-functional burdens. It might be an external project that uses the jme3-examples module as a library, but that's just my point of view.

[pavly-gerges]

Hello Wyatt, thanks for your opinion, I was hesitating about the package too, I don't really know what the right thing to do about this, because this is a backend api (so there are abstraction layers that should be at the core level), so I guess I will wait about more opinions or another idea.

In any case, I still have many doubts about the usefulness of this architecture.

This is a backend code for testing jMonkeyEngine apps and states, this should replace the current backend of TestChooser and should work on android too (still preparing a testcase), the brief of the feature is that it searches a package for Testables and launch only tagged ones in synchronicity.

This is a requested feature, I wouldn't have written it, if it wasn't a requested feature in the issue.

EDIT: May be one of the strong reasons to keep it at the core level, is that it supports android too and you could do your own Testables and test your own apps and states (since it's an API), so I think it wouldn't be appropriate if you added the module jme3-examples to implement the jme3 Testable API for your project.

EDIT2: As for publication, the feature is now ready and fully mature, with desktop prove cases, one thing that's missing is a prove case for android..you could make a quick review.

EDIT3: For example, i could even make a good use of this new api, to fix this issue #632 too, later on, so it's pretty useful when thinking out of the box.

[pavly-gerges]

I will separate this into a plugin soon, you can find it available on my repositories later. There is no need to clog-up the core with a new API, though the mere purpose of this PR was the opened issue with the bizarre requirements anyway...