How could I record game from Keyboard control

[frogoscar]

@bzier @kevinhughes27

For the moment, TensorKart records from Joystick input control when calling record.py

As I do not have a joystick, I am trying to change the code so that it can record from Keyboard input control.

I read http://mupen64plus.org/wiki/index.php?title=KeyboardSetup and when I play the MarioKart, I use Left Shift to do "Go" and Up/Left/Down/Right to control the direction.

So should I change the code in https://github.com/kevinhughes27/mupen64plus-input-bot/blob/master/src/controller.c ? Need I also change code in https://github.com/kevinhughes27/TensorKart/blob/master/record.py ?

Is the file SDL_keysym.h http://www.libsdl.org/release/SDL-1.2.15/include/SDL_keysym.h useful ?

Thanks a lot!

[kevinhughes27]

The input plugin is how the AI controls the emulator. On each game loop the emulator asks the input plugin for the input and it makes a remote call to the AI server in order to get it. That's done in this file you linked: https://github.com/kevinhughes27/mupen64plus-input-bot/blob/master/src/controller.c. This process is not coupled to any real input device - its all virtual.

The emulator already has an abstraction between what input you are using and a virtual n64 controller. You only need to modify record.py to interpret keyboard actions and map them to controller actions before saving the training data in order to train using the keyboard.

The process will look like this:

1. Train data using the keyboard
2. Save the output as if it was a generic n64 controller not coupled to the actual input device used
3. Train the model
4. Play - this will play using a virtual n64 controller which is not related to the input device used.

[frogoscar]

@kevinhughes27

Thanks a lot. I will try.

[bzier]

The only other thing that I would add is that the current XboxContoller class is defined in utils.py, so it may be a combination of modifying that file in addition to record.py.

[bzier]

To be clear, that class is what is used to read the gamepad input. In your case, you would be providing similar functionality, but reading from the keyboard input instead. I just wanted to point out where that magic currently happens.

[frogoscar]

@bzier

Thanks a lot

[frogoscar]

@kevinhughes27

I think I should modify the utils.py to change the class XboxController to something like KeyboardController, right ?

class XboxController(object):
    MAX_TRIG_VAL = math.pow(2, 8)
    MAX_JOY_VAL = math.pow(2, 15)

    def __init__(self):

        self.LeftJoystickY = 0
        self.LeftJoystickX = 0
        self.RightJoystickY = 0
        self.RightJoystickX = 0
        self.LeftTrigger = 0
        self.RightTrigger = 0
        self.LeftBumper = 0
        self.RightBumper = 0
        self.A = 0
        self.X = 0
        self.Y = 0
        self.B = 0
        self.LeftThumb = 0
        self.RightThumb = 0
        self.Back = 0
        self.Start = 0
        self.LeftDPad = 0
        self.RightDPad = 0
        self.UpDPad = 0
        self.DownDPad = 0

        self._monitor_thread = threading.Thread(target=self._monitor_controller, args=())
        self._monitor_thread.daemon = True
        self._monitor_thread.start()


    def read(self):
        x = self.LeftJoystickX
        y = self.LeftJoystickY
        a = self.A
        b = self.X # b=1, x=2
        rb = self.RightBumper
        return [x, y, a, b, rb]


    def _monitor_controller(self):
        while True:
            events = get_gamepad()
            for event in events:
                if event.code == 'ABS_Y':
                    self.LeftJoystickY = event.state / XboxController.MAX_JOY_VAL # normalize between -1 and 1
                elif event.code == 'ABS_X':
                    self.LeftJoystickX = event.state / XboxController.MAX_JOY_VAL # normalize between -1 and 1
                elif event.code == 'ABS_RY':
                    self.RightJoystickY = event.state / XboxController.MAX_JOY_VAL # normalize between -1 and 1
                elif event.code == 'ABS_RX':
                    self.RightJoystickX = event.state / XboxController.MAX_JOY_VAL # normalize between -1 and 1
                elif event.code == 'ABS_Z':
                    self.LeftTrigger = event.state / XboxController.MAX_TRIG_VAL # normalize between 0 and 1
                elif event.code == 'ABS_RZ':
                    self.RightTrigger = event.state / XboxController.MAX_TRIG_VAL # normalize between 0 and 1
                elif event.code == 'BTN_TL':
                    self.LeftBumper = event.state
                elif event.code == 'BTN_TR':
                    self.RightBumper = event.state
                elif event.code == 'BTN_SOUTH':
                    self.A = event.state
                elif event.code == 'BTN_NORTH':
                    self.X = event.state
                elif event.code == 'BTN_WEST':
                    self.Y = event.state
                elif event.code == 'BTN_EAST':
                    self.B = event.state
                elif event.code == 'BTN_THUMBL':
                    self.LeftThumb = event.state
                elif event.code == 'BTN_THUMBR':
                    self.RightThumb = event.state
                elif event.code == 'BTN_SELECT':
                    self.Back = event.state
                elif event.code == 'BTN_START':
                    self.Start = event.state
                elif event.code == 'BTN_TRIGGER_HAPPY1':
                    self.LeftDPad = event.state
                elif event.code == 'BTN_TRIGGER_HAPPY2':
                    self.RightDPad = event.state
                elif event.code == 'BTN_TRIGGER_HAPPY3':
                    self.UpDPad = event.state
                elif event.code == 'BTN_TRIGGER_HAPPY4':
                    self.DownDPad = event.state

Thanks a lot.

[kevinhughes27]

Yup!

[frogoscar]

thanks

[jaskarannagi19]

@frogoscar any updates pal I am stuck on same I want to use keyboard instead of gamepad I dont have gamepad need to finish this by today