[Feature Request] Custom input processing in TAStudio

[kierio04]

Premise

Currently, TAStudio only supports non-altered input processing (i.e. A being pressed in TAStudio results in A being pressed in the game, an analog stick at value of 96 on the X axis results in the analog stick being put at 96 on the X axis in the game). In most situations, there is no need to go beyond this behaviour. However, some games would benefit from being able to customise how the inputs are processed.

Setup idea

There would be a section in TAStudio dedicated to how inputs are processed. A dropdown menu would provide access to selecting default or custom options. There would be buttons for creating ("New") or opening/editing ("Edit") the input definition files. More info on definition files below.

.tasproj files would include a reference to the definition file in the header. Without this reference, TAStudio should just assume the default format for backwards compatibility. With this reference, the definition file must be located in a pre-determined directory, comparable to a savestate or an avi dump.

Default example

Below is pseudocode for the default format definitions of select input options

input_A = tastudio_A
input_B = tastudio_B
input_analog_X = tastudio_analog_X
input_analog_Y = tastudio_analog_Y

Custom examples

Below is pseudocode for notched analog format definitions

analogs_translated = [59, 68, 77, 86, 95, 104, 112, 128, 152, 161, 170, 179, 188, 197, 205]
input_analog_X = analogs_translated[tastudio_analog_X + 7]
input_analog_Y = analogs_translated[tastudio_analog_Y + 7]
// e.g. 0 becomes 128 (neutral), -7 becomes 59 (full left), 7 becomes 205 (full right)

Other: angle/magnitude format

Summary

This may seem like a really ambitious minor improvement to TAStudio, but this could bring an increase in TASing efficiency for some angular-dependent or input-unspecific games. Please reply to this if you need any further clarification as I don't expect this to be fully understood for everybody at first read.

[YoshiRulz]

You're right, I don't understand it at all. The analog octagonal/notched constraint is already available for N64, and it's part of the IController stack so IIRC it applies to TAStudio. Polar representation is tracked as #581 + #2419. What additional purpose do programmable mappings serve?

[kierio04]

The analog octagonal/notched constraint is already available for N64

Maybe “notched” was poor word choice… some games’ inputs are compressed to those less precise than, for example, -127 to +127, and are converted to, example again, -7 to +7 (this is the demo pseudo code I had in the original post) based on given ranges within the -127 to +127. This isn’t some clamp restricting the range of inputs, it’s a compression of more precise inputs into less precise ones.

What additional purpose do programmable mappings serve?

It’s worth noting that it would be possible to just have an option in one of the toolbar drop downs that opens a mini window for setting up custom input decompression ranges. That being said, even though pure polar inputs are going to probably have functionality by adding virtual pad into TAStudio itself, there are some games with variable camera angles that would benefit from a programmatic input given input angle/magnitude, and camera angle. Doing this without some ability to programmatically map inputs would be near if not totally impossible. Besides, just because nobody has requested this idea doesn’t mean there might not be purposes of its use invented as a result of it being added and the option to programmatically map inputs being given to TASers.

[YoshiRulz]

But we already have Lua for that?

[kierio04]

How so? Does this exist already?

[YoshiRulz]

You can certainly read the camera rotation from memory, calculate which way to hold the stick, and write it to TAStudio from Lua.

[kierio04]

That still requires a desired facing rotation, which is what you’d be wanting to edit live, like inputs. Writing something to TAStudio is not the aim of this. Reading something from TAStudio is.

You input, say 16384 in a game’s angle units, which is converted to viable inputs given the camera angle, which are read by the game. That doesn’t work out so you change it to 16284 and test, 16184 and test, etc.

[YoshiRulz]

That would be difficult without control over the Virtual Pad from Lua, but I'm sure you could create something like it with drawings. Switching to C# and re-using the custom control would be easier. edit: Or if you only care about stepping the value, skip the GUI and make custom hotkeys with input.get.

[kierio04]

If C# is used to take the inputs from TAStudio to the game itself then do it with C#. I just suggested Lua because I thought that’d be easier to work with for scripting. If there exists something already to create a custom controller mapping like this then reusing it seems like a good plan, what exactly is it though? Lastly, I was only skipping values as an example, that wouldn’t be the general usecase, and if I understand TAStudio correctly, that’s not something you’d do in it.

[YoshiRulz]

By "custom control" I meant the VirtualPadAnalogStick control used in the Virtual Pad tool. I thought it would be possible to use that in ext. tools, but I now see it depends on some EmuHawk internals so it can't be.

I'm going to stop arguing and just write you a script.

[YoshiRulz]

This took ages because maths is hard. Fill in calc_camera_rotation, set axis_name_x/axis_name_y and bounds_x/bounds_y to the correct values, and optionally change ui_pos_desired and the client.SetGameExtraPadding call (I couldn't be bothered to calculate this from ui_pos_desired).

-- customisable Virtual Pad
-- by YoshiRulz, released under GPL v3+
-- version 2022-05-25

local pol2rec = function(p) return { p[1] * math.cos(p[2]), p[1] * math.sin(p[2]) }; end;
local rec2pol = function(p)
	local th = math.atan2(p[2], p[1]);
	if th < 0 then th = th + 2 * math.pi; end
	return { math.sqrt(p[1] * p[1] + p[2] * p[2]), th };
end;

local axis_name_x = "P1 X Axis";
local axis_name_y = "P1 Y Axis";
local bounds_x = { -128, 127 };
local bounds_y = { -128, 127 };
local bounds_w = math.abs(bounds_x[2] - bounds_x[1]);
local bounds_h = math.abs(bounds_y[2] - bounds_y[1]);
-- this outer bounds nonsense is so the red dot following the cursor has enough space to get the rotated red dot to any valid point
-- btw everything breaks if the range isn't a nice square centered at 0 so don't do that
local bounds_outer_temp = pol2rec({ rec2pol({ math.max(math.abs(bounds_x[1]), math.abs(bounds_x[2])), math.max(math.abs(bounds_y[1]), math.abs(bounds_y[2])) })[1], 0 })[1];
local bounds_outer_x = { -bounds_outer_temp, bounds_outer_temp };
local bounds_outer_y = { -bounds_outer_temp, bounds_outer_temp };
local scythe_r_scale = 2 * math.sqrt(2);
local scythe_th = math.pi / 4;
local ui_pos_desired = { -200, 250 };
local zero_scythe = { { 0, 0 }, { 0, 0 }, { 0, 0 }, 0, 0 };

local clamp = function(n, range)
	if n < range[1] then return range[1]; end
	if range[2] < n then return range[2]; end
	return n;
end;
local clamp_xy = function(p, range_x, range_y) return { clamp(p[1], range_x), clamp(p[2], range_y) }; end;
local rad2deg = function(r) return r * 180 / math.pi; end;
local calc_camera_rotation = function()
	return -math.pi / 4;
end
local calc_scythe_data = function(pos, camera_rotation)
	local pos_polar = rec2pol(pos);
	local rotated = clamp_xy(pol2rec({ pos_polar[1], pos_polar[2] + camera_rotation }), bounds_x, bounds_y);
	return {
		pos, -- p
		rotated, -- q
		pol2rec({ rec2pol(rotated)[1] * scythe_r_scale, scythe_th }), -- arc_wh
		pos_polar[2], camera_rotation -- arc_a, arc_b
	};
end;

local state = {
	camera_rotation = -math.pi / 4, -- must be negative I think
	camera_rotation_prev = -math.pi / 4, -- not updated properly when loadstating!
	ghost = zero_scythe,
	ghost_frame = 0,
	hover = zero_scythe,
	is_movie = false,
	pending = { 0, 0 },
	ui_pos_x = 0,
	ui_pos_y = 0,
	update = function(self)
		local frame = emu.framecount();
		local game_scale = client.getwindowsize();
		local raw_mouse = input.getmouse();
		self.camera_rotation_prev = self.camera_rotation;
		self.camera_rotation = calc_camera_rotation();
		if self.is_movie then
			if frame > 0 then
				local prev_input = movie.getinput(frame - 1);
				self.ghost = calc_scythe_data({ prev_input[axis_name_x], prev_input[axis_name_y] }, self.camera_rotation_prev);
			else
				self.ghost = zero_scythe;
			end
		end
		self.hover = calc_scythe_data(
			clamp_xy({
				raw_mouse["X"] * game_scale - self.ui_pos_desired[1],
				raw_mouse["Y"] * game_scale - self.ui_pos_desired[2]
			}, bounds_outer_x, bounds_outer_y),
			self.camera_rotation
		);
		if raw_mouse["Left"] then
			self.pending = self.hover[2];
			tastudio.submitanalogchange(frame, axis_name_x, self.hover[1][1]);
			tastudio.submitanalogchange(frame, axis_name_y, self.hover[1][2]);
			tastudio.applyinputchanges();
		end
	end,
	draw_scythe = function(self, px, py, qx, qy, arc_w, arc_h, arc_a, arc_b, stroke, fill_p, fill_q)
		gui.drawPie(self.ui_pos_x - arc_w / 2, self.ui_pos_y - arc_h / 2, arc_w, arc_h, rad2deg(arc_a), rad2deg(arc_b), stroke, "transparent");
		gui.drawEllipse(self.ui_pos_x + px - 2, self.ui_pos_y + py - 2, 5, 5, fill_p, fill_p);
		gui.drawEllipse(self.ui_pos_x + qx - 2, self.ui_pos_y + qy - 2, 5, 5, fill_q, fill_q);
	end,
	draw = function(self)
		gui.drawRectangle(self.ui_pos_x + bounds_x[1], self.ui_pos_y + bounds_y[1], bounds_w, bounds_h, "gray", "gray");
		gui.drawEllipse(self.ui_pos_x + bounds_x[1], self.ui_pos_y + bounds_y[1], bounds_w, bounds_h, "white", "white");
		self:draw_scythe(
			self.ghost[1][1], self.ghost[1][2],
			self.ghost[2][1], self.ghost[2][2],
			self.ghost[3][1], self.ghost[3][2],
			self.ghost[4], self.ghost[5],
			"gray", "transparent", "gray"
		);
		gui.drawEllipse(self.ui_pos_x + self.pending[1] - 2, self.ui_pos_y + self.pending[2] - 2, 5, 5, "lime", "lime");
		self:draw_scythe(
			self.hover[1][1], self.hover[1][2],
			self.hover[2][1], self.hover[2][2],
			self.hover[3][1], self.hover[3][2],
			self.hover[4], self.hover[5],
			"blue", "red", "lime"
		);
	end,
};

gui.use_surface("client");
client.SetGameExtraPadding(200, 0, 0, 0);
local ui_offset = client.transformPoint(0, 0);
state.ui_pos_x = state.ui_pos_desired[1] + ui_offset.x;
state.ui_pos_y = state.ui_pos_desired[2] + ui_offset.y;
state.is_movie = movie.mode() ~= "INACTIVE";
while true do
	state:update();
	state:draw();
	emu.yield();
end

[kierio04]

So what does this script do and how would you go about using it given the scenario I had outlined?

[YoshiRulz]

It draws an interactive Virtual Pad-like stick on the main window which compensates for camera movement. Left click to set the point under the cursor as the pending value, then frame advance to write it to TAStudio. The previous frame's value is shown in grey. For more precision, you could add stepping hotkeys, or just make the stick twice the size so it's easier to click on.