Convert the For objects to state object arguments

[dphfox]

Right now, we have three different For objects, depending on which arguments need to be used. We do this so we can skip some recomputations when the key or the value doesn't matter.

However, Fusion already has a mechanism for ignoring things you're not using - the reactive graph doesn't update things that don't need to be updated already.

So, my theory is that we could replace the keys and values passed to these functions with state objects. If you want to use the key or the value in your computation, you use() it, and the objects' pre-existing dependency management system can handle recomputation when it changes. This unlocks a very powerful dynamic optimisation; we only run the processor when there's a new unique key-value pair, and there isn't a previous processor invocation available to take over the new pair. That is to say, we effectively unpin the processor callback from representing any particular pair, which means we can reduce destructor calls and effectively recycle almost every single pair we generate.

The extra-cool news about a system like this is that we no longer need all three For objects - just ForPairs alone can represent all three's optimisation patterns, and even more dynamic optimisation patterns enabled by Fusion's dynamic dependency capturing system.

[dphfox]

Related to #205

[dphfox]

Perhaps we should prototype this first with a dedicated object and use it as a way to unify the existing objects under the hood. We can evaluate later whether the specific objects are necessary.

[funwolf7]

I decided to attempt to code this up and finished a prototype, but when going to test it, I ran into a problem.

Consider the case where someone wants to recreate the current ForValues. Say you want to map an array to another array, where the keys should always stay the same. If an entry moves around, we shouldn't need to recompute it, as by using ForValues, we tell Fusion that the index is not included in the calculation.

With this system where there is only one For object and the key and value are both state objects, how would I do this? I need to return the key I want it mapped to, so I need to call use on the key then return it. However, Fusion now has to assume that the value may be based off of the key, so it needs to run the function again, even when only the key changes.

local array = Value({1, 2, 3})
local result = GenericFor(array, function(use, key, value)
	print("ran")
	return use(key), use(value) * 10
end) --> ran (x3)
print(peek(result)) --> {10, 20, 30}

-- With the current ForValues, this doesn't need to run the function again
-- With the new idea, we are forced to run it 3 times again
array:set({2, 3, 1}) --> ran (x3)
print(peek(result)) --> {20, 30, 10}

The best way to get around this that I can think of is to allow returning nil as the key/value, in which case it will simply use the input key/value as the output key/value.

[funwolf7]

An alternative to the above idea of returning nil is to allow returning state objects, which will then automatically change the table (without running the processor function again) when they are changed. I didn't think this would be possible at first in my implementation, but I looked back and it's a pretty easy change.

A potential drawback of this is that a user may genuinely want to have a state object as a key/value (to create a nested state object). This could be worked around by wrapping it in a Value, but it would be annoying to have to do.

[funwolf7]

I experimented with allowing the processor to return StateObjects and I really like it. It allows for more complicated processors that run less often. Consider the following example, where extremelyExpensiveFunction is something that you want to run as little as possible:

local function extremelyExpensiveFunction(input: number): number
	print("ran the extremely expensive function (very bad)")
	return input + 1
end

local test = Value({
	a = 1,
	b = 2,
	c = 3,
})

-- runs the extremely expensive function 3 times
local thing = GenericFor(test, function(use, key, value)
	return Computed(function(innerUse)
		return innerUse(key) .. " Modified"
	end), extremelyExpensiveFunction(use(value))
end)


print(peek(thing))

-- does not run the extremely expensive function
test:set({
	A = 1,
	B = 2,
	C = 3,
})

print(peek(thing))

-- runs the extremely expensive function twice
test:set({
	A = 1,
	B = 4,
	C = 3,
	D = 10,
})

print(peek(thing))

-- does not run the extremely expensive function
test:set({
	a = 4,
	c = 10,
})

print(peek(thing))

You are able to have a modification done on the key without having to run the processor again. More generally, you can essentially separate the key and value processor if needed by using Computeds, allowing for functions to run less overall.

[dphfox]

I've been thinking about this too, but not quite with the clarity you present here. I think this is good for keeping complexity down, but I'll have to think about the syntax. Perhaps we keep the existing For objects as syntax sugar?