Nodes workflow requirements

[Durman]

Problem statement

Creating of new nodes is quite a big part of Sverckok development. It is not optimized now and has some problems. This issue is intend to clear what problems should be solved during nodes coding.

Problems

• Testing nodes whole code. #3932

To do such tests now is nearly impossible or quite hard task. It's not only node should be created in a Blender node tree but also node should be connected with other sockets where fake data should be placed.

• Simple reuse of node code for example in SNL scripts. #3939
• Porting Sverchok into another environment.

Probably someday we would like to use Sverchok somewhere else. Now with node code tightly bounded with Blender UI it's nearly impossible.

• Suspensibility of node evaluation (require update system changes)

If we would like to have really comfortable workflow the process of evaluation of each node should be cancelable. It can be done the easier the more breakpoints it has. It also requires changing in update system and most likely they should be done first.

• Multithreading (require update system changes) #3646 #4323

My guess is that until nodes are using socket.sv_get and socket.sv_set methods it's impossible to do any multithreading. The solution could be to put data management to update system.

• Vectorization #3954 #4068

Vectorization code is repeated in each node and could be separated from their code.

• Automatization of node creation (auto generating UI node code) #3437

Some part of UI node code probably could be auto generated but as it shows previous experiment it is hard task. Probably it does not worth efforts.

• Support muting nodes

[Durman]

I will take Find closest value node for prototyping new API on its example.

The simplest solution of solving many problems above is to create separate function for each node with implementation of main logic.

def find_closest_value_node(**props):
    data = props.get('data')
    value = props.get('value')
    mode = props.get('mode')

    # main logic

    # node can have different output
    if mode == 'a':
        return out_value, out_index
    else:
        return out_value


class BlenderNode:
    ...
    def process(self):

        # this code will be still bounded to UI and can't be tested
        # but my believe is that this code will be outdated sooner or later anyway 
        # and node functions can be used directly by new update system

        result = find_closest_value_node(data=self.inputs['Data'].sv_get(deepcopy=False), ..., mode=self.mode)
        self.outputs['Closest value'].sv_set(result[0])

Usage

• Unit tests code

from nodes.find_closest_value import find_closest_value_node

result = find_closest_value_node(data=a, value=b, mode=c)
self.assertEqual(result, expecting_result)

• Using in SNL scripts.

Possible usage is to create pipeline from nodes

from nodes import node_a, node_b, node_c

verts, faces = node_a(param1=a, param2=b)
data = node_b(param1=c)
result = node_c(param1=verts, param2=faces, param3=data)

• Vectorization

sockets_properties = {
    'socket_a': {'vectorize' = True},
    'socket_b': {'vectorize' = False}}

@vectorize(**sockets_properties)
def find_closest_value_node(**props):
   ...

• Suspensibility

Solution is to convert node function into iterator

@suspend(**sockets_properties)
def find_closest_value_node(**props):
    ...

def suspend(f, *, **sock_props):
    if f is None:
        return lambda func: suspend(func, **sock_props)

    def inner(**kwargs):
        for sock_params in vectorize(**kwargs):
            yield f(**sock_params)

    return inner

In general this looks like flexible and simple solution. However there are some concerns of passing data via such functions. A node can have huge number of input sockets, parameters and output sockets. Also inputs and output sockets can be changed by switching node parameters. All this can lead to greater number of mistakes in passing parameters from node function to node function.