joaopauloschuler

Regarding AVX/AVX2/AVX512, if your hardware has it, you should enable. This API is smart enough to use and to not to use it. The decision is based on `csMinAvxSize` constant....

OpenCL is actually slower in this experiment. I'm wondering if the number of weights/neurons is so small in this experiment that OpenCL has no advantage.

I'm about to start working on this.

On dense (fully connected layers), OpenCL is called only when there is enough neurons/weights to compensate the overhead that it adds: ``` FShouldOpenCL := (FNeurons.Count >= 512) and (pPrevLayer.Output.Size >=...

I've just tested the following and it works for me: ``` program Hypotenuse; (* Hypotenuse: learns how to calculate hypotenuse sqrt(X^2 + Y^2). Copyright (C) 2019 Joao Paulo Schwarz Schuler...

I've just tested the following and it also works for me: ``` //NFit.MaxThreadNum := 1; NFit.Fit(NN, TrainingPairs, nil, nil, {batchsize=}32, {epochs=}50); ``` and ``` NN.AddLayer([ TNNetInput.Create(2), TNNetFullConnectReLU.Create(512), TNNetFullConnectReLU.Create(512), TNNetFullConnectReLU.Create(512), TNNetFullConnectLinear.Create(1)...

In the case that it helps, this is how neural.cl is loaded: ``` constructor TNeuralKernel.Create(pCurrentPlatform: cl_platform_id; pCurrentDevice: cl_device_id; kernelname: string = 'cai_dot_product'); begin inherited Create(); SetCurrentPlatform(pCurrentPlatform); SetCurrentDevice(pCurrentDevice); // Create the...

Hi @Dzandaa, For a `Layers[LayerCnt]` layer, except for concatenating layers and input layer, you can get the input via: `NNAutoencoder.Layers[LayerCnt].PrevLayer.Output;` The `PrevLayer` gives you the previous layer. Then, you can...

Denoising is one of the most interesting applications for neural networks in my opinion. I should dedicate more time for it myself. In the case that you publish your source...

@Kryuski , Many thanks for reporting. I'm reopening until I can reproduce and apply a fix to the source code.