neural-api
neural-api copied to clipboard
joaopauloschuler
Enabling OpenCL causes access violation on a certain net structure
First ... Great Work! I'm still struggling a bit since that is a BIG library but I think it's great!
Following your examples I tried to use the lib on a ECG problem using the following code:
NN := TNNet.Create;
inputLayer := TNNetInput.Create(numFeatures);
NN.AddLayer( inputLayer );
NN.AddMovingNorm(False, inputLayer);
NN.AddLayer( TNNetConvolutionReLU.Create( 20, 50, 0, 1 ) );
NN.AddLayer( TNNetMaxPool.Create( 2 ));
NN.AddLayer( TNNetConvolutionReLU.Create( 20, 50, 0, 1 ) );
NN.AddLayer( TNNetMaxPool.Create( 2 ));
NN.AddLayer( TNNetDropout.Create( 0.25 ) );
NN.AddLayer( TNNetConvolutionReLU.Create( 24, 30, 0, 1 ) );
NN.AddLayer( TNNetMaxPool.Create( 2 ));
NN.AddLayer( TNNetDropout.Create( 0.25 ) );
NN.AddLayer( TNNetConvolutionReLU.Create( 24, 30, 0, 1 ) );
NN.AddLayer( TNNetMaxPool.Create( 2 ));
NN.AddLayer( TNNetDropout.Create( 0.25 ) );
NN.AddLayer( TNNetConvolutionReLU.Create( 24, 10, 0, 1 ) );
NN.AddLayer( TNNetMaxPool.Create( 2 ));
NN.AddLayer( TNNetDropout.Create( 0.25 ) );
NN.AddLayer( TNNetConvolutionReLU.Create( 12, 10, 0, 1 ) );
NN.AddLayer( TNNetMaxPool.Create( 2 ));
NN.AddLayer( TNNetDropout.Create( 0.25 ) );
// this layer is by mike:
NN.AddLayer( TNNetFullConnectLinear.Create(2) );
NN.AddLayer( TNNetSoftMax.Create );
NeuralFit := TNeuralFit.Create;
NeuralFit.FileNameBase := 'ECG_' + FormatDateTime( 'ddmmyy_hhnn', now );
NeuralFit.InitialLearningRate := 0.01; // learning rate of 0.1 does not seem to work properly
NeuralFit.LearningRateDecay := 0.001;
NeuralFit.StaircaseEpochs := 30;
NeuralFit.Inertia := 0.9;
NeuralFit.L2Decay := 0.00001;
NeuralFit.MaxCropSize := 0;
NeuralFit.InferHitFn := ClassCompare;
NeuralFit.LossFn := TNeuralFitHack(NeuralFit).DefaultLossFn;
netSave := TNotifyEvtObj.Create(NeuralFit);
NeuralFit.OnAfterEpoch := netSave.OnAfterEpoch;
EasyOpenCL := TEasyOpenCL.Create();
if EasyOpenCL.GetPlatformCount() > 0 then
begin
WriteLn('Setting platform to: ', EasyOpenCL.PlatformNames[0]);
EasyOpenCL.SetCurrentPlatform(EasyOpenCL.PlatformIds[0]);
if EasyOpenCL.GetDeviceCount() > 0 then
begin
EasyOpenCL.SetCurrentDevice(EasyOpenCL.Devices[0]);
WriteLn('Setting device to: ', EasyOpenCL.DeviceNames[0]);
NeuralFit.EnableOpenCL(EasyOpenCL.PlatformIds[0], EasyOpenCL.Devices[0]);
end
else
begin
WriteLn('No OpenCL capable device has been found for platform ',EasyOpenCL.PlatformNames[0]);
WriteLn('Falling back to CPU.');
end;
end
else
begin
WriteLn('No OpenCL platform has been found. Falling back to CPU.');
end;
NeuralFit.Fit(NN, EDRTrainingVolumes, EDRValidationVolumes, EDRTestVolumes, {batchsize=}256, {epochs=}400);
NeuralFit.Free;
netSave.Free;
NN.SaveToFile('NN.dat');
Input is 10 seconds of ECG aka 1000samples.
It seems that it may happen that fHasOpenCL is true but fShouldOpenCL is false when the base function TNNetConvolutionBase.EnableOpenCL is called. This leads to an unassigned fDotCL object which then raises an exception.
I needed to change TNNetConvolutionBase a bit to get it started (though I don't know if it is correct):
{$IFDEF OpenCL}
procedure TNNetConvolutionBase.EnableOpenCL(DotProductKernel: TDotProductKernel);
begin
inherited EnableOpenCL(DotProductKernel);
// fDotCL is not assigned in case fShouldOpenCL is false
if Assigned(FDotCL)
then
FDotCL.PrepareForCompute(FConcatedWInter, FInputPrepared, FVectorSize)
else
FHasOpenCL := False;
end;
let me know if that fix is ok for you.
Very interesting experiment!
Thank you for the detailed report. Do you refer to this dataset? https://www.kaggle.com/shayanfazeli/heartbeat
Hi,
On Wed, Mar 2, 2022 at 4:08 PM joaopauloschuler @.***> wrote:
Very interesting experiment!
Thank you for the detailed report. Do you refer to this dataset? https://www.kaggle.com/shayanfazeli/heartbeat
Actually no - the aim is to have an Apnea classifier based on the raw ecg. The database can be found here: https://physionet.org/content/apnea-ecg/1.0.0/
kind regards Mike
— Reply to this email directly, view it on GitHub https://github.com/joaopauloschuler/neural-api/issues/85#issuecomment-1057037644, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACX47A4MFEXDP6JGFN4CLSDU5577BANCNFSM5PXWCMPQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID: @.***>
Hello @mikerabat, I tried to reproduce the problem at my end. But, with my own code, I had no problem. I'm wondering if you have your full source code on an open repository for me to test with your own code?
Dear @joaopauloschuler/neural-api @.***>
I couldn't do create one but I managed to create a minimal project that shows the problem (including the changes I made so it doesn't throw an error). Please note that I needed rename the files because they conflicted with other projects I'm working on so I hope that is not a big deal.... Please also note that this is a Delphi 11 project ...
kind regards Mike
On Thu, Mar 10, 2022 at 12:28 AM joaopauloschuler @.***> wrote:
Hello @mikerabat https://github.com/mikerabat, I tried to reproduce the problem at my end. But, with my own code, I had no problem. I'm wondering if you have your full source code on an open repository for me to test with your own code?
— Reply to this email directly, view it on GitHub https://github.com/joaopauloschuler/neural-api/issues/85#issuecomment-1063482367, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACX47A5TRE7XPTOSPC2Y4X3U7EX3JANCNFSM5PXWCMPQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: @.***>
Dear @joaopauloschuler/neural-api @.***>
I couldn't do create one but I managed to create a minimal project that shows the problem (including the changes I made so it doesn't throw an error). Please note that I needed rename the files because they conflicted with other projects I'm working on so I hope that is not a big deal.... Please also note that this is a Delphi 11 project ...
kind regards Mike
On Thu, Mar 10, 2022 at 12:28 AM joaopauloschuler @.***> wrote:
Hello @mikerabat https://github.com/mikerabat, I tried to reproduce the problem at my end. But, with my own code, I had no problem. I'm wondering if you have your full source code on an open repository for me to test with your own code?
— Reply to this email directly, view it on GitHub https://github.com/joaopauloschuler/neural-api/issues/85#issuecomment-1063482367, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACX47A5TRE7XPTOSPC2Y4X3U7EX3JANCNFSM5PXWCMPQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: @.***>
Dear @mikerabat , If you could share please your minimal project that shows the problem so I can reproduce it at my end, this would be greatly appreciated.
Kind regards, JP.
Dear JP,
hm... I attached the project as 7z file to the mail.. seems it was deleted in the proces... try this link here: https://cloud.mrsoft.org/s/qsozetbgPWT7ytb
kind regards Mike
On Thu, Mar 10, 2022 at 8:00 PM joaopauloschuler @.***> wrote:
Dear @mikerabat https://github.com/mikerabat , If you could share please your minimal project that shows the problem so I can reproduce it at my end, this would be greatly appreciated.
Kind regards, JP.
— Reply to this email directly, view it on GitHub https://github.com/joaopauloschuler/neural-api/issues/85#issuecomment-1064392138, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACX47A3HRKRIAQNAJP3SESTU7JBFVANCNFSM5PXWCMPQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: @.***>
Ah and by the way... That code includes the fixes I did (hope it wasn't too bad...) so.. you would need to use the original sources eventually..
kind regards Mike
On Thu, Mar 10, 2022 at 8:13 PM Michael Rabatscher @.***> wrote:
Dear JP,
hm... I attached the project as 7z file to the mail.. seems it was deleted in the proces... try this link here: https://cloud.mrsoft.org/s/qsozetbgPWT7ytb
kind regards Mike
On Thu, Mar 10, 2022 at 8:00 PM joaopauloschuler @.***> wrote:
Dear @mikerabat https://github.com/mikerabat , If you could share please your minimal project that shows the problem so I can reproduce it at my end, this would be greatly appreciated.
Kind regards, JP.
— Reply to this email directly, view it on GitHub https://github.com/joaopauloschuler/neural-api/issues/85#issuecomment-1064392138, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACX47A3HRKRIAQNAJP3SESTU7JBFVANCNFSM5PXWCMPQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: @.***>
Hi!
I just wanted to check back with you and ask if you were able to download the project. If you could not reproduce my problem could you let me know what I did wrong?
kind regards Mike
On Thu, Mar 10, 2022 at 8:17 PM Michael Rabatscher @.***> wrote:
Ah and by the way... That code includes the fixes I did (hope it wasn't too bad...) so.. you would need to use the original sources eventually..
kind regards Mike
On Thu, Mar 10, 2022 at 8:13 PM Michael Rabatscher @.***> wrote:
Dear JP,
hm... I attached the project as 7z file to the mail.. seems it was deleted in the proces... try this link here: https://cloud.mrsoft.org/s/qsozetbgPWT7ytb
kind regards Mike
On Thu, Mar 10, 2022 at 8:00 PM joaopauloschuler < @.***> wrote:
Dear @mikerabat https://github.com/mikerabat , If you could share please your minimal project that shows the problem so I can reproduce it at my end, this would be greatly appreciated.
Kind regards, JP.
— Reply to this email directly, view it on GitHub https://github.com/joaopauloschuler/neural-api/issues/85#issuecomment-1064392138, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACX47A3HRKRIAQNAJP3SESTU7JBFVANCNFSM5PXWCMPQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: @.***>