neural-api
neural-api copied to clipboard
Some data parts are not stored using "SaveStructureToString"
E.g. the base class TNNetAddNoiseBase does not store the "Enabled" property in that structure. That has the following impact: Assume you have a TNNetDropout layer in your net. Based on that definition if the enabled flag is set on each learning iteration it randomly changes the which activations are used in a learning step. Now assume you finished learning and save the net to a file - later loading it and use it to classify an unseen example.
Based on randomness the outcome is unpredictable and changes on each iteration! (I found that when I tried to analyze some ECG recordings for sleeping apneas and the output probabilites changed each time)
My workaround is to manually go through each layer after loading or cloning (for threads) and set the flag "Enabled" to false. Would be nice though to have either a general switch or have a function to remove TNNetAddNoiseBase layers at all for production runs that rely on same input = same output principle ;) or have the structure enhanced by storing additional properties...
Dear @mikerabat , I'll look into it and FUP.
Many thanks for reporting the bug. The Enabled property is controlled by TNNet.EnableDropouts and TNNetDataParallelism.EnableDropouts so it can be enabled for training and disabled for testing. You are correct at reporting the bug: it should be false by default and activated only at training time.
Dear @mikerabat , I believe that this bug has been fixed. If you believe otherwise, please feel free reopening it with additional details.