Running a trained LSTM network is VERY slow (around 15ms for each result)

[Sharcoux]

What is wrong?

Running a trained LSTM network on each input is VERY slow.

Where does it happen?

Just running node on my machine. See the code below.

How do we replicate the issue?

Here is a code that reproduce the issue

const brain = require('brain.js')

const net = new brain.recurrent.LSTM()

const trainingOptions = {
  // Defaults values --> expected validation
  iterations: 200, // the maximum times to iterate the training data --> number greater than 0
  errorThresh: 0.005, // the acceptable error percentage from training data --> number between 0 and 1
  log: true, // true to use console.log, when a function is supplied it is used --> Either true or a function
  logPeriod: 100, // iterations between logging out --> number greater than 0
  learningRate: 0.3, // scales with delta to effect training rate --> number between 0 and 1
  // Momentum is missing in the type declaration. See : https://github.com/BrainJS/brain.js/issues/806
  // momentum: 0.1, // scales with next layer's change value --> number between 0 and 1
  callbackPeriod: 10, // the number of iterations through the training data between callback calls --> number greater than 0
  timeout: Infinity // the max number of milliseconds to train for --> number greater than 0
}

for (let loop = 0; loop < 15; loop++) {
  net.train(
    [
      { input: 'I feel great about the world!', output: 'happy' },
      { input: 'The world is a terrible place!', output: 'sad' }
    ],
    trainingOptions
  )
  let oldDate = Date.now()
  for (let i = 1; i <= 3000; i++) {
    net.run('I feel great about the world!')
    if (i % 1000 === 0) {
      console.log('Each run took about', (Date.now() - oldDate) / 1000, 'ms to complete')
      oldDate = Date.now()
    }
  }
}

Here is a sample output:

Sometimes, with some training, you get a better result. Check out the last 2 runs

iterations: 0, training error: 5919581.560087977
iterations: 100, training error: 0.10638371212248059
Each run took about 12.506 ms to complete
Each run took about 13.651 ms to complete
Each run took about 14.936 ms to complete
iterations: 0, training error: 0.08807550982775042
iterations: 100, training error: 0.07916010305806237
Each run took about 13.474 ms to complete
Each run took about 13.946 ms to complete
Each run took about 13.448 ms to complete
iterations: 0, training error: 0.07403161373261806
iterations: 100, training error: 0.07549036517869813
Each run took about 13.23 ms to complete
Each run took about 13.455 ms to complete
Each run took about 13.073 ms to complete
iterations: 0, training error: 0.07460482166846667
iterations: 100, training error: 0.07202732715101895
Each run took about 12.981 ms to complete
Each run took about 13.769 ms to complete
Each run took about 16.111 ms to complete
iterations: 0, training error: 0.0656084483947608
iterations: 100, training error: 0.06511340031555182
Each run took about 14.251 ms to complete
Each run took about 14.39 ms to complete
Each run took about 15.322 ms to complete
iterations: 0, training error: 0.06458603058643311
iterations: 100, training error: 0.06371472513073857
Each run took about 0.384 ms to complete
Each run took about 0.462 ms to complete
Each run took about 0.437 ms to complete
iterations: 0, training error: 0.17024263005272064
iterations: 100, training error: 0.10909141758992613
Each run took about 2.994 ms to complete
Each run took about 3.489 ms to complete
Each run took about 3.322 ms to complete

Expected behavior (i.e. solution)

From my understanding, trained networks should be quite fast at doing what they were trained to do. Here, the training seems faster than running a case.

Version information

Nodejs: 14.19.3

Browser: N/A

Brain.js: 2.0.0-beta.15

How important is this (1-5)?

5

Other Comments

I see that many people complain about training being slow, but I think that this is expected. However, I have never heard about running a trained network being slow. It goes against my understanding of neural nets.

[Meleeman01]

i've also had this same problem and i was going to try and compensate by attempting to spawn child processes. as the training the set takes place on 1 thread of a cpu. adding gpu support for lstm would be cool. may be something else in the library that could be optimized.

[imkane]

Will GPU support ever be available for LSTMTimeStep? :thinking:

[roysG]

Any update on this?