magenta
magenta copied to clipboard
magenta
NoExtractedExamplesError: Not all <class 'note_seq.protobuf.music_pb2.NoteSequence'> objects can be encoded using MusicVAE
Hello,
I am using the pretrained MusicVAE to decode an arbitrary latent space z from numpy.random.randn(batch, latent_dims). After decoding, I re-encode the NoteSequence to extract z, mu, and sigma. The error arises from the fact that not all NoteSequence objects that are generated by decoding an arbitrary latent space z are valid. Why does the NoExtractedExamplesError arise for some NoteSequence objects but not others? Is this behavior intended? Here is the colab notebook that demonstrates this problem. Please note, if you re-run the notebook, the number of iterations before the error occurs will change. To see my ongoing troubleshooting of this problem, please refer to this expanded colab notebook
Defining the trained model:
batch_size = 4
mel_2bar_config = configs.CONFIG_MAP['cat-mel_2bar_big']
path = 'gs://download.magenta.tensorflow.org/models/music_vae/colab2/checkpoints/mel_2bar_big.ckpt'
mel_2bar = TrainedModel(mel_2bar_config, batch_size=batch_size, checkpoint_dir_or_path=path)
A loop that shows the error:
#@title Loop Decode-Encode Until Error
# Counter for how many iterations of this loop can
# occur before the `NoExamplesExtractedError` occurs.
cnt = 0
## Dimensions for ranodm normal z to be decoded
ldims = mel_2bar._z_input.shape[1]
batch_size = 4 # This is still 4 from the earlier assignment, I put this here as a reminder
# Infinite loop for error testing
while True:
# Generate random normal data
z = np.random.randn(batch_size, ldims).astype(np.float32)
# Decode the random normal data with arbitary `length` arg
audios = mel_2bar.decode(z, length=32)
# Attempt to re-encode the `audios` variable
# Note, `audios` is a <class 'list'> of NoteSequence objs
try:
# Returns from TrainedModel `encode` method
z, mu, log_var = mel_2bar.encode(audios)
# Increment counter if no error occurs
cnt += 1
except:
print(f'{cnt} Iterations Before Error')
raise
>>> 62 Iterations Before Error
---------------------------------------------------------------------------
NoExtractedExamplesError Traceback (most recent call last)
<ipython-input-35-155a3af8d166> in <module>()
21 try:
22 # Returns from TrainModel encoder
---> 23 z, mu, log_var = mel_2bar.encode(audios)
24
25 # Increment counter if no error occurs
/usr/local/lib/python3.7/dist-packages/magenta/models/music_vae/trained_model.py in encode(self, note_sequences, assert_same_length)
215 if not extracted_tensors.inputs:
216 raise NoExtractedExamplesError(
--> 217 'No examples extracted from NoteSequence: %s' % note_sequence)
218 if len(extracted_tensors.inputs) > 1:
219 raise MultipleExtractedExamplesError(
NoExtractedExamplesError: No examples extracted from NoteSequence: ticks_per_quarter: 220
tempos {
qpm: 120.0
}
notes {
pitch: 39
velocity: 80
start_time: 0.375
end_time: 0.5
}
notes {
pitch: 37
velocity: 80
start_time: 0.5
end_time: 0.625
}
notes {
pitch: 41
velocity: 80
start_time: 2.875
end_time: 3.25
}
notes {
pitch: 44
velocity: 80
start_time: 3.25
end_time: 3.75
}
total_time: 3.75
Best,
Jared
I ran into the same issue. I am trying to use the GrooVAE model to generate predictions for predetermined note sequences so that I can compare its performance against a different model's performance. (I am using the Tap2Drum model, and the drumify function as defined in the GrooVAE colab notebook). For example, this sequence always raises the NoExtractedExamplesError:
time_signatures {
numerator: 4
denominator: 4
}
tempos {
qpm: 130.00013000013
}
notes {
pitch: 42
velocity: 10
start_time: 0.15769214999999998
end_time: 0.21538439999999986
instrument: 9
is_drum: true
}
notes {
pitch: 42
velocity: 48
start_time: 0.2721151125
end_time: 0.32980736249999987
instrument: 9
is_drum: true
}
notes {
pitch: 42
velocity: 79
start_time: 0.2999997
end_time: 0.35769194999999987
instrument: 9
is_drum: true
}
notes {
pitch: 42
velocity: 14
start_time: 0.45961492499999995
end_time: 0.5173071749999998
instrument: 9
is_drum: true
}
notes {
pitch: 42
velocity: 74
start_time: 0.6173070749999999
end_time: 0.6749993249999998
instrument: 9
is_drum: true
}
notes {
pitch: 42
velocity: 14
start_time: 0.7798069124999999
end_time: 0.8374991624999998
instrument: 9
is_drum: true
}
notes {
pitch: 42
velocity: 103
start_time: 0.9346144499999999
end_time: 0.9923066999999998
instrument: 9
is_drum: true
}
notes {
pitch: 42
velocity: 62
start_time: 1.0894219875
end_time: 1.1471142374999999
instrument: 9
is_drum: true
}
notes {
pitch: 42
velocity: 56
start_time: 1.236537225
end_time: 1.2942294749999999
instrument: 9
is_drum: true
}
notes {
pitch: 42
velocity: 127
start_time: 1.3913447625
end_time: 1.4490370124999998
instrument: 9
is_drum: true
}
notes {
pitch: 42
velocity: 93
start_time: 1.5461523
end_time: 1.6038445499999998
instrument: 9
is_drum: true
}
notes {
pitch: 42
velocity: 26
start_time: 1.6903829249999998
end_time: 1.7480751749999996
instrument: 9
is_drum: true
}
total_time: 1.7480751749999996
thank you in advance! 😄
I had the same problem when trying to encode melodies previously generated by the model's (MusicVAE) sample function.
The following example illustrates how seemingly arbitrary the difference is between a sequence provoking the error and one that doesn't:
seq = music_pb2.NoteSequence()
seq.tempos.add(qpm=120)
seq.ticks_per_quarter = 220
seq.total_time = 5.0
seq.notes.add(pitch=51, start_time=3.0, end_time=5.0, velocity=80)
_, enc, _ = mel_2bar.encode([seq]) # this works
seq2 = music_pb2.NoteSequence()
seq2.tempos.add(qpm=120)
seq2.ticks_per_quarter=220
seq2.total_time=4.0
seq2.notes.add(pitch=51, start_time=2.0, end_time=4.0, velocity=80)
_, enc2, _ = mel_2bar.encode([seq2]) # this yields NoExtractedExamplesError
It would be really great if this could be fixed or at least explained. Thanks in advance!
