midilab
Feature request: Register multiple onStepCallback subscribers, each with independent shuffle templates
Right now, a single callback for getting shuffle-enabled 16th steps is handled via onStepCallback. This is well and good. However, when using uClock in a multi-track system, you may want to disengage a track from being shuffled so that it's played on the straight beat instead.
Also, for doing microtiming adjustments on 16th steps for each individual track, one option could be to just not use the onStepCallback and instead process everything in the 96ppqn callback function, and track the 16th steps inside there. But I think this would be more intense on the CPU having to do more work inside the 96ppqn callback, which also might make the clock a bit less tight.
So, what if instead we could generate/register multiple 16th step handlers, for any number of tracks? And each handler could have its own shuffle settings. This way, you could have the main clock processing freed up from having to do all of this extra stuff, and you could have separate track-style instances of 16th step handlers, each with their own shuffle settings to allow for microtiming adjustments.
Seems like if you also want to enable live pattern recording (tapping in steps live while a pattern is playing) then that would best be handled in the 96ppqn callback so you could determine where the nearest step is (plus/minus any microtiming adjustment) with some quantization algorithm.
Let me know if this makes sense.
Hey @grayxr ,
Yes it makes total sense, i will review your PR and get back asap. Thanks for the development work on your side!
Are you running this multi shuffle schema on your sequencer? any videos for me to take a look?
Hey @midilab - yes here is a demo video (sorry for the rotated video)
By applying the step microtiming on steps, the hi-hat track gets its own shuffle template. I made it also so that it merges in any shuffle from the pattern shuffle template as well, but only to steps which are not overridden with their own microtiming offsets.
This looks nice, 1 have a 2 track classic arpeggiator that would be good to add some variation to. Are there any examples of basic shuffle operation?
Cheers, Paul
This does indeed sound very cool. I'm going to have to do some serious thinking about how to restructure my project in order to take advantage of this :D
Hey all, good to see this development going on! I'll try to test the Add track-based callback with shuffle per track PR over the next weekend.
I was just working on adding grooves to my clocking device. I have a switch that toggles between quarter notes (4, 8 ,16) and triplet (3, 6, 12) speeds. Adding shuffle with the onStepCallback works fine for the quarter note speeds, but is harder to work with when manipulating triplets.
It would be great to have a version of the onStepCallback based on 24 steps instead of 16. This way we can easily manipulate triplets. What do you think?
I've finally gotten around to giving this a try in my project, and think I've got the basics working (only one track atm but planning to make that configurable, as well as making an editor for the shuffle template and possibly also smooth modulation of shuffle).
I used the more recent track-based2 branch from @grayxr 's fork but look what was different at all -- anything I should be aware of @grayxr ?
Will report back when I've got further into it.
So, I've been playing with this some more, have it so I can assign different shuffle tracks to my drum tracks, and have an editor inside my app to tweak the pattern on the fly...
I'm not sure if I'm using the shuffle pattern template stuff correctly (apologies if this is documented somewhere and I've missed it?).
Is the idea that each step of the shuffle pattern corresponds to one step of the bar, and that the step value is the offset for the tick? e.g., -3 means the step triggers 3 ticks early, while +3 means the step triggers 3 ticks late?
If that's how its meant to work then I'm not sure that I'm seeing 'late' shuffles working correctly.
Also at times I seem to be able to glitch the entire shuffle pattern into playing at the wrong times -- I think maybe this happens when two shuffled steps overlap or are maybe too close to each other?
I've tried with both @grayxr 's track-based2 and tracked-based1 branches (first merging the more recent uClock main version to bring them up to the latest version and then applying my 'continue' patch from #49 ) and couldn't seem to get either working quite right. I've put what I've got on my repos https://github.com/doctea/uClock/tree/track-based2-merge and https://github.com/doctea/uClock/tree/tracked-based1-merge.
@grayxr and @midilab , could you let me know if there is anything I'm misunderstanding about how the shuffle patterns are meant to work? Which branch should I ideally be using, @grayxr ?
@doctea
The shuffle works based on output ppqn setup, and it is between each step that things happen:
Here it is the calculus for min and max values for each output_ppqn Negative shuffle: -(output_ppqn/4)-1 ticks Positive shuffle: (output_ppqn/4)-1 ticks
If you are using PPQN_96 as output_ppqn (the default), then you have a range of -23 to 23 to use as shuffle. Going outside this range will cause overflow on shuffle counting. PPQN_96 shuflle range: Min: -23 Max: 23
Another example: If you are working with 960 PPQN, which is the higher resolution of uClock, then the max and min should be: PPQN_960 shuffle range: Min: -239 Max: 239
The higher the output PPQN resolution, the more possibilities you'll have to fine-tune your shuffles. Another important part of shuffles is that it raises the last note of current one being shuffled, so at each callback you have the length difference so you can correct with the step data length to sum and create the new groove length that preceds a shuffled note
A MPC60 or TR909 groove signature for PPQN_96 setup:
// MPC60 groove signatures?
// Same as 909
uint8_t current_shuffle = 0;
uint8_t template_size = 2;
int8_t shuffle_50[2] = {0, 0};
int8_t shuffle_54[2] = {0, 2};
int8_t shuffle_58[2] = {0, 4};
int8_t shuffle_62[2] = {0, 6};
int8_t shuffle_66[2] = {0, 8};
int8_t shuffle_71[2] = {0, 10};
int8_t shuffle_75[2] = {0, 12};
String shuffle_name[7] = {"50%", "54%", "58%", "62%", "66%", "71%", "75%"};
int8_t* shuffle_templates[7] = {shuffle_50, shuffle_54, shuffle_58, shuffle_62, shuffle_66, shuffle_71, shuffle_75};
uClock.setShuffleTemplate(shuffle_templates[current_shuffle], template_size);
a better view of shuffle with length incresase before shuffled note:
By the way @grayxr , im planing to review the PR this next week probrably to add it to the new release.
Thanks @midilab ... I'll have to test again. I could definitely hear a shuffle effect with +3 as the shuffle pattern, a larger effect than I'd expect if its only an 8th of a step different. But could not hear a difference with -3. I'll try connecting a scope to see if I can get a visual indicator on what is happening.
Regarding longer note lengths and your image, to get my drum sequencer working with the shuffle I have to keep the note length short and send the 'note off' if the note duration has exceeded the expected length -- its not possible for me to join the two notes together like that as if there is no gap between the "note off" and "note on" then the CV gate output will never go off. If that makes sense! So that is a way that modular requirements differ from midi outputs. Fortunately as they're all percussive hits anyway the note duration does not matter so much.
I'll have to test again. I could definitely hear a shuffle effect with +3 as the shuffle pattern, a larger effect than I'd expect if its only an 8th of a step different. But could not hear a difference with -3. I'll try connecting a scope to see if I can get a visual indicator on what is happening.
Check what is your PPQN output resolution and the max and min range for shuffle tick, maybe 3 its almost nothing to perceive if it is set the output PPQN to 96... try the values and template i've show you as example (MPC60/TR-909 swing signature tempalte).
Regarding longer note lengths and your image, to get my drum sequencer working with the shuffle I have to keep the note length short and send the 'note off' if the note duration has exceeded the expected length -- its not possible for me to join the two notes together like that as if there is no gap between the "note off" and "note on" then the CV gate output will never go off. If that makes sense! So that is a way that modular requirements differ from midi outputs. Fortunately as they're all percussive hits anyway the note duration does not matter so much.
For that you can use uClock.getShuffleLength(). i dont know how it works on multi track shuffle because i didnt test or validade this pull request code yet.
But on main branch uClock.getShuffleLength() returns the difference of length genrated for current step when it is +tick or -tick so you have fine control of how to short or increase the lentgh for the groove as show in the image. SO you take the getShuffleLength() return and sum to your current note length to avoid exceed the length adn create the perfect swing feel specially for melodic and basses tracks, not much for drums since its a trigger most of the times not related with length of on/off gate.
A example of use getShuffleLength() on aciduino:
void Aciduino::onStepCallback(uint32_t step)
{
// sequencer tick
aciduino.seq.onStep(step, uClock.getShuffleLength());
}
That way each step call has the difference length created by the shuffle schema to sum with current step lentgh.
You simply call current_stepLength += uClock.getShuffleLength();, and it adapts the length for the neighboring step or adjusts the shuffle size to avoid note overlap. This also helps create the swing effect or increases/decreases the length of notes.
Thanks as always for the help @midilab :).
I've fixed some bugs on my side of the code, and with the increased shuffle range I now have both positive and negative step values appearing to work correctly -- with the caveats below...
But I still run into these problems.
These could be only problems with my project and how I am using uClock+shuffles, but documenting them here in case they can be replicated outside of my project, or in case the description of the problems seems familiar.
(FIXED!) 1. the second shuffle track (as in, track_shuffles[1]) always seems to play 'wrong' -- it is as if the pattern is 15 steps long instead of 16 so it does not loop correctly, or is being stepped too often. All other track_shuffles seem to work correctly without this problem. Maybe this is a problem in my code, but it seems very odd and I can't track it down.
My serial logging indicates that the wrong step numbers are being passed into the callback, eg:-
callback is on_step_shuffled(int track, int step)
at tick 77, received on_step_shuffled(1, 13) callback for shuffled track 1 at tick 77, received on_step_shuffled(1, 14) callback for shuffled track 1 [...] at tick 7980, received on_step_shuffled(0, 1330) callback for shuffled track 0 at tick 7980, received on_step_shuffled(1, 1519) callback for shuffled track 1 at tick 7980, received on_step_shuffled(2, 1330) callback for shuffled track 2 at tick 7985, received on_step_shuffled(1, 1520) callback for shuffled track 1 at tick 7986, received on_step_shuffled(0, 1331) callback for shuffled track 0 [...] at tick 7992, received on_step_shuffled(2, 1332) callback for shuffled track 2 at tick 7992, received on_step_shuffled(1, 1521) callback for shuffled track 1 at tick 7998, received on_step_shuffled(0, 1333) callback for shuffled track 0_
FIXED!! Was a problem in with code (accidentally initialised track 1 with a shuffle pattern using code below-- although why that would lead to these symptoms, I'm not sure... so including here in case it becomes useful later if another bug is exposed)
int8_t shuff[] = {
(int8_t)-2, (int8_t)2, (int8_t)3, (int8_t)3, (int8_t)3, (int8_t)-3, (int8_t)-2, (int8_t)3,
(int8_t)1, (int8_t)3, (int8_t)3, (int8_t)1, (int8_t)-2, (int8_t)-2, (int8_t)-1, (int8_t)0
};
uClock.setTrackShuffleTemplate(1, shuff, 16);
uClock.setTrackShuffle(1, true);
2. shuffle track slips out of sync when negative shuffle is used on the first note of the sequence. This will require a bit more background to describe..
My sequencer has a feature where the user can 'queue a restart' that will happen on the first step of the next bar. When the restart is queued, once the end of the bar is reached the clock is stopped, counters reset, and clock is started, so that the song plays from the beginning with ticks = 0. (This is used in my project to ensure that connected devices have their clocks synced by sending MIDI start+stop messages to all connected devices, or just to restart the song).
Conditions to replicate issue:-
- the first step of the shuffle pattern is set to a negative number (eg -12).
- a note is placed on the first step of the sequence.
- restart is queued for the next bar.
Behaviour:-
- as the playback reaches the final note of the bar, the note from the next bar that is shuffled to be early plays correctly.
- the clock then resets to 0 and the first step plays.
- the first note of the bar is incorrectly played again, even though it should not be.
- the shuffle pattern used then goes "out of sync" with the other shuffle patterns/rigid clock, such that any subsequent events on this track play later than they should do (possibly by a whole step later?)
- queueing another restart seems to bring the shuffle pattern back into sync.
The investigation continues...
thank you @doctea for all the tests and information:
have you also test using main branch? since the multitrack implementation is based on the main branch i would like to know if the bug reported happens on multitrack version of shuffle and main branch too(where there is no multitrack shuffle, only global shuffle).
have you also test using main branch? since the multitrack implementation is based on the main branch i would like to know if the bug reported happens on multitrack version of shuffle and main branch too(where there is no multitrack shuffle, only global shuffle).
Just adjusted my code to work with the main branch in order to test this.
While doing so I found a code snippet that was causing my "problem 1" (facepalm) -- updated comment above with info on this since although it appears to be working now, I don't see how that problem would be caused in this way... but think it can be disregarded for now unless a similar problem appears again.
For problem 2, this still seems to exist with the main branch:-
- 'double trigger' of first sequencer note if clock is restarted after an 'early' note played at end of previous bar.
- more difficult to tell this because when everything is shuffled there isn't a more rigid clock to compare against, but comparing to MIDI events from other devices it appears that the shuffle pattern does then end up playing late again after this glitch in the same way.
Hey Guys,
sorry, but an absolute noob here... Untill now, my project was working all fine with "one uclock" that ticks my 8 Tracks. Each Track has its own SequenceLength and clockDivision. (As a noob i am, i decided to make each Track´s "Clip/Sequence" 96Miditicks (aka one Bar or 16x 16th steps, in the 24ppqn resolution) long. Since my control interface is a matrix of 16(steps) x 8(Tracks), it was very fine to me,
but then!... @doctea told me about the multitrack shuffle function! (He´s my very patient "mentor" :) )
Now i have many question?? :D
- As mentioned, the shuffle only works for the onStepCallback function, how can i play/work-with shorter notes, or notes that dont start/end at an exact 16th step. Beside the shuffle?
- Can there somewhere be a simple example for the multitrack shuffle. How to set them up, interact with them.
- As i´m storing the whole clip information (Notes and no Notes for all 96 ticks), can i store then in stepevolution 16x 16th steps (with info about startTick and notelength). Or would i still have to store all ticks? ( 3) As mentioned, would it be possible to have a shuffle for the onSync24Callback? I guess that would be less CPU intense then 96ppqn).,
Sorry for my questions, as the "Multishuffle" is completly new to me, thanks to doctea ;P
Hi @steven-law ,
As mentioned, the shuffle only works for the onStepCallback function, how can i play/work-with shorter notes, or notes that dont start/end at an exact 16th step. Beside the shuffle?
When using shuffle or multi shuffle the callback out of exact 16th is transparent from coder perspective, if it is time to be called it will, nothing else neeeds to be done, so same code without shuffle is compatible.
Can there somewhere be a simple example for the multitrack shuffle. How to set them up, interact with them.
I have implement final version of multi shuffle at develop branch, i will merge into main and update the documentation along with examples for shuffle.
As i´m storing the whole clip information (Notes and no Notes for all 96 ticks), can i store then in stepevolution 16x 16th steps (with info about startTick and notelength). Or would i still have to store all ticks? ( 3) As mentioned, would it be possible to have a shuffle for the onSync24Callback? I guess that would be less CPU intense then 96ppqn).,
You can do both, all the sequencers i write with uClock uses startTick and notelength for smaller memory usage. For notelength implementation i make use of a noteOn stack to track where the note needs to be shot off, you can check a example on uClock README.md for the acidsequencer.
Be aware that the develop branch has some breakchanges for API usage, use the README.md of develop branch as a guide.