Roland MKS-70 and MKS-50 implementation

[christofmuc]

This is to capture my knowledge on those two old Roland rack synths. I had an MKS-50 for a brief time and during that implemented it fully for the predecessor of the KnobKraft Orm, and I could reactivate it.

The MKS-70 is a much simpler version than the MKS-80, it has the same concept of APR/IPR messages, but ditches the memory-saving BLK bulk dump format which required bit-fiddling to get BLK dumps into APR format. So this is not necessary. Sadly, the MKS-70 also does not allow for handshake dumps, so you could only intitiate the dump from the device in "receive manual dump" mode.

I think given both have the Roland patch/tone split, we need to first complete the #58 feature before it makes much sense.

[christofmuc]

I revivied the MKS-50 implementation and put it into release 1.13.4. As I non longer own an MKS-50, I'll rely on the community to report bugs!

[markusschloesser]

the free PG8X is, to the best of my knowledge, a great replica of the JX8P, which is basically the MKS-70 in keyboard format. The plugin also reads JX8P sysex, so imho could be used to test a possible implementation.

[jmfreeland]

I, very frustratingly, purchased the single instrument version of Midiquest solely to work with my MKS-70 only to discover it can't import sysex banks!

I did, however, find that the seemingly free version of Midiquest Librarian on the iPad can and is also able to send the patches to my MKS-70 through my PC.

You can't edit or reorder them, but you can import a sysex bank and send the patches over, which might be enough.

[markusschloesser]

my MKS-70 will be arriving next week, so I will tackle an adaptation for this. Now mine will have the Vecoven 4 firmware and the new PWM board, and iirc Fred changed the sysex implementation in order to work properly with the synth, so my adaptation will not work with all MKS-70, only with the Vecoven ones (probably). I also learned a bit of python in the meantime plus ChatGPT knows about knobkraft and thinks it can write an adaptation so lets see how that goes :-)

FYI: that's the GPT4 chat about kk: Are you aware of the software knobkraft orm and can you write a python based adaptation for the Roland mks 70 if I provide the syses specification? Yes, I'm aware of the KnobKraft Orm software, which is a sysex librarian and patch editor for various synthesizers. While I cannot create a complete Python adaptation for the Roland MKS-70 within this response, I can give you a high-level overview of the process and provide some Python code snippets to get you started.

First, you'll need to understand the SysEx specification for the Roland MKS-70. Here's an example of the basic structure of a SysEx message for the MKS-70:

F0 - SysEx Start 41 - Roland Manufacturer ID 10 - Device ID (default is 10) 16 - Model ID (MKS-70) 11 - Command ID (Data Set) Address MSB - Most Significant Byte of the Address Address ISB - Intermediate Significant Byte of the Address Address LSB - Least Significant Byte of the Address Data - The actual data (patch parameters, etc.) Checksum - XOR of the Address and Data bytes F7 - SysEx End

To begin, you'll want to use a MIDI library for Python, like mido. Install it using pip:

pip install mido Next, create a function to send a SysEx message:

python

import mido

def send_sysex(device_id, model_id, command_id, address, data):
    msg = mido.Message('sysex', data=[0x41, device_id, model_id, command_id] + address + data)
    checksum = sum(address + data) & 0x7F
    msg.data.append(checksum)
    msg.data.append(0xF7)
    return msg

Now, you can create a function to request a patch from the MKS-70:

python

def request_patch(device_id, model_id, patch_number):
    address = [0x03, (patch_number >> 7) & 0x01, patch_number & 0x7F]
    return send_sysex(device_id, model_id, 0x11, address, []) 

And another function to send a patch to the MKS-70:

python

def send_patch(device_id, model_id, patch_number, patch_data):
    address = [0x03, (patch_number >> 7) & 0x01, patch_number & 0x7F]
    return send_sysex(device_id, model_id, 0x12, address, patch_data)

With these functions, you can now request and send patches to and from the MKS-70. You'll need to create a connection to your MIDI device and handle incoming MIDI messages. To do this, you can use the mido library's input/output ports.

This is just a starting point, and you'll need to implement other features like handling responses, editing patch parameters, and creating a user interface if you wish to make a complete Python-based adaptation for the Roland MKS-70 similar to KnobKraft Orm.

me: This is good, but I don't want something similar to knobkraft, I want to use the mks70 with knobkraft

Apologies for the misunderstanding. If you want to use the MKS-70 with KnobKraft Orm, you'll need to create a new synthesizer adaptation module for the MKS-70. KnobKraft Orm is an open-source project, and you can find its source code on GitHub: https://github.com/christofmuc/KnobKraft-orm

To create an MKS-70 module, follow these steps:

Clone the KnobKraft Orm repository: bash Copy code git clone https://github.com/christofmuc/KnobKraft-orm.git Navigate to the src/learn directory, where you will find existing adaptation modules for various synthesizers.

Create a new Python file for your MKS-70 module (e.g., roland_mks70.py).

In the new file, create a Python class that inherits from the Synth class defined in learn/synth.py. This class will implement the required methods for the MKS-70, such as:

load_sysex_dump: Parse a SysEx dump and return a list of patches. name_from_patch: Extract the patch name from a patch. friendly_name: Return a human-readable name for the synthesizer. supports_bulk_dump: Return True if the synthesizer supports SysEx bulk dumps. create_bulk_dump_request: Create a SysEx message to request a bulk dump. Implement the required methods for the MKS-70 module. You can use the code snippets I provided earlier to help you with the SysEx message structure and the mido library for MIDI handling.

Register your MKS-70 module in the init.py file in the src/learn directory by adding an import statement for your new module and appending an instance of your class to the SYNTHS list.

After implementing the required methods and registering your module, KnobKraft Orm should recognize the MKS-70 and allow you to work with it using the software. For reference, you can look at existing adaptation modules in the src/learn directory to understand how they are implemented.```

[markusschloesser]

this is sooo fucking awesome 😲😁:

whats the sysex for createDeviceDetectMessage?

To create a device detect message using SysEx, you typically send a SysEx message requesting a specific response from the synthesizer. In the case of the Roland MKS-70, you can use the "Request Identity" SysEx message. The synthesizer should then respond with an "Identity Reply" message that contains its model information.

The SysEx message format for the "Request Identity" message is:

F0 7E <Device_ID> 06 01 F7 Where:

F0: SysEx Start 7E: Universal Non-Real Time message (for device inquiry) <Device_ID>: The target device ID (use 7F for all devices) 06: Sub-ID #1 (General Information) 01: Sub-ID #2 (Identity Request) F7: SysEx End

For the MKS-70, you can use the following message to request its identity: F0 7E 7F 06 01 F7

Upon receiving this message, the MKS-70 should respond with an "Identity Reply" message in this format: F0 7E <Device_ID> 06 02 <Manufacturer_ID> <Device_Family_Code> <Device_Family_Member_Code> <Software_Version> F7 For the Roland MKS-70, the Manufacturer ID is 41. You can use this information in your createDeviceDetectMessage function to generate the SysEx message to request the identity of the MKS-70.

[markusschloesser]

me: what do i have to set the channelIfValidDeviceResponse so that knobkraft knows on which midi channel responds to the createDeviceDetectMessage?

GPT: To set the channelIfValidDeviceResponse in the KnobKraft Orm synthesizer adaptation, you should implement a method in your synthesizer class to handle the identity reply message and determine the MIDI channel from the response.

In the case of the Roland MKS-70, the synthesizer does not include the MIDI channel information in its identity reply message. However, you can infer the MIDI channel by sending a program change message on each channel and listening for the corresponding response from the MKS-70.

Here's a high-level overview of how you can implement this in your synthesizer class:

Send a program change message to the MKS-70 on each MIDI channel (0 to 15). Listen for a response from the MKS-70. If you receive a valid response, infer the MIDI channel from the response. To implement this method in your synthesizer class, you can define a method like detect_midi_channel, which takes the incoming message as a parameter:

python def detect_midi_channel(self, message): if message.type == 'program_change': return message.channel return None Then, in your synthesizer class's implementation of the channelIfValidDeviceResponse method, call this new detect_midi_channel method:

python def channelIfValidDeviceResponse(self, message): channel = self.detect_midi_channel(message) if channel is not None: return channel return -1 Now, KnobKraft Orm should be able to detect the MIDI channel that the MKS-70 responds on. Note that this implementation assumes that the MKS-70 will respond to program change messages. If the MKS-70 does not respond to these messages, you may need to use a different approach to infer the MIDI channel.

@christofmuc is this doable? UPDATE: sorry, just realized that it is exactly what needsChannelSpecificDetection is for 🙄 But I am not sure how to than get the channel out of it in channelIfValidDeviceResponse ?

[Andy2No]

Is it quoting from the Adaptation Programming Guide?

It's quite impressive that it can find the right part, but I'm guessing that's what it's doing - with some changes to put in MKS-70, instead of "synth", or similar.

[markusschloesser]

Is it quoting from the Adaptation Programming Guide?

It's quite impressive that it can find the right part, but I'm guessing that's what it's doing - with some changes to put in MKS-70, instead of "synth", or similar.

indeed very impressive. but it's also applying that logic in cross reference to the mks70 sysex spec. For me, that is more impressive :-)

[markusschloesser]

ok, me and chatgpt implemented quite a few functions for the mks70, but i cannot test them, yet. Knobkraft loads the adaptation but if someone wants to check if it works in the meantime? @jmfreeland ? Roland_MKS-70.zip

[jmfreeland]

Happy to give this a try as soon as I can!

[markusschloesser]

Great Little warning though, while I have been playing /tinkering with python for 3 years, I never properly learned and can basically only do ableton midi remote scripts. And I'm good at learning from others. So as the existing knobkraft adaptation codebase is a lot smaller than the remote script one, I had less learning materials. Also ChatGPT kept confusing adaptations with developing knobkraft itself (I think). So make a backup before you start 😉 Hopefully I will be able to test with my own mks70 in a week.

[jmfreeland]

No luck so far. Tried to import a sysex bank from disk to see if I could send any patches and failed there. I also just tried to verify that it was connecting and see error text scrolling by. Could be user error but no progress so far.

[markusschloesser]

thanks for testing! Can you please post the midi log here?

[markusschloesser]

ok, please try this one. I can successfully import single sounds and bulk single sounds, but currently have a problem with banks. So if you could try to import a single sound and send it to the MKS70? Roland_MKS-70.zip

[markusschloesser]

now that my mks70 has arrived, I can test myself. Current state:

1. only thing that works, is receiving manual bulk dump for INTERNAL bank (and even that produces error message) knobKraft-Adaptations\Roland_MKS-70.py(173): extractPatchesFromBank 18:56:40: error Adaptation[Roland MKS-70]: Error calling extractPatchesFromBank: TypeError: object of type 'int' has no len() but hey at least something works 💪☺

2. loading from cartridge import without names

3. edit buffer request does not work

4. bank dump request does not work Will continue

[christofmuc]

@markusschloesser Awesome! We need to teach chatGPT how to read sysex specs and implement the adaptations for us :-)

Let me know if I can be of help, I had done the MKS-80 in MKS-50 in C++ because of the bit shuffling required reading the bank dumps, but by now I have found out how to do that even more simple in Python (it is done in the Yamaha DX-7 code)

[markusschloesser]

@christofmuc ChatGPT knows a lot about sysex! Question always is, if it is hallucinations 🤔😂 (see my message from the 23rd of March) Could really use your help on the "TypeError: object of type 'int' has no len()". No clue where this is coming from and I keep running into this all the time

[christofmuc]

Ah, the int/len bug is exactly the mismatch of message vs messages: You have a loop over a list of messages in your code, but actually messages is just one message, which turns the loop over a list of messages into a loop over a list of bytes (the single message is a list of bytes), and suddenly you have an int instead of a list, and then len(int) is not defined. This is a typical problem for untyped languages.

So I think the correct solution is that extractPatchesFromBank() will be called multiple times, once for each mesasge that was detected as being part of the bank dump. And it is allowed to return multiple patches, but again in a single list of bytes that forms multiple MIDI sysex messages. I think in the MKS-80 code in C++ each message contains 4 patches, so 16 mesages result in 64 patches per bank.

I remember from reading the MKS-70 docs that it actually is more limited in MIDI than the MKS-80 though it is a later synth, and e..g bank dump request doesn't exist, so you'd always need to use manual dump function.

If you have a file from the MKS-70 you could attach here, I'd be able to help!

[markusschloesser]

Ah, the int/len bug is exactly the mismatch of message vs messages: You have a loop over a list of messages in your code, but actually messages is just one message, which turns the loop over a list of messages into a loop over a list of bytes (the single message is a list of bytes), and suddenly you have an int instead of a list, and then len(int) is not defined. This is a typical problem for untyped languages.

So I think the correct solution is that extractPatchesFromBank() will be called multiple times, once for each mesasge that was detected as being part of the bank dump. And it is allowed to return multiple patches, but again in a single list of bytes that forms multiple MIDI sysex messages. I think in the MKS-80 code in C++ each message contains 4 patches, so 16 mesages result in 64 patches per bank.

ah ok, good to know, will try to fix next week!

I remember from reading the MKS-70 docs that it actually is more limited in MIDI than the MKS-80 though it is a later synth, and e..g bank dump request doesn't exist, so you'd always need to use manual dump function.

the jx8p and jx10 are very limited, the mks70 however is better. And Fred Vecoven fixed afaik all the bad midi and sysex bugs. There editors out there doing a bank dump request and what I have done in the attached adaptation further up, comes from chatgpt (and doesnt work, but that might be because of the len/int thing). Anyway I asked on GS in the vecoven thread and will also sniff next week with midi-ox when using other programs.

If you have a file from the MKS-70 you could attach here, I'd be able to help!

Kalimba.zip

sysex spec, patch file (attached) or adaptation (further up https://github.com/christofmuc/KnobKraft-orm/issues/100#issuecomment-1487712412 )?

[markusschloesser]

this is a bank sysex MKSSYNTH.zip

[markusschloesser]

ok, changed messages to message. When receiving manual dump from cartridge, I still get no names (error 1) and still get error message

21:45:21: error Adaptation[Roland MKS-70]: Error calling extractPatchesFromBank: TypeError: object of type 'int' has no len()

At:
  C:\Users\Markus\KnobKraft-Adaptations\Roland_MKS-70.py(188): unescapeSysex
  C:\Users\Markus\KnobKraft-Adaptations\Roland_MKS-70.py(172): extractPatchesFromBank

(error 2) Strangely the manual bank dump receive gets imported under "Edit Buffer Imports"? Also before import starts, I see this in the log " warning Did not find item in tree: EditBufferImport" Another strange thing: patches do get colorized?? 🤔 But patches do get imported (I assume, cannot really test, as sending patches doesn't work yet) @christofmuc Ideas? EDIT: current py attached Roland_MKS-70.zip EDIT2: now import from files on disk, do no get names as well?? (but are also colorized)

[christofmuc]

@markusschloesser I did my own try on the MKS-70, attached here: https://raw.githubusercontent.com/christofmuc/KnobKraft-orm/mks70/adaptions/Roland_MKS-70.py

We have several challenges here, just like the MKS-80.

It has Patches and Tones, and Patches reference Tones at a certain slot. It has no request edit buffer call, but I think it should be able to receive a patch into the current edit buffer (getting just an APR message and no PRG message). Request Program Dump works by sending just a program change (?) Request Bank Dump is not possible, but initiating a bank dump from the synth should work with the receive manual dump menu entry. Most patches in the Internet seem to be in the form of saved bank dumps (BLD type messages).

I have implemented reading the Tones from a Bulk Dump and importing them as individual patches in Edit Buffer format, so loading a file and sending to synth could work.

Not sure about the detection, but that should actually also work, sending program changes on all channels should make the synth answer with an APR and a PGR message.

I interpret: a pair of PRG and APR messages as a single program a single APR message as an edit buffer BLDs are only read, never produced

I currently skip patches from BLD dumps, because the format documented is different in byte count than the APR message, unlike with tones. We need to experiment a bit more with that.

Let me know what you can find out!

[markusschloesser]

Do you also own one? If yes, with Vecoven firmware? From the Vecoven manual: "When a program change is sent, the JX reloads the Patch/Tones. With System Exclusive ON, the JX-10 was also sending all the patch and tones parameters as 3 messages (1 Patch APR, 2 Tones APR). Now, data sent over MIDI can be exactly chosen." and also page 51 ff from http://www.vecoven.com/superjx/downloads/files/JX_10_MANUAL_V3.21.pdf

I already asked at Gearspace, cos some editors DO use a bank dump request. EDIT: sniffed with Midi-OX, bank dump requests are done by doing multiple program changes.

Thanks, will check and report back! BTW You are using messages, not message?

[markusschloesser]

tried your adaptation.

1. When opening my existing db file i get "error Adaptation[Roland MKS-70]: Error calling calculateFingerprint: Exception: Can't calculate fingerprint of non-edit buffer or program dump message At: C:\Users\Markus\KnobKraft-Adaptations\Roland_MKS-70.py(314): calculateFingerprint 01:44:57: error Adaptation[Roland MKS-70]: Error calling calculateFingerprint: Exception: Can't calculate fingerprint of non-edit buffer or program dump message" all over in the log. And all my previous names are now invalid.
2. manual dump from synth does not work. First lots of "warning Ignoring sysex message found, not implemented: f0 41 3a 00 24 30 01 00 00 53 59 4e 54 48 20 42 00 41 53 53 20 2f 20 50 00 41 44 20 20 3f 54 27 00 26 00 4a 13 3d 00 0b 41 00 4e 1f 00 41 0c 43 00 1b 01 4b 0c 10 30 00 03 00 7e 7f 27 04 00 20 40 f7" followed by lots of "warning Ignoring sysex message found, not implemented: f0 41 3a 00 24 20 01 00 00 53 59 4e 54 48 20 42 53 53 01 20 45 00 00 38 41 00 00 40 00 00 40 00 00 7f 32 00 3c 0f 00 00 3d 28 7f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7f 00 7f 00 7f 00 00 1c 27 00 00 44 11 00 00 00 5c 00 05 10 00 2d 6d 01 27 08 00 01 00 40 f7"
3. bank and program import from file works. EDIT: some work, some don't. There's even a couple of banks where some patches work, whereas the rest shows invalid and "Error calling calculateFingerprint: Exception: Can't calculate fingerprint of non-edit buffer or program dump message"
4. sending a patch to synth, does something (mks70 shows changes), but not the right thing, I think it overwrites ONE Tone (B?), but should overwrite a patch

[christofmuc]

Ok, that's encouraging! You need to use a new database, as my format is very different from your version of the adaptation.

Did autodetection work?

Sending a patch seems to do the right thing - currently, I would expect it sends one tone into the highest tone slot (49). It will not be picked up by the patch yet unless you set the patch to use tone 49.

I have to investigate the sysex errors tonight, that seems odd.

And no, I don't have one (yet).

[markusschloesser]

Ok, that's encouraging! You need to use a new database, as my format is very different from your version of the adaptation.

OK, will test with fresh version

Did autodetection work?

didn't try that, from my experience it so rarely works (not because of kk, but probably because shitty synth or because very complex midi setup here) EDIT: tried it, auto detect did not find the mks70

Sending a patch seems to do the right thing - currently, I would expect it sends one tone into the highest tone slot (49). It will not be picked up by the patch yet unless you set the patch to use tone 49.

No, it sends a (1) Tone to the currently selected patch. I can here the new Tone together with the other layer and if I move the layer slider to the one layer, I only hear the new tone .Imho it should send a patch (not a tone) EDIT: Now I see that you wrote "

I have implemented reading the Tones from a Bulk Dump and importing them as individual patches in Edit Buffer format, so loading a file and sending to synth could work.

So that makes sense, that I only get tones and not patches and also why with those, the tones themselves appear multiple times in KK (cos Tones get reused in patches) :-)

And no, I don't have one (yet).

You should get one (with Vecoven), they are great! :-)

BTW Did you see the comment about bank dump request? tried importing from synth, KK sends " DIN8 MKS70 Sysex [f0 41 10 16 11 00 01 00 f7]", which won't work. How do I send consecutive program changes instead? Is there a synth adaptation that does that?

[christofmuc]

@markusschloesser I don't understand the hex message. It starts 0xf0 0x41 0x3a - what is the 0x3a supposed to be? The Verhoeven docs you linked and the original manual don't have any message with code 0x3a. I did not find anything that the new firmware also introduces new messages?

[christofmuc]

My guess would be they have added a new bulk dump format because of the new PWM features. But why is it not documented?

[christofmuc]

Ok, I went a bit deeper into that Rabbit hole, but it seems we're stuck without a change in the program.

The proposal would be:

• One "patch" in the sense of KnobKraft is an MKS-70 Patch with its two Tones.
• An EditBuffer would be three APR messages, one patch APR and two tone APR
• A SingleProgramDump would be either
  • Three BLD messages - one Patch BLD and two Tone BLDs or
  • Six messages - one Patch PRG, one Patch APR, and two Tone PRG and two Tone APRs (of course in the order PRG/APR/PRG/APR)

The open issues:

• Your synth seems to like to use 0x3A for bulk dumps, while 0x37 would be correct. Maybe these are extended bulk dumps for the new firmware?
• The documentation for the BLD of a patch states it is nibbled data, and 96 nibble bytes are sent. But this only results in 48 bytes of patch data, while the APR of a patch needs 52 bytes. Where are the four bytes/parameters?
• I assume when we sent a patch into the synth's memory, we need to update the upper and lower tone numbers to represent the two tones we're going to send as well? If it works like the MKS-80, that is not a problem when sending to the edit buffer without program number, because it seems the Patch always uses the two tones in the tone edit buffers, no matter the patch number. Which is fine for sending edit buffers, but not for bank management.
• For bank management, you need to drag the Tones individually into the Tone bank, and how are the references going to work?
• The C++ code cannot deal with the extractPatchesFromBank() returning multiple patches with multiple MIDI messages per patch. This needs to be changed, and needs a new release. While we're at it, we could also change the code to be more in line with the rest of the Capabilities, I just need to check how many synths have implemented bank dump already. Not too many, I think.

[christofmuc]

@markusschloesser Forgot to tag you on the previous message, but here is one more for the bulk dump request: This is great - I already implemented createProgramDumpRequest to send program change messages. It was a bit unclear on which MIDI channel this should be done. But if you kick off a bank import for the patch bank, you should get a MIDI log (which would be very interesting to see!). Basically, forget about bulk dump request, we can just iterate the bank like with all other synths. The bank dump is only important for loading files from the Internet.