christofmuc
suggestion: find common ground / levels of working together with the Elektra One guys
I own an Elektra One ( https://electra.one https://github.com/martinpavlas/electra.one https://forum.electra.one ) and imho it might makes sense to figure if you can work together. Possible scenarios:
- use Elektra as a device to remote control knobcraft (e.g. patch selection)
- use knobcraft for Elektra preset management (can also be offline!)
- sysex patch management and parsing: I don't know exactly what might come out of it, but I have a feeling that it might be good things :-) I really really like that both of your ways of working (open source, using github etc). I will make the same suggestion at their github
Hi Markus,
thank you for the suggestion! I had discovered the Electra one actually last month only, and really like it a lot - no surprise, because I am a big fan of the Behringer BCR2000 MIDI controller, which features 32 encoders with programmable LED rings, next best thing to a touch screen. The one main feature I am missing with the BCR2000 is that you have to remember which encoder is what, and there is no display to show you the name of the parameter you are changing. I added that via a software running on the PC ;)
Ideas how to utilize the Elektra, in ascending order of complexity. Not sure if all of these are possible technically, but I don't see any reason why not:
- Use the Elektra Controller as I use the BCR2000 now (not part of the current KnobKraft release) - send MIDI CC messages to the computer running the KnobKraft, and it will translate the parameter changes into synth specific changes so you can edit vintage synths without bothering about the sysex stuff.
- Expand that by sending the names of the parameters into the Elektra, so you have a nice explanation of what the knobs do
- Use the KnobKraft software to generate the device information for the Elektra. I already do this for the BCR2000, as all the parameters are known for some of the synths already, I can generate the BCR2000 control code automatically, no need to manually setup it up. I can also regenerate in case a MIDI channel is switched and all sysex codes need to be adjustet. This would allow the Elektra One to be used standalone as an editor.
- You can use the Elektra Controller to remote control the KnobKraft librarian software. Not sure how useful this is, as my middle term plan is to compile the KnobKraft also for iPad and Android Tablet (which is why the UI is rather touch-based design than Mouse design), and you could attach a MIDI cable to your tablet and then use the tablet's touch screen to use the librarian, bolting it to the keyboard rack.
- Share device definition (epr) files, or load them by the Orm in addition to adaptations
- Compile a version of KnobKraft to run solely standalone on the Elektra. Not sure if the Elektra is a Linux based system, then it would be straight forward as KnobKraft runs on Linux already. If Elektra is running on a different OS, this might be not feasible, I would need an OS supported by JUCE.
That's just a first shot!
Hi Chris, yep, the display on the Elektra is a huge advantage and with the new fw you can do now really cool stuff like displaying ADRS envelopes etc. Your suggestions sound good! Specifically:
- not sure about that, the Elektra can do sysex and system common (btw there are mks80 presets for the elektra in the preset website, maybe helpful for your mks80 adaptation?)
- would be great to expand amount of supported devices quickly
- could be good for patch management Best regards Markus
I looked a bit more into the Elektra files, and some more thoughts pop up:
- The instrument definition files I could generate for all synths where I already have the parameter info for, but those are not too many (mostly my own): Matrix 1000, MKS-80, Rev2, DW-8000, and K3M.
- I see that the Elektra has a feature for syncing with dumps, but that is not fully documented yet I think.
- This looks so useful I wonder if there is a PC version of the Elektra editor. It might completely replace CTRLR for me, and I don't need the hardware, but just the editor definition files.
- I don't understand the difference between the EPR and IPR files. It seems they contain the same data in two different structures (Technobear made a conversion tool)
Hi,
sorry for belated response. Electra made my life a bit of a rollercoaster recently.
I will try answer your questions on the lines below.
Electra runs on a baremetal ARM with 16MB of RAM. That means, unfortunately, that JUCE app will not run on it. The rest of your points, however, could be done.
The EPR file is a preset definition. It s a source file with the actual preset definition that can be loaded to Electra controller. It has been designed so that it can take up to 16 midi devices in the play and have up to 432 controls (faders, lists, envelopes, pads). Each device may have also a special definitions for parsing incoming sysex. This parser translates invidividual bytes and bits into logical paramaters inside Electra. This is not really documented yet, I am working on it these days.
The EIF (instrument file) is a definition of the MIDI implmentation of a particular MIDI device. Say, if you have Kawai K3, the EIF file will describe all parameters, their types, text labels, MIDI messaging, as well as the messages to request SysEx patch dump and to rules to parse incoming SysEx patch dump. For now, we put EIF files a bit aside. Majority of the users is more "preset" oriented. Also, a big reason is that we have a preset editor, but the instrument file editor does not exist yet :) I think, however, that the idea of EIF is a of Electra's key features. The idea is that there is a handful of EIF files available and users can use them to build presets. When you build a preset using an EIF file, you basically just drag and drop controls in place. You do not need to thing about MIDI implementation at all - it's all covered in EIF.
So from above you can say, that EIF is a MIDI implmentation description of a particular synth and can be used to build presets (and cannot be loaded to electra). EPR is preset, ie. a collection of pages and controls for up to 16 MIDI devices at once. Some of the devices can use information from the EIF file, while others might be just an ad hoc messages defined at the time of creating a preset.
What @thetechnobear did - and thanks for that - is that his tool can generate an EIF file out of EPR files. Users created many presets that are just an implementation of one instrument.
There is also an intention that the EIF file would be used to generate documentatio of a MIDI implmentation. ie. you could use it to create Electra presets, but even if you did not have Electra it would still give you info about how to communicate with your synth. Nothing is worse than hunting old scanned user manuals.
I am fully open to discussion about the format and such. With EIF it is completely open field, with EPRs I am restricted to being backwards compatible. There is support for versioning, but the controller is a bit limited in terms of resources (especially RAM, 16MB might sound enough, but it is also used as a frame buffer).
The editor is now a webmidi enabled website only. I played around with JUCE and will definitely use it for native apps development.
Cheers,
Martin
I think perhaps an approach is for KnobKraft to generate and upload presets to the Electra One.
This means you don't need anything more on the Electra One, but rather just enhance KnobKraft to support its format. Ive been working on a project recently where I do something similar.
As a result of this, I have a C++ library that can communicate with the Electra One to do things like upload presets, so perhaps that might be useful to you?!
Im very excited by what Martin is doing with the EIF format... one of the problems with midi is that we dont have an metadata format for the midi messages , so lots of devices end up building something like this - though many are quite simple (e.g. mapping cc and note numbers).
of course midi 2.0 will change this with MIDI-CI. but if we have the metadata in EIF, then hopefully we can easily transform this to midi 2.0 ci 'format'.
I see midi-ci becoming huge (once we start seeing widespread adoption) , musicians have been frustrated for years with the nonsense of 'mapping' CC (etc) to 'real world' parameters :)
Wow, many "thinks" in this thread! Thanks @martinpavlas for getting back on this, totally understand your priorities right now (and great work on the Electra, I think I need one ;-). Thank you @TheTechnobear for joining in.
Let me try to structure:
I. There is common code/knowledge in both tools that could be shared in order to increase the number of devices supported by both.
In Electra, these are are the EIF files, which is JSON data, in KnobKraft these are the adaptation files which is Python code.
I had briefly worked with a data based approach as well (as most Librarians do), but quickly decided that a snippet of "generic code", that has no external dependencies, with a defined interface for the Librarian might do a better job. Working with vintage gear every one you touch brings up new surprises and exceptions, and even with the code/API based approach you constantly need to expand until you have covered all "capability profiles". I also looked at the other free editors and librarians, they mostly favor data over code (with the notable exception of Laser Mammoth, which does a mix of data and code), but I think the data structures will get more complex and arcane when you add more devices that are not "simple". Also, the Python code is much more pleasent to read if you want to understand how the device works than a big JSON file, so in terms of long-term documentation of our beloved vintage gear, I think it is a good thing.
As an example, if you look at this, you will find some complexity e.g. when it gets to poking the name into an existing edit buffer dump, you need to unescape the sysex and escape it again. Other synths maybe do a character set lookup (Access Virus!) and a checksum algorithm in addition. Encoding this in data upfront might be hard: https://github.com/christofmuc/KnobKraft-orm/blob/master/adaptions/DSI%20Prophet%2012.py
Generating EIF files is possible. I have code that generates BCL, the programming language of the BCR2000, which is very similar to the EIF. (Docs for BCL here, German, but you'll get it if you look at the commands and the examples: https://www.sequencer.de/synth/index.php/B-Control-Tokenreferenz)
On the other side - would the Electra theoretically be capable to run Python? I guess not, Lua would have been a better choice I think for embedded hardware. As you will use a PC anyway to load the Electra with its presets for later standalone use, generating EIF (and EPR, as you do with the web editor) is probably the way to go.
The information on the KnobKraft side is found e.g. here for the KawaiK3 at the top of this file, it will look very familar: https://github.com/christofmuc/MidiKraft-kawai-k3/blob/master/KawaiK3Parameter.cpp
Not all synths from KnobKraft are already in Python, I started doing them in C++ and later decided that for extensibility and easier accessibility I should probably not ask people to compile their Librarian ;-)
I could probably make an "EIF" adaptation, the Python code loaded by C++ loading an EIF file, so I could use the EIF files directly. Not sure if everything I need for the Librarian is already in there, though.
II. The KnobKraft will get Editor capabilities eventually, because it is easy with all the code already written.
I have not started that work in earnest because I am aware the main problem in getting a good editor is the layout, and the Electra web designer is the solution for this! So basically I can add support for loading EPR files and display (with JUCE) the panels as were designed for Electra. There will be some code overlap, but generally the EPR format is quite straightforward. Most complexity probably comes from the multi-device/MIDI routing setup, that is of no good use when you run a software-Electra.
This editor will allow external control surfaces like the BCR2000 to remote control it (which was my original use case, mapping a patch that came in via sysex back into the LED rings on the BCR2000).
III. The KnobKraft can be used as a Librarian for the Electra one - that one is probably simple, as I saw that the preset handling is also done via Sysex, and for just storing and recalling presets not much more is needed than to extract the name from the BLOB, and else just store it and send it out again. @TheTechnobear Not sure what you C++ library does, if it generating the sysex messages and doing MIDI communication, that is probably already there in my code. Is it anywhere where I could have a look at it?
- how about sounddiver "ada" files as a base/source for .eif files? There are still a lot of them out there. Question of course would be IP rights issues for manufacture provided files or ones originally from emagic, but that could be mitigated by allowing the user to import into knobcraft locally and not "touching" files themselves online. I briefly looked at one ada file and it looks a lot like text which could be used.
- Alternatively / additionally midiquest adaptations, but I know even less about that
- A "sync" process between knobcraft and Electra for library and for patch editing purposed could be promising down the road, but I don't know yet exactly how/where from.
- I'm really happy I got everybody talking :-) Only good things can come out of this and hopefully everybody involved can save time/profit from it (not in monetary way, but as in only doing things once and proper and re-using other stuff).
I think the ADA file format is closed and not documented? You can see what's inside if you just load the file into SoundDiver and open the adaptation editor, but I am not aware that you can read and understand the ADA files. Don't know anything about MidiQuest.
Actually exactly that situation, that every editor and librarian needs to re-invent the code required to talk to a specific synth, made me create the KnobKraft Orm and the Python adaptation mechanism. Even if currently only the Orm loads and runs these files, they are as code completely independent from any library that is not standard python, and in theory could be used by any other Librarian or Editor in a similar way. Also, they are nice to read (if you know Python) really documentating the behavior of the synth better than the dreaded sysex docs. So in a way, I think these files are the right way to archive the knowledge about the vintage devices long term. Running code is always better than documentation.
Other tools have used many ways for the same purpose, but as far as I know this is the first pure code-based approach that is made public.
Other tools that contain wisdom:
- Sound Diver in its Adaptations. When you have a Sound Diver license, you can open these and see the Sysex operations in the Adaptation Editor.
- Edisyn is an open source editor, with device code public in Java for about 20 devices
- LaserMammoth is an open source librarian for the web, with device code public in a mix of json and code. Sadly it uses the complex Angular library, so you cannot just extract the device code and re-use it in some other app.
- MidiManager is a closed source librarian for the web, but has the possibilty to add devices via the community in json data form. So you're limited in what the closed source allows in that data, if a device is beyond the capabilites of the base, you cannot use it.
- Electra one is closed source hardware editor, with open source data-based EPR editor presets and IPR instrument files. Same situation as with MidiManager - if the data structure is not already prepared for the functionality of the device, you're stuck.
- KnobKraft Orm allows you to implement any device of any complexity in C++, because it is OpenSource, and allows you to implement simpler devices using the Python single-file modules. The Python modules will get more and more functions, so maybe at one point in the future you will not have any need for the C++ implementation.
- CTRLR has many editor panels available, and they are open source and in Lua. If you download a Panel, you can inspect the Lua to see how the device works. In a way, that inspired KnobKraft. Sadly, the complexity of the panels is quite high, and the amount of code duplication inside some of these panels is insane, I have looked at the source code of quite a few. I cannot see any way to reuse any of that outside of the original CTRLR hosting app.
i can open ada files in a text editor and while there's some code, there's also a lot of text info. Will try to find out, if there's any documentation about it
i can open ada files in a text editor and while there's some code, there's also a lot of text info. Will try to find out, if there's any documentation about it
That would be great! And it is easier if you have a SoundDiver - I can open the Ada files in the SoundDiver adaption editor, and then basically you see all of the textual information in context.
You might be aware of those already, but in caseyou're not: https://www.deepsonic.ch/deep/docs_manuals/emagic_apple_sounddiver_programming_manual.pdf and https://www.deepsonic.ch/deep/docs_manuals/emagic_apple_sounddiver_programming_manual_v.3.1_addendum.pdf especially the first one is massive and also a well written reference doc. Mentions all the crazy special implementations, transmission formats, lookup tables for manufacturer IDs etc. Still haven't found a proper file documentation though.
Yes, thank you, I know these. Best I've seen on sysex in general.
I think the ADA format is binary undocumented, making it improbably. I once tried to read the SoundDiver bank files, but even those are stubborn and probably much easier. The best idea is to just load the ADA file into a SoundDiver instance and look at the values using the SoundDiver UI. The license is available in the internet, I posted a link at #53
Another (also w re to your Midi CI / eif discussion): jzml files (lemur templates) contain lots of info about synths. For example see attached the mks80 template. All relevant sysex strings are in there. I think this could be used as something that makes adaptations and/or eif files simpled to produce. In the long run a publicly open repository for translation/adaptation/eif files for midi 1.0 synths would be helpful. What do you guys think of that? MKS-80-Editor-v3.4.jzml.zip
@markusschloesser I am pretty strict about intellectual property rights, as my intention is to build here an AGPL licensable repository of code, that encodes the wisdom from manuals and forum postings on how the devices operate. If the manual from the vendor doesn't have enough information (or is, as often, full of bugs), research needs to be done.
The research that is ok I think is to look at publicly available information on forums and other websites, as well as source code implementations in other open source software - as long as it is not code that is copied, but only information about the device itself is gained.
I am already skeptical of using ADA files, as they are probably still owned by Apple, as they bought eMagic. That is for the ADA files that come with the original SoundDiver, not adaptations made by a third party. There, the rights to that belong to that third party.
Lemur seems to be a commercial product, so I'd rather stay away, because the last thing I need is to get into an Intellectual Property case while developing this nice OpenSource software.
Honestly, when you know what you're doing normally the manual is the best source anyway, because many implementations get it wrong or don't support all features of a device. For the Andromeda, the manual alone was all the information needed.
It can get tricky when you try to support a device that hasn't any official sysex documentation, because in the end you're starting to reverse engineer the communication. But then, when I asked Arturia for the sysex for the Keystep, they just said "just look at the MIDI our editor for it is sending out, it is easy!"...