copying multiple lines, and pasting into another app, inserts unwanted "sentinel" characters

[oresmus]

I'm testing with latest firepad and codemirror, configured for rich text as in the "userlist" example (which I am running a modified copy of). I'm on a Mac and using Firefox, running from local files (no server involved).

If I copy some text which includes multiple lines, and paste that into another app such as TextEdit or Terminal, there are extra (unwanted) unicode characters between lines. This is what it looks like in TextEdit:

This does not happen in an online demo of CodeMirror which contains rich text.

Skimming the code of firepad.js, I am guessing it is related to the "sentinel" characters -- for example, this code says it removes them when text is pasted into firepad:

RichTextCodeMirror.prototype.onCodeMirrorBeforeChange_ = function(cm, change) {
    // Remove LineSentinelCharacters from incoming input (e.g copy/pasting)
    if (change.origin === '+input' || change.origin === 'paste') {
      var newText = [];
      for(var i = 0; i < change.text.length; i++) {
        var t = change.text[i];
        t = t.replace(new RegExp('[' + LineSentinelCharacter + EntitySentinelCharacter + ']', 'g'), '');
        newText.push(t);
      }
      change.update(change.from, change.to, newText);
    }
};

So what is needed might be as simple as also removing them when text is grabbed from the model and inserted into the system clipboard. (I could not find any code involving the sentinel characters which looks like it's trying to do that, but I didn't yet find the code that handles that "copy to clipboard" operation.)

This is tangentially related to #35, in the sense that their fixes might involve the same code, but logically it's an independent and higher-priority issue. Note that this issue has nothing to do with rich text per se -- it happens even when copying lines of plain text (though I did not test whether it happens if the entire editor buffer contains only plain text).

[oresmus]

It looks like the code which modifies the clipboard on a "copy" is only in CodeMirror itself (function onCopyCut), probably not handled by Firepad at all. So perhaps CodeMirror could be patched (or given an option taking a callback function) to filter the copied stuff to remove those characters. Or if we want to avoid modifying CodeMirror, the relevant parts of its code could be duplicated in Firepad, then modified. I guess the gist by @iclems in #35 essentially shows how to do this -- in fact it looks to me like it fixes this bug as well as that one. [edited to remove bad ideas and add better ones]