Squircle-CE icon indicating copy to clipboard operation
Squircle-CE copied to clipboard

feat: show non printable charactors

Open liyujiang-gzu opened this issue 5 years ago • 11 comments

  • If your text file encoding is UTF-8 with BOM, \ufeff is invisible, you don't known can remove BOM whether or not.
  • For indention, you use 4 whitespace instead TAB (\t), but you don't known current indent delimitor is \t or 4 whitespace.
  • For word wrap, if line number is disabled, you don't known the next line is break word or break by \n.

Describe the solution you'd like

Add a preference to setting non printable chars (such as whitespace, \n, \t...) can show or not.

Describe alternatives you've considered

Codemiror (JavaScript edition) ACE editor (JavaScript edition) TextWarrior (Java edition)

    private static final String GLYPH_NEWLINE = "\u21b5"; //↵
    private static final String GLYPH_SPACE = "\u00b7"; //·
    private static final String GLYPH_TAB = "\u00bb"; //»

liyujiang-gzu avatar Jul 31 '20 15:07 liyujiang-gzu

I'm sorry! I tried to modify onDraw, but because I was not familiar with kotlin, so failed.

liyujiang-gzu avatar Jul 31 '20 15:07 liyujiang-gzu

In fact, I don't know too much about encodings 😂 Could you send me a file with UTF-8 w/ BOM so I could test it? And what did you tried to do in onDraw?

~~I think the main problem here is in the LocalFilesystem's loadFile method, and how Kotlin reading a file.~~

massivemadness avatar Jul 31 '20 16:07 massivemadness

  • ACE Editor Screenshot_2020_0801_031008

  • CodeMirror Screenshot_2020_0801_031113

  • ModPE IDE Screenshot_2020_0801_031213

liyujiang-gzu avatar Jul 31 '20 19:07 liyujiang-gzu

https://github.com/liyujiang-gzu/ModPE-IDE/blob/liyujiang-gzu-patch-1/utf8_with_bom.js

liyujiang-gzu avatar Jul 31 '20 19:07 liyujiang-gzu

Oh, now I get it 😂 But I don't have any idea how they implemented that. All I know is that they using WebView and all their code written in javascript, so simply copy-paste wouldn't work 🤷‍♂️ 😂

massivemadness avatar Aug 01 '20 07:08 massivemadness

For displaying non printable characters, I try to custom NonPrintableSpan for matching substitution (similar to SyntaxHighlightSpan), which seems to work, but except for carriage-return (\r) and line-feed (\n). screenshot

public class NonPrintableSpan extends ReplacementSpan {

    public NonPrintableSpan() {
        super();
    }

    @Override
    public int getSize(@NonNull Paint paint, CharSequence text, int start, int end, Paint.FontMetricsInt fm) {
        CharSequence subSequence = text.subSequence(start, end);
        if (TextUtils.equals(subSequence, "\ufeff")) {
            return (int) paint.measureText(" ");
        }
        if (TextUtils.equals(subSequence, "\t")) {
            return (int) paint.measureText("    ");
        }
        return (int) paint.measureText(subSequence.toString());
    }

    @Override
    public void draw(@NonNull Canvas canvas, CharSequence text, int start, int end,
                     float x, int top, int y, int bottom, @NonNull Paint paint) {
        CharSequence subSequence = text.subSequence(start, end);
        int codePoint = Character.codePointAt(subSequence, 0);
        LogUtils.debug(getClass().getSimpleName() + ".draw: start=" + start + ", end=" + end +
                ", x=" + x + ", y=" + y + ", codePoint=" + codePoint);
        String replaceStr = subSequence.toString();
        switch (codePoint) {
            case 0xfeff: // UTF-8 with BOM (Byte Order Mark: 0xef 0xbb 0xbf)
                LogUtils.debug("绘制BOM头");
                replaceStr = "\u00a4";//¤
                break;
//            case 0xa: // Line-Feed (LF,\n)
//            case 0xd: // Carriage-Return (CR,\r)
//                // 坑:回车换行时并不会触发本方法
//                LogUtils.debug("绘制换行符");
//                replaceStr = "\u21b5";//↵
//                break;
            case 0x9: // Tab (\t)
                LogUtils.debug("绘制制表符");
                replaceStr = "\u00bb";//»
                break;
            case 0x20: // Space
                LogUtils.debug("绘制空格符");
                replaceStr = "\u00b7";//·
                break;
            default:
                LogUtils.debug("绘制其他符");
                break;
        }
        paint.setColor(0xFF999999);
        canvas.drawText(replaceStr, x, y, paint);
    }

}

liyujiang-gzu avatar Aug 01 '20 16:08 liyujiang-gzu

Non printable characters regex:

[\u0000-\u001f\u0020\u007f-\u009f\u00ad\u061c\u200b-\u200f\u2028\u2029\ufeff\ufff9-\ufffc]

liyujiang-gzu avatar Aug 01 '20 16:08 liyujiang-gzu

Looks awesome. I wasn't sure that we could use spans for this, but it seemed to be OK

massivemadness avatar Aug 01 '20 17:08 massivemadness

Looks awesome. I wasn't sure that we could use spans for this, but it seemed to be OK

A preliminary test shows that replacing non printable characters with visible characters with custom ReplacementSpan is as perfect as SyntaxHighlightSpan. screenshot

liyujiang-gzu avatar Aug 02 '20 02:08 liyujiang-gzu

Simple code :

            val nonPrintable = Pattern.compile(
                "[\\s\u0000-\u001f\u0020\u007f-\u009f\u00ad\u061c" +
                    "\u200b-\u200f\u2028\u2029\ufeff\ufff9-\ufffc]"
            )
            val matcher = nonPrintable.matcher(text.subSequence(lineStart, lineEnd))
            while (matcher.find()) {
                text.setSpan(
                    NonPrintableSpan(color = "#ff666666".toColorInt()),
                    matcher.start() + lineStart, matcher.end() + lineStart,
                    Spannable.SPAN_INCLUSIVE_INCLUSIVE
                )
            }
/**
 * Created by liyujiang on 2020/08/01
 */
class NonPrintableSpan(@param:ColorInt private val color: Int) : ReplacementSpan() {
    override fun getSize(
        paint: Paint,
        text: CharSequence,
        start: Int,
        end: Int,
        fm: FontMetricsInt?
    ): Int {
        val subSequence = text.subSequence(start, end)
        if (TextUtils.equals(subSequence, "\ufeff")) {
            return paint.measureText(" ").toInt()
        }
        return if (TextUtils.equals(subSequence, "\t")) {
            paint.measureText("    ").toInt()
        } else paint.measureText(subSequence.toString())
            .toInt()
    }

    override fun draw(
        canvas: Canvas, text: CharSequence, start: Int, end: Int,
        x: Float, top: Int, y: Int, bottom: Int, paint: Paint
    ) {
        val subSequence = text.subSequence(start, end)
        val codePoint = Character.codePointAt(subSequence, 0)
        var replaceStr = subSequence.toString()
        when (codePoint) {
            0xfeff -> replaceStr = "\u00a4"
            0x9 -> replaceStr = "\u3009"
            0x20 -> replaceStr = "\u002e"
            else -> {
            }
        }
        paint.color = color
        paint.textSize = 8.dpToPx().toFloat()
        canvas.drawText(replaceStr, x, y.toFloat(), paint)
    }

}

liyujiang-gzu avatar Aug 02 '20 03:08 liyujiang-gzu

@liyujiang-gzu It works pretty well, but it's just too laggy when working with large files (especially scrolling), the reason is amount of spans. Right now I have no idea how to optimize this, so I don't think it would be a good idea to implement this in the next version Here's the file for tests JavaScriptAPI.zip

P.S I forgot to change tab size on screenshots

massivemadness avatar Aug 24 '20 17:08 massivemadness