TwentyWorkspaces

[colba]

TwentyWorkspaces

Modifications to increase number of Workspaces from 10 to 20.

Sucessfully built and tested on Windows 11.

Why ?

I love live coding in Sonic-Pi but when experimenting with new ideas or having lots of elements to my composition I end up running out of Workspaces and have to use multiple COMMENT / UNCOMMENT blocks in Workspaces to control whats happening.

I thought it would be nice to have more than the default 10 Workspaces so I've made a few mods to have 20 Workspaces.

Changed files :

.\app\gui\qt\mainwindow.h
.\app\gui\qt\mainwindow.cpp
.\app\api\src\sonicpi_api.cpp
.\app\api\src\string_utils.cpp

Result

20 Workspace tabs instead of 10

next/previous keyboard shortcuts S-M-[ / S-M-] work

Each Workspace is saved to ~\.sonic-pi\store\default\ and reloaded at startup

~\.sonic-pi\store\default\workspace_zero.spi
~\.sonic-pi\store\default\workspace_one.spi
~\.sonic-pi\store\default\workspace_two.spi
~\.sonic-pi\store\default\workspace_three.spi
~\.sonic-pi\store\default\workspace_four.spi
~\.sonic-pi\store\default\workspace_five.spi
~\.sonic-pi\store\default\workspace_six.spi
~\.sonic-pi\store\default\workspace_seven.spi
~\.sonic-pi\store\default\workspace_eight.spi
~\.sonic-pi\store\default\workspace_nine.spi
~\.sonic-pi\store\default\workspace_ten.spi
~\.sonic-pi\store\default\workspace_eleven.spi
~\.sonic-pi\store\default\workspace_twelve.spi
~\.sonic-pi\store\default\workspace_thirteen.spi
~\.sonic-pi\store\default\workspace_4teen.spi
~\.sonic-pi\store\default\workspace_fifteen.spi
~\.sonic-pi\store\default\workspace_6teen.spi
~\.sonic-pi\store\default\workspace_7teen.spi
~\.sonic-pi\store\default\workspace_8teen.spi
~\.sonic-pi\store\default\workspace_9teen.spi

• Note the names had to be chosen to avoid incorrect number/name matching
• i.e. workspace 14 as fourteen, workspace 4 would match four and fourteen

Future improvements :

The way the number of Workspaces and their naming is implemented isn't ideal and could be improved by having the maximum number defined in one place in a single header file.

The functions/methods to generate the Workspace file name from the Workspace number and from name to number could be done in one place in one source file and be able to uniquely name the file for any sensible number.

ToDo:

1. change Workspace number/name conversion to work for any number creating a unique name match.
2. make the number of Workspaces configurable in Preferences

[FRex]

I fixed Image and took a look at Font a bit, and I've dealt with Unicode a decent amount in few languages and OSes so I'll take a look at these later this week when I have time.

I assume it'll just be converting from using C libs char* functions to use callbacks to std::fstream that we open with the given std::filesystem::path.

The one shot (open, load all, close) classes (like Image) are easy, but Music/Font will be tricky. Now a C lib keeps file open OR user must keep memory or SFML stream alive. After this change we'll have to keep a file open. For Font maybe we can put it in struct Font::FontHandles, for Music I'll have to look.

Shader actually already does it right but I'll verify it too (and tests could be added): https://github.com/SFML/SFML/blob/master/src/SFML/Graphics/Shader.cpp#L85

If we don't want to write callbacks for each C lib we use, we could create SFML stream backed by std::fstream and pass it to existing load/save stream functions too.

BTW: the Image PR also fixes printing Unicode filenames to sf::err() (making them UTF-8). Good filenames to test with and explanations are there too (Latin 1, outside Latin 1, CJK, non-BMP emoji).

P.S.: If someone has Unicode question feel free to ask here or on Discord. I don't know everything (especially codepages and A versions of WinAPI functions, I always stuck to Unicode aware W versions), but I know a fair bit and I test my own C/C++ programs with tricky filename like aa😝日本語🔪.bmp on Windows (the only problematic OS, where just assuming "char* is UTF-8, just be careful with length/cutting/display" doesn't work).

[ChrisThrasher]

https://github.com/SFML/SFML/pull/1964

This PR replaces uses of std::string with std::filesystem::path when representing filesystem paths. This better expresses intent and opens the door for better supporting non-ASCII file paths.

This was an explicit goal from the start of using std::filesystem::path so I'd love to see us make progress on this! Hopefully because we're using std::filesystem for all filesystem-related APIs, this can be achieved without any API breakages. I believe it can.

[FRex]

sf::Shader::loadFromFile() is fine, so once test is added it can be marked done.

#include <SFML/Graphics.hpp>
#include <iostream>

int main(void)
{
    const char32_t totry[] = {
        0xF1, // spanish n tilde
        0x144, // polish n accent
        0x65E5, // sun cjk char
        0x1F40C, // snail emoji
    };

    sf::Shader shader;
    std::cout << std::boolalpha;
    for(const auto c : totry)
    {
        char32_t fname[] = U"shaderX.txt";
        fname[6] = c;
        const std::filesystem::path path{fname};
        const bool ok = shader.loadFromFile(fname, sf::Shader::Type::Vertex);
        std::cout << (const char*)path.u8string().c_str() << ' ' << ok << std::endl;
    }
}

Outputs (when piped to file, file became UTF-8):

shaderñ.txt true
shaderń.txt true
shader日.txt true
shader🐌.txt true

To cmd terminal:

shader├▒.txt true
shaderń.txt true
shaderµùÑ.txt true
shader­ƒÉî.txt true

[FRex]

I looked at sf::Utf<8> and it has some issues too. I didn't look at String, 16 or 32 (yet?).

1. docs have some bad wording, like saying Unicode is 32-bit or saying "UTF-8 character" which is not right wording, there is only Unicode "characters", UTF-8 has code units, plus when dealing with non-ASCII and Unicode, you have more than one code unit, so you deal with a UTF-8 "sequence"). IMO we should use terms exactly as https://www.unicode.org/glossary/ uses them.
2. Links to references are dead, except unicode org one. We could use more specific links, like https://www.unicode.org/glossary/ (for all the terms like code point, code unit, etc.) https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-3/#G7404 https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-3/#G28070 for specific features. The Unicode Standard has a section on how to cite/refer to it too.
3. Decoder handles 5 and 6 byte long sequences, which is a spec violation and will produce too high values (might surprise someone and won't ever map to any character anyway). It also handles overly long sequences (e.g. 2 byte encoding of an ASCII char), which spec says is a violation too.
4. Decoder also says it'll check for invalid sequences but checks only for truncated ones, for ones that are missing the front byte it'll happily decode continuation bytes incorrectly (see below). It'll also not check that bytes following initial one are actually continuation bytes, it just blindly reads them and subtracts 0b10000000 (via the precomputed offsets array). It'll also decode surrogates (meant only for UTF-16 use) which is a spec violation, but: WinAPI does that too (that's why some people sometimes call UTF-16 and 8 on Windows as WTF-16 and 8 https://simonsapin.github.io/wtf-8/ ), Python 3 doesn't, it replaces them with replacement char U+FFFD.
5. Encoder won't encode too high values (that'd be 5-6 byte sequences), but it also means that encode(decode(sequence)) will not produce the original sequence back, so it's wrong and not self-consistent with each other.
6. Few places do static_cast<typename Out::container_type::value_type> as if assuming only standard container iterators are used. It won't allow ostream output iterator, pointers, etc. Since C++11 added decltype this cast can be rewritten using std::remove_reference and decltype to allow any outputting to anything that has operator* and ++. This is just C++ stuff, not Unicode.
7. The replacement character U+FFFD could also be mentioned in our docs since it's for purpose of appearing/replacing things upon error, etc.

Fun fact (this is not by design or on purpose): since we decode surrogates and overly long sequences, we accidentally support decoding Java JNI's "Modified UTF-8" that does both of those (spec violating) things: https://docs.oracle.com/javase/8/docs/api/java/io/DataInput.html#modified-utf-8

Note: for well formed UTF-8 it works/decodes correctly!

Here's a program that (incorrectly) decodes single continuation byte (0b10 at front as a leading sequence byte never can have) as a yen sign ¥, then encodes it (properly) to a 2 byte sequence.

#include <SFML/System/Utf.hpp>
#include <vector>
#include <stdio.h>

int main(void)
{
    const std::vector<unsigned char> buff1 = {0b10'100'101}; // yen sign value, looks like a utf8 continuation byte
    for(const unsigned char c : buff1) printf("0x%02x, ", (unsigned)c);
    puts("");

    char32_t utf32;
    sf::Utf<8>::decode(buff1.begin(), buff1.end(), utf32);
    printf("decoded to U+%x\n", (unsigned)utf32);

    std::vector<unsigned char> buff2;
    sf::Utf<8>::encode(utf32, std::back_inserter(buff2));
    buff2.push_back('\0');
    printf("encoded to: >%s< aka: ", buff2.data());
    buff2.pop_back();
    for(const unsigned char c : buff2) printf("0x%02x, ", (unsigned)c);
    puts("");
}

Other than official Unicode org website and Wikipedia, Python also has has very nice unicodedata module and bytes.encode and str.decode functions to play with Unicode. I also have a simple website to inspect Unicode: https://frex.github.io/unicode.html

[FRex]

Loading SoundBuffer or Music from file already works, since they use InputSoundFile which uses FileInputStream that uses _wfopen on Windows. Saving SoundBuffer works, except for flac files because there is filename.string().c_str() there (which throws).

Here's a test program, using files in dir test🐌:

#include <SFML/Graphics.hpp>
#include <SFML/Audio.hpp>
#include <filesystem>
#include <iostream>

int main(void)
{
    const std::filesystem::path dir(U"test\U0001F40C");

    sf::Texture tex;
    sf::SoundBuffer buffer;
    sf::Music music;

    std::cout << std::boolalpha;
    std::cout << tex.loadFromFile(dir / "png.png") << std::endl;
    std::cout << buffer.loadFromFile(dir / "wav.wav") << std::endl;
    std::cout << buffer.saveToFile(dir / "output-wav.wav") << std::endl;
    std::cout << buffer.saveToFile(dir / "output-wav.ogg") << std::endl;
    //std::cout << buffer.saveToFile(dir / "output-wav.flac") << std::endl;
    std::cout << music.openFromFile(dir / "ogg.ogg") << std::endl;

    music.setLooping(true);
    music.play();

    sf::Sound sound(buffer);
    sf::Sprite spr(tex);

    sf::RenderWindow win(sf::VideoMode(sf::Vector2u(800, 600)), "Testing Unicode");
    win.setFramerateLimit(30u);

    while(win.isOpen())
    {
        while(const auto eve = win.pollEvent())
        {
            if(eve->is<sf::Event::Closed>())
                win.close();

            if(eve->is<sf::Event::MouseButtonPressed>())
                if(sound.getStatus() != sf::Sound::Status::Playing)
                    sound.play();
        }

        win.clear();
        spr.setPosition(win.mapPixelToCoords(sf::Mouse::getPosition(win)));
        win.draw(spr);
        win.display();
    }
}

[FRex]

The snail has finally proven its purpose.

Window title is broken for non-BMP (and only them) chars on Windows because of how sf::String::toWideString is implemented. BMP chars (most languages anyone uses, including non-obscure non-academic CJK characters) do work.

Same for sf::Clipboard::setString (also uses the sf::String::toWideString).

sf::Clipboard::getString is also wrong, it copies given wchar_t so if 日🐌 is in clipboard it will turn into 3 long string, the CJK, and then the two surrogates, this isn't valid UTF-32, the surrogates are only for UTF-16. Possibly more? I didn't look.

Example: The snail emoji is stripped from the title, but BMP (first 0xffff values) Unicode (the 日 CJK char) is okay.

If you start with 日🐌 in your clipboard and get window title via WinAPI and string from clipboard, it's clear the snail emoji aka 0x1f40c was removed:

codepoints in ctrlv: d83d dc0c 65e5
codepoints in sfstr: 1f40c 65e5
codepoints in title: 65e5
codepoints in ctrlc: 65e5

2nd line has the right values, Unicode codepoints for both chars. Window title and getting from clipboard after setting is wrong - snail is gone. Getting from clipboard is wrong - the snail is split into surrogates.

#include <SFML/Graphics.hpp>
#include <SFML/Audio.hpp>
#include <filesystem>
#include <iostream>

#define WIN32_LEAN_AND_MEAN 
#include <Windows.h>

int main(void)
{
    const std::u32string utf32str(U"\U0001F40C\u65E5");
    const sf::String sfstr(utf32str);

    std::cout << "codepoints in ctrlv:";
    for(const char32_t c : sf::Clipboard::getString())
        std::cout << ' ' << std::hex << (unsigned)c;
    std::cout << std::endl;

    sf::Clipboard::setString(sfstr);

    sf::RenderWindow win(sf::VideoMode(sf::Vector2u(800, 600)), sfstr);

    wchar_t buffer[100] = {0x0};
    GetWindowTextW(win.getNativeHandle(), buffer, 100);

    std::cout << "codepoints in sfstr:";
    for(const char32_t c : sfstr)
        std::cout << ' ' << std::hex << (unsigned)c;
    std::cout << std::endl;

    std::cout << "codepoints in title:";
    for(const wchar_t * cptr = buffer; *cptr; ++cptr)
        std::cout << ' ' << std::hex << (unsigned)*cptr;
    std::cout << std::endl;

    const sf::String fromclipboard = sf::Clipboard::getString();
    std::cout << "codepoints in ctrlc:";
    for(const char32_t c : sf::Clipboard::getString())
        std::cout << ' ' << std::hex << (unsigned)c;
    std::cout << std::endl;

    win.setFramerateLimit(30u);
    while(win.isOpen())
    {
        while(const auto eve = win.pollEvent())
            if(eve->is<sf::Event::Closed>())
                win.close();

        win.clear();
        win.display();
    }
}

[ChrisThrasher]

The snail has finally proven its purpose.

Window title is broken for non-BMP (and only them) chars on Windows because of how sf::String::toWideString is implemented. BMP chars (most languages anyone uses, including non-obscure non-academic CJK characters) do work.

https://github.com/ChrisThrasher/SFML/actions/runs/13401476868

I wrote some tests to confirm this. Everything up to Emoji works fine on all platforms. Emoji work just fine on macOS and Linux as expected.

EDIT: I fixed it https://github.com/SFML/SFML/pull/3435