[Branch: openmath] Use MicroTeX as a static library ?

[jokteur]

Hello again,

I will probably post a few issues as I am discovering how to use this library. I managed to solve #127 by following the examples. It compiles with MSVC and Imgui. I did a quick implementation of graphic_imgui.h and graphic_imgui.cpp.

For now, I have the following problem: as I implement graphic_imgui, which will be exported to a shared library (dll with MSVC), it is impossible to use static variables in the original GUI library. I will make an example to be clearer.

Lets say that loading a font needs a static variable (to check if the application is initialized):

MyGUILibrary::LoadFont(...) {
    // Check app initialization
    if (!App::app_state) { // Calls a static variable
        // Do stuff
    }
}

Now when implementing graphic_imgui, if I do the following:

Font_imgui::Font_imgui(const std::string& file, float size) {
    MyGUILibrary::LoadFont(...);
}

there will be a problem. Indeed, as Font_imgui::Font_imgui symbol will be in the shared library (lets call it microtex-imgui), as soon as this dll will be loaded, the static variable App::app_state will be reinitialized, even if it was correctly initialized in the main thread of the application before the dll was loaded (this is exactly what happens when I test MicroTeX with my dll microtex-imgui).

I do know that static variables are evil, and extern variables even more so. But it may happen that the GUI library uses some internal static variables that may be used when loading fonts or other things. I feel like (when following the examples) that MicroTeX forces me to create a shared library where I need to create an interface within this library that interacts with the rest of the project. It also possible that I missunderstood how to use correctly this project.

I see here three workarounds:

1. Use MicroTeX as a static library, such that the interface graphic_xxx does not need to be loaded at runtime. My question would be: how to do this ? I tried to change the CMakeLists.txt in lib/ but failed miserably.
2. Implement the interface graphic_xxx in an abstract way, such that it don't directly makes calls to the GUI library.
3. Rework completly the GUI library such that it may never ever use static variables -> I don't see this as a viable solution.

I thank you in advance for your answers.

[sp1ritCS]

You could modify the Font_imgui constuctor in a way that it either takes a function pointer to MyGUILibrary::LoadFont (this obviously needs to be a static method for this to work) or you design it and LoadFont to get App as a pointer passed to it.

Then you just need to rework the parts of the library that interact with microtex to never use static variables

[jokteur]

The static variable problem in dll is also a problem whenever I want to use a singleton. I do have a job scheduler that should only be initialized once in the whole program, and calling it from the dll would mean having two job schedulers.

I ended up making an interface that collects all the calls and redistributes them to a separate interface that is not in the dll. This way, my static variables are not reinitialized.

[cesss]

Do you really need to build as a DLL, @jokteur ? When you said "static", I really thought you wanted to build it as a static library, not a DLL. Many years ago, I decided to never build DLLs again (nor shared libs on UNIX either). I took the everything-static approach forever, and my life has been much easier since then 😄 These days, prebuilt apps are in the order of hundreds of MB, and games in the order of GBs (even if they include dozens and dozens of DLLs/DSOs in their installation). My apps are much smaller than that (usually they take about 10MB or so), even if I build everything static.

[cesss]

By the way, can you share your imgui backend? I'd like to be able to build MicroTeX without Cairo and without any dependencies (apart from the dependencies required to use imgui, obviously).

[NanoMichael]

The static build is available now, you may pull the newest code from branch openmath, and put the following code in the file MicroTeX/CMakeLists.txt just works:

set(_BUILD_STATIC TRUE)

include(MicroTeXInstall.cmake)
add_subdirectory(lib)

# ...

I'll make a CMake option to make it work.

[jokteur]

@NanoMichael thank you for your static build. I will test it probably next week.

@cesss it is currently a big WIP where a lot is not working on the moment. I will share it once I have something satisfactory.

[NanoMichael]

OK, I've updated the CMake build, now with the newest code on branch openmath, you can just give the -DBUILD_STATIC=ON flag to make it work:

cmake -DBUILD_STATIC=ON ..

[NanoMichael]

I'd like to be able to build MicroTeX without Cairo and without any dependencies (apart from the dependencies required to use imgui, obviously).

It can be done via the cwapper now. You may want to check out the file lib/cwapper.h, and the wasm impl. In short, the cwrapper implements a "drawing command serializer" which converts drawing commands to a byte sequence via the interface microtex_getDrawingData. Once you take the "drawing data", you just deserialize it and translate it to drawing commands based on the graphics backend whatever you want without any dependencies.

[cesss]

This is great, @NanoMichael , just what I wanted!! One question, though: I see there's a serialized command for loading a font file... when my code reads this code from a byte sequence, what font format should it support, and on what directory should it look for the font file name it receives? This is the point I find hardest to understand in this moment. Would stb_truetype be adequate for parsing the font file that such command can specify?

[NanoMichael]

Do you mean the command code 3 (set font by family name)? MicroTeX doesn't need font files, the "font" concept is transparent to MicroTeX, actually a "font file" is just a key to your graphics backed to find your platform-specific font.

That's somewhat a little bit hard to understand :joy:, let me explain. In short, MicroTeX uses the "clm" (stands for cLaTeXMath, it is the original name of MicroTeX) data file which contains the font info, glyph metrics, math tables, and other significant information to layout math formulas, the "clm" data was generated by prebuilt/otf2clm.sh, and all these info was taken from the OTF font file.

Take res/xits/XITSMath-Regular.otf (its family name is "XITS Math") as an example, there're 2 conditions to consider:

• with glyph paths (you must compile with option -DGLYPH_RENDER_TYPE=0 which means draw glyphs use paths and typeface both, or -DGLYPH_RENDER_TYPE=1 which means draw glyphs use paths only), the generated "clm" data file will contain all the glyphs paths (that means the "clm" data file will be larger), and its filename will be XITSMath-Regular.clm2. Thus you don't need the font file anymore, the graphical paths will be used when drawing glyphs, and the drawGlyph (command code 9), setFont (command code 3), and setFontSize (command code 4) will never be used. The code below shows a minimal requirement:

// read "clm" data file to get its contents as a byte buffer
// assume the 'read_binary_data' reads the binary contents from a  file
unsigned long len;
const unsigned char* data = read_binary_data("your_clm_data_file_path", &len);
// initialize the context
microtex_init(len, data);
// ... use the context

• without glyph paths (compiled with option -DGLYPH_RENDER_TYPE=2 which means render glyphs use typeface only), the generated "clm" data file will NOT contain glyphs paths, and its filename will be XITSMath-Regular.clm1, then the font file XITSMath-Regular.otf is needed to draw glyphs. But if your computer has the XITS Math font installed and your graphics backend is able to find the font correctly, you don't need to load the font also.

// load 'XITSMath-Regular.otf' to your graphics backend
auto your_platform_specific_font = load_font("res/xits/XITSMath-Regular.otf");
// or your graphics backend can find the font by its name (probably with style and weight,
// for XITSMath-Regular.otf, the style is `Regular` and the weight is `Normal`)
auto your_platform_specific_font = load_font_from_family_name("XITS Math");
// in a word, you must have the font "XITS Math" loaded before using it,
// and is able to get the font by its family name

// read "clm" data file to get its contents as a byte buffer
// assume the 'read_binary_data' reads the binary contents from a  file
// the family name was read from the clm data
unsigned long len;
const unsigned char* data = read_binary_data("your_clm_data_file_path", &len);
// initialize the context
microtex_init(len, data);
// ... use the context

// impl the deserializer
void set_font(const char* family) {
  // set the font to your graphics backend
}

[cesss]

This is really great, @NanoMichael !! Please keep a copy of what you wrote in this reply because I think it can be very useful to be added to the documentation! I'll be using -DGLYPH_RENDER_TYPE=1 with a clm2 file, because that way I don't need to care about loading fonts, just draw the paths and fill them, which is what I want.

BTW, what clm2 font do you suggest me to use? Which is the one you chose for rendering the samples? (I don't have a preference in font style, I just want to choose the one you consider safer or more tested).

[sp1ritCS]

@cesss I did plan to include a modified version of this in the documentation I'm working on at #129 .

Fonts are typically a preference thing, however I'd go with Latin Modern (a port of the Computer Modern METAFONT to OTF), as it's the one the proper (La)TeX compiler uses by default (and I really like it :)). But others a probably fine too, as long as you don't pick Fira Math (that one actually looks hideous IMO). The samples are still from cLaTeXMath, which used glyphs from a plethora of different fonts intermixed, but I think it was mostly Computer Modern glyphs.

[cesss]

Understood, @sp1ritCS . Looking at their licenses (OFL and GFL), I'm not a lawyer, but from what I've read they don't seem to have copyleft effects in applications that embed them, so what I'm going to do is to convert all the clm2 files to source code arrays, so that my static build of MicroTeX is completely static even for the fonts, and I can try and compare fonts easily at runtime.

[NanoMichael]

it is currently a big WIP where a lot is not working on the moment. I will share it once I have something satisfactory.

@jokteur It is OK for a WIP PR request 😄, we may help you to find out some quick fixes in case you have misunderstood the API design, but if you don't want to share your code for some reason that's just fine, feel free to make PR or issues.

[jokteur]

it is currently a big WIP where a lot is not working on the moment. I will share it once I have something satisfactory.

@jokteur It is OK for a WIP PR request 😄, we may help you to find out some quick fixes in case you have misunderstood the API design, but if you don't want to share your code for some reason that's just fine, feel free to make PR or issues.

Because of limitations of ImGui (see #132, cannot use glyphId, only utf8), there is currently no way to have a satisfactory solution. I would need to hack my way around the freetype library and ImGui internals. Otherwise, I can directly draw in ImGui with paths, but performance is not good enough (ImGui likes rasterized triangles, not paths). For now, I use GLYPH_RENDER_TYPE=0 and draw directly into an opencv image (because of its non convex fillPath algorithm), and then display the image.

I guess, a satisfactory solution would be to have a renderer agnostic implementation (no Qt, no skia, no opencv, ...), where it takes the draw calls as an input and outputs a RGBA 8 bit per channel image which can then be used by any renderer. But it needs some serious coding, as fillPath algorithms and antialiasing are not trivial. Maybe I will come back and try to code this implementation, but not immediately.

[sp1ritCS]

I guess, a satisfactory solution would be to have a renderer agnostic implementation (no Qt, no skia, no opencv, ...), where it takes the draw calls as an input and outputs a RGBA 8 bit per channel image which can then be used by any renderer.

why not use Cairo's ImageSurface with ARGB32 format for that?

[cesss]

My current idea is to bake the tessellation of each glyph (I'm working with vector graphics, using tri-meshes, so I need to tessellate the glyphs paths). What I don't know is if MicroTeX can behave that way.

I mean, let's image we are going to display "5 + 5 + 3". What I'd need is that the sequence of rendering commands is as follows:

ISSUE BEZIER PATHS VERTICES FOR GLYPH "5" AND ASSIGN TO IT ID=1
ISSUE COMMANDS FOR MOVE/SCALE (for displaying "5" in the desired position)
ISSUE COMMAND FOR REUSING PATH WITH ID=1
ISSUE BEZIER PATHS VERTICES FOR GLYPH "+" AND ASSIGN TO IT ID=2
ISSUE COMMANDS FOR MOVE/SCALE (for displaying "+" in the desired position)
ISSUE COMMAND FOR REUSING PATH WITH ID=2
ISSUE COMMANDS FOR MOVE/SCALE (for displaying "5" in the desired position)
ISSUE COMMAND FOR REUSING PATH WITH ID=1
ISSUE COMMANDS FOR MOVE/SCALE (for displaying "+" in the desired position)
ISSUE COMMAND FOR REUSING PATH WITH ID=2
ISSUE BEZIER PATHS VERTICES FOR GLYPH "3" AND ASSIGN TO IT ID=3
ISSUE COMMANDS FOR MOVE/SCALE (for displaying "3" in the desired position)
ISSUE COMMAND FOR REUSING PATH WITH ID=3

In this way, my code would generate the triangle meshes for the repeated glyphs just once for each one, and it will later just reuse such meshes.

Is this already the behaviour of MicroTeX, or it doesn't behave like this?

[jokteur]

I guess, a satisfactory solution would be to have a renderer agnostic implementation (no Qt, no skia, no opencv, ...), where it takes the draw calls as an input and outputs a RGBA 8 bit per channel image which can then be used by any renderer.

why not use Cairo's ImageSurface with ARGB32 format for that?

It could be any format. I happened to use opencv because this is what was installed on my computer at the moment.

My current idea is to bake the tessellation of each glyph (I'm working with vector graphics, using tri-meshes, so I need to tessellate the glyphs paths). What I don't know is if MicroTeX can behave that way.

@cesss If you want to understand how microtex draws latex, I suggest you look at the different implementations graphics_xxx.cpp: https://github.com/NanoMichael/MicroTeX/tree/openmath/platform

[NanoMichael]

I guess, a satisfactory solution would be to have a renderer agnostic implementation (no Qt, no skia, no opencv, ...), where it takes the draw calls as an input and outputs a RGBA 8 bit per channel image which can then be used by any renderer.

@jokteur I'm not familiar with imGui, does it use FreeType to rasterize glyphs? Perhaps you can use FreeType to generate bitmaps from glyphs and pass them to imGui to draw.

See Managing Glyphs from the FreeType2 tutorial.

[jokteur]

I guess, a satisfactory solution would be to have a renderer agnostic implementation (no Qt, no skia, no opencv, ...), where it takes the draw calls as an input and outputs a RGBA 8 bit per channel image which can then be used by any renderer.

@jokteur I'm not familiar with imGui, does it use FreeType to rasterize glyphs? Perhaps you can use FreeType to generate bitmaps from glyphs and pass them to imGui to draw.

See Managing Glyphs from the FreeType2 tutorial.

It uses stb_truetype, but there is a freetype extension. It is just that behind the scenes, everything is in utf8, so I can't use glyphId. When ImGui encounters two glyphs with the same utf8 point, the second glyph gets ignored (see https://github.com/ocornut/imgui/issues/5647). I kind of know what I should do for ImGui, I was just trying to go fast and wanted to avoid touching the ImGui internals.

[sp1ritCS]

I guess, a satisfactory solution would be to have a renderer agnostic implementation (no Qt, no skia, no opencv, ...), where it takes the draw calls as an input and outputs a RGBA 8 bit per channel image which can then be used by any renderer.

why not use Cairo's ImageSurface with ARGB32 format for that?

It could be any format. I happened to use opencv because this is what was installed on my computer at the moment.

Well I just suggested ARGB32, because I assume it's almost the same as that RGBA 8 you wanted (just with the alpha byte shifted to the front). It's also what MicroTeX uses for colors, so if you want to go with simply letting cairo render the equation as memory bitmap, that would probably be the way to go.