Would you be open to using a tracking issue or project board for the font work (sample categories: draft, waiting on other PRs, ready for review, please review first)?

I think breaking everything up into small PRs is fantastic but I'm finding it a bit overwhelming to keep track of the order for reviewing things (and figure I'm not alone here) and how these PRs relate to each other.

Sure, I'll do that. I'm sorry for the mess, but this is basically me trying to remember all the patches or even just ideas I have accumulated over the years that went nowhere because they would thrash all the baseline images, and that I now try to present given that I see an opportunity with the FreeType upgrade. Also, some of them are not fully complete patches but things that I believe can be made complete with relatively little effort (for some value of "little effort"...) simply because I do not have the time now to finalize them but want to put them up for discussion as the underlying idea should be clear.

Edit: see the wiki; attn @QuLogic as well too.

I'm sorry for the mess, but

No apologies needed, I commend you for taking advantage of the freetype upgrade. And thanks for the wiki, though some reason can't link cleanly ☹️

link fixed

Edit: fraction bar rendering has been added too.

A few warnings from compiling locally:

[2/4] Compiling C++ object src/ft2font.cpython-313-x86_64-linux-gnu.so.p/ft2font_wrapper.cpp.o
../../src/ft2font_wrapper.cpp: In lambda function:
../../src/ft2font_wrapper.cpp:1785:22: warning: unused variable ‘error’ [-Wunused-variable]
 1785 |             if (auto error = FT_Load_Glyph(face, idx, static_cast<FT_Int32>(flags))) {
      |                      ^~~~~
../../src/ft2font_wrapper.cpp:1788:22: warning: unused variable ‘error’ [-Wunused-variable]
 1788 |             if (auto error = FT_Render_Glyph(face->glyph, FT_RENDER_MODE_NORMAL)) {
      |                      ^~~~~
../../src/ft2font_wrapper.cpp: In lambda function:
../../src/ft2font_wrapper.cpp:1804:26: warning: unused variable ‘error’ [-Wunused-variable]
 1804 |                 if (auto error = FT_Glyph_To_Bitmap(&g, FT_RENDER_MODE_NORMAL, &origin, 1)) {
      |                          ^~~~~

This is semi-intentional: if these checks fail (error > 0) then the error should really be raised using throw_ft_error, not with a plain raise as I currently do, but throw_ft_error is not visible from ft2font_wrapper.cpp so some code needs to be moved around, e.g. make throw_ft_error an inline function defined in ft2font.h, or move the new implementations into real C++ FT2Font methods in ft2font.cpp and add wrapper lambdas in ft2font_wrapper.cpp, or (would be my preferred choice) switch error checking to the FT_CHECK macro used in mplcairo (see mplcairo's macros.h).
I left the warning as is just to keep this as a side discussion point, but it can trivially be suppressed by not capturing the error value too.

On a side note, I'm in favour of this in general as removing the intermediate buffer will make emoji/colour glyphs easier since they won't need to go through it.

On the call, we were discussing whether it made sense to target all these font-related (and requiring full image rebuild) PRs to a separate feature branch, which would contain the final test image updates. Do you have any thoughts on retargetting this PR (instead of folding it in to #29816)?

On the call, we were discussing whether it made sense to target all these font-related (and requiring full image rebuild) PRs to a separate feature branch, which would contain the final test image updates. Do you have any thoughts on retargetting this PR (instead of folding it in to #29816)?

Sure, I don't really mind either way? Should the branch live on this repo and be ultimately pruned out, or on yours?

Fixed #30059 (comment) by using FT_CHECK (thus also needs #30123).

I think you may have dropped the rebase when doing that... I've rebased and fixed the conflict. However, I've not pushed the test images, because it appears there might be a bug with antialiasing disabled:

(this is test_text/antialiased.png)

test modified from #29551 to also include mathtext:

If I extend this test to include \frac{abc123}{def456}, then the fraction bar is jumpy as well, because in the angle=0 case, we draw the bars with a NumPy array at a rounded position.

This could be fixed by always using the non-0-angle implementation which draws a Path instead, so long as we also call gc1.set_snap(False). I'm not sure whether we want to do that? It of course causes the bars to instead be smeared across two pixels periodically instead, so I don't know if we want to optimize for that over better static images. Or perhaps we should just let the user decide by setting snap on the Text object (which may or may not need additional threading through as I don't know if it cared about snapping before)?

How do fonts handle dashes and underscores? Would that be a reasonable precedent behavior?

Controlling based on Artist.get_snap() sounds like a good idea (as you mention, manually inserting gc.set_snap(False) works); I pushed that (and checked that it works). (However, from a quick look, maybe the exact way rounding works in snapping is different? The fraction bar looks shifted down?)

Edit: Likely the proper way to rely on path snapping requires changing the fraction bar from being a filled rectangle (with zero-linewidth) to being a single line with fixed linewidth; still todo, and maybe a problem for the rare case of drawing vertical bars.

Fonts likely control that based on hinting (https://en.wikipedia.org/wiki/Font_hinting), which could also be the switch, but I thought having separate controls for fraction bars and for the rest of hinting would be better.

So I've experimented a little bit to try and see where the changes arise; I've added a note below of one that I think I see.

Using the following small script:

import itertools
import string
import sys

import matplotlib.pyplot as plt

fig = plt.figure()

for i, fontsize in enumerate([None, 20, 32]):
    for j, text in enumerate(itertools.batched(string.ascii_lowercase, 3)):
        fig.text(0.1 + 0.2 * i, 0.1 + 0.1 * j, ''.join(text), fontsize=fontsize)

fig.savefig(sys.argv[1] if len(sys.argv) > 1 else 'bar.png')

in RendererAgg.draw_text, the x/y are almost always integers (or 3e-14 close to one) so fractional positioning shouldn't matter much. But when I compare results with the text-overhaul branch, almost every string has a difference in only the first character vs the rest. Do you have any idea why it might only affect that one advance?

I figured that out. There's a bug in draw_glyphs_to_bitmap, which is fixed by

diff --git i/src/ft2font.cpp w/src/ft2font.cpp
index 8838f68ee5..96ccafe1b0 100644
--- i/src/ft2font.cpp
+++ w/src/ft2font.cpp
@@ -711,8 +711,8 @@ void FT2Font::draw_glyphs_to_bitmap(bool antialiased)
         // now, draw to our target surface (convert position)

         // bitmap left and top in pixel, string bbox in subpixel
-        FT_Int x = (FT_Int)(bitmap->left - (bbox.xMin * (1. / 64.)));
-        FT_Int y = (FT_Int)((bbox.yMax * (1. / 64.)) - bitmap->top + 1);
+        FT_Int x = (FT_Int)std::floor(bitmap->left - (bbox.xMin * (1. / 64.)));
+        FT_Int y = (FT_Int)std::floor((bbox.yMax * (1. / 64.)) - bitmap->top + 1);

         draw_bitmap(image, &bitmap->bitmap, x, y);
     }

The problem being that for the very first glyph only, bitmap->left - (bbox.xMin * (1. / 64.)) is negative and thus the cast to FT_Int rounds up, whereas for the others, the cast rounds down, so the first glyph ends up shifted one pixel to the right relative to the others. (I didn't actually check for y but a similar issue seems likely.)

I figured that out. There's a bug in draw_glyphs_to_bitmap, which is fixed by

OK, so then it's an additional fix here; it did seem better in several cases (e.g., for This, the Th did seem to hit each other before).

How do fonts handle dashes and underscores? Would that be a reasonable precedent behavior?

This depends on the internal hinting and FreeType settings. Depending on the hinter, it can align vertical and horizontal lines to the pixel grid (which is why scaling up or down text isn't always correct), but it could just as easily not do that if the font doesn't implement it.

I tested the positioning of the fraction bar a bit more:

import sys
import matplotlib.pyplot as plt

fig = plt.figure(figsize=(10, 2))
fig.suptitle(sys.argv[1])
for i in range(100):
    fig.text(0.01 * i, .1 + .001 * i, r'$\frac{A}{B}$', snap=False),
    fig.text(0.01 * i, .2 + .001 * i, r'$\frac{A}{B}$', snap=True),
plt.show()

Both text-overhaul and direct-render are clearly better than main. direct-render looks blurry when snapping is disabled, but that's intentional. However, it looks like direct-render also draws the fraction bar slightly too low, whereas text-overhaul draws it slightly too long to the right?

Comparing main with text-overhaul, the difference looks like the glyphs only; the bars stay in the same spot and are the same width. Comparing text-overhaul with direct-render, it looks like the bar is a bit narrower, but based on the not-snapped line, it's actually a partial pixel on the right side, which seems to have snapped to a whole one those lines. WRT the line vertical position, there seems to be some finagling with +/- 1 in the old code:

I forget, was there more to be done here, or just reviewing the images changes?

We still need to agree that text.get_snap() is the correct API to control fraction bar blurring (that seems reasonable enough to me), and maybe try to figure out whether there's any off-by-1 issue with the fraction bar positioning. Some help on that would be welcome.

I am 👍🏻 on using snapping for this.

#22852 is also related to this.

Throwing latex into this as well, I think it looks like our bar is a bit too low.