switch/application : shared font workaround

[CaiMiao]

replaces #76
added a wrapper Application::addFontFallback() for adding font fallback, to make the code looks neat

to be discussed:
should setGetSystemLanguage() and setMakeLanguage() of libnx get warpped for developers' sake?

[CaiMiao]

japanese locale under applet mode and application mode, fallback should be okay

is the pink/red block intended or still bugs need to be fixed?

[natinusala]

Thanks for the PR!

The mechanism of font loading has been changed in the yoga rework. The rule of thumb is: I don't want to see any #ifdef __SWITCH__ in the code outside of platform selection.

Fonts are defined in a fully platform agnostic way in the font.hpp header. The fonts here must be fully platform agnostic. The Switch icons font is an icons font like any other (Material, Font Awesome, Glyphicons...).

Each platform font loader loads as much font as it can. At runtime, one platform is selected and its code is executed. Then, the platform agnostic code in Application takes all loaded fonts and sets up all the fallback.

So the proper approach would be to add the fonts in the header, as you did, and then load them in the Switch font loader. The glfw one should stay empty (but put a comment to say that the fonts are not supported). Then setup the fallback the same way it's done for Korean at the moment.

That requires abstracting out the system language (setGetSystemLanguage()) and right now it's not done. I wanted to do it in the near future as part of i18n abstraction. You can leave raw calls to setGetSystemLanguage() for now, I'll cleanup after you.

[natinusala]

I just pushed something that abstracts out the system locale - you can use Application::getLocale() to get the current locale as string, and then compare it with the LOCALE constants in the i18n header.

[CaiMiao]

Then setup the fallback the same way it's done for Korean at the moment.

I don't really get how you did fallback with korean atm, it is loaded but fallback wasn't set.
or should I set the fallback directly via switch_font ?

[natinusala]

Did I forget to do Korean fallback? My bad then

Then I would only load Korean or Chinese in the font loaders IF the language is set to Korean or Chinese, and then setup fallback if the Korean or Chinese font are loaded.

In the font loader: if the locale is Chinese then load Chinese font in the stash
In Application: if Chinese font is in the stash then setup fallback

And do the same for Korean if you want, while you are at it

[CaiMiao]

In Application: if Chinese font is in the stash then setup fallback

Now I get your point:
I should add extra ifs to current changes, in order to detect switch only entries in font stash (as I've changed all switch fonts to an unique name, then redirects regular entry to one of them)

am i right?

also i have concern on locale string. we can't use string in switch case so it may cause if panic, as i still need to use it to determine regular redirect destination atm

still, as a multilinguist, i want to have full fallback in non-applet mode, in order to display as much unicode chars as we can
(e.g. a pokemon manager homebrew managing pokemons from various region and get all names properly displayed)

[natinusala]

You should not consider the Chinese and Korean fonts as Switch exclusive fonts. They are Borealis fonts, of which the Switch provides one implementation (the shared fonts). PC (glfw) could provide an implementation too, by loading whatever TTF on the disk.

So you should not see if you are on Switch, you should see if the fonts are loaded (aka if the platform implementation loaded the font).

also i have concern on locale string. we can't use string in switch case so it may cause if panic, as i still need to use it to determine regular redirect destination atm

I don't think we can avoid if panic sadly, but do you have an example of what you have in mind with the if panic?

still, as a multilinguist, i want to have full fallback in non-applet mode, in order to display as much unicode chars as we can
(e.g. a pokemon manager homebrew managing pokemons from various region and get all names properly displayed)

This is why the regular font should always be loaded, Chinese font should be set as fallback, what do you think?

[CaiMiao]

This is why the regular font should always be loaded, Chinese font should be set as fallback, what do you think?

You might not aware, but for hanzi/kanji (chinese glyphs) they have numerous difference (different stroke in different part) in general for different regions.
If it's hard to imagine, check the website of Source Han Serif. It provided an example of the difference.
https://source.typekit.com/source-han-serif/

I also want to get the glyphs displaying correct, and that's why I insist to replace regular font stash with respective shared font (and have slightly different fallback order). I think i18n should be implement correctly and so I spent some effort on the library instead of just counting on others, even I'm not really a programmer.

[CaiMiao]

Now I detects if regular font is empty, and redirect it to other font stash on demand.
As current implement of switch_font does not load shared fonts to regular, it works okay for demo, and should not contain any console specific codes in application anymore.

But there's one more minor problem here:
see my application.cpp : L161,
I merged fallback chain building and locale respective regular redirection together.
If I seperate them, there will be a bunch more if and makes the code hard to read.
I tried my best though.

Let me know if any further modification is needed, but I'm afraid that may exceed my current skill...

[natinusala]

Oh so what you are saying is that the Chinese system uses a different latin font AND a Chinese font on top of that? So for English we need the standard font, and for Chinese we need the standard Chinese + the extented Chinese, and not the standard font?

I just want to fully understand the problem before continuing to comment haha

[CaiMiao]

for Chinese we need the standard Chinese + the extented Chinese, and not the standard font?

the fallback mechanism of HOS is more complicated,
but in simple terms, when you use S.Chinese in HOS, all shared font are loaded and S.Chinese font have the highest priority than others (it's fine to just assume everything like how I did atm).
also, S.Chinese and T.Chinese should fallback each other as they share the zh scope.

Chinese and Korean shared fonts complies corresponding national standards in respective regions, in terms of supported characters and scripting (the writing difference I mentioned previously).
Each standard only includes Chinese glyphs commonly used in respective region,
so when you compare them each other, you can see many of glyphs (and potentially some western latins) are missing.

But, yeah, when western languages and Japanese share a same font file, and many games only available in Japanese (in terms of metadata), fallback to standard is a must to cover the most i18n needs.

In short: you need both standard font and locale specific font, and let latter have highest priority.

[CaiMiao]

oh i get what you talk about, the extend chinese font just add extra chinese glyphs in order to complies with latest national standard

[natinusala]

Okay, I see. So what prevents you from implementing exactly that, while keeping the current design? Is it code legibility?

[CaiMiao]

I'm not very good at English, so I might get my demand unclear.

At the very beginning, I just wanted to solve the font loading problem for Switch correctly by

• load locale responsive shared font
  • load other shared font on demand (by detecting non applet mode) for ultimate universal i18n sake
• build a correct fallback chain
  • additionally, for both app mode

Personally I don't care of cross-platform at all, as 99% of that glyph missing problem happens on Switch, when you can just use one Source Han Sans font for other platform to solve it dirty but for good.

You later told me on yoga branch it should be all abstracted out, I totally agree with it as I don't want to break the intergration you (the author) intended. But... I am not that skilled, and I tried my best to make changes.

As of my latest commit, it works as I intended on Switch, and now I'm generally okay with it.
Next I just want to ask if you are okay with my changes or further modification/optimization is needed.
If something you required me to do later goes beyond my skill, I'm okay to do it if you are willing to teach me how to, otherwise I can not do anything about it...

[CaiMiao]

I also have another question in mind (maybe unrelated?):
When we load shared font in the library, do we share a same instance as HOS loaded, or we consume extra RAM to load it?

[natinusala]

I checked the new code, and the logic you implemented seems correct to me, and the approach is right too.

However is there a need to load all 4 Chinese and Korean fonts if the system locale is not a Chinese or Korean locale? I feel like it might take a lot of RAM for nothing on non-Chinese devices.

When we load shared font in the library, do we share a same instance as HOS loaded, or we consume extra RAM to load it?

The TTF itself is made available using shared memory, so the TTF itself is never copied anywhere. However loading the TTF takes more RAM and VRAM as it's transformed in some internal structure + a gfx texture.

[CaiMiao]

force pushed.

However is there a need to load all 4 Chinese and Korean fonts if the system locale is not a Chinese or Korean locale? I feel like it might take a lot of RAM for nothing on non-Chinese devices.

Since it checks for title takeover, I think it is acceptable though? (assuming it have enough RAM to waste)
At least HOS have been loaded them for nothing lol

If we must do it, we will need to check for certain languages, and another bunch of if shall be added,
or possibly we could get some related information directly from HOS via pl service,
but that definately goes beyond my poor skill.

(i guess GetSharedFontInOrderOfPriority have some sort of use but i just dont know how to use it)

[natinusala]

Since it checks for title takeover, I think it is acceptable though?

Just because RAM is available doesn't mean we can waste it indeed. I would rather use as least as possible, especially if it's not used.

If we must do it, we will need to check for certain languages, and another bunch of if shall be added,

Sure, I don't mind having more conditions in the code if it's to achieve proper design.

However GetSharedFontInOrderOfPriority is a really good find. The wiki here explains how to use it: https://switchbrew.org/wiki/Shared_Database_services#GetSharedFontInOrderOfPriority

Do you think we could replace our entire manual fallback logic to only use that function?

[CaiMiao]

Do you think we could replace our entire manual fallback logic to only use that function?

If I know how to I would do it, but I just don't know lol

Don't see any example use and totally headache with C/++
and I also guess it will load every shared font as well so some extra step is needed, and back to manual fallback eventually

[natinusala]

What is blocking you? The wiki says how to use the function, I can assist if you want

[CaiMiao]

I just can't understand how to use it without a real use example.
Frankly speaking I only learn the "copy & paste development™"...

and fyi I'm not available recently. if you can provide me example i will see what i can do when i have spare time

[natinusala]

Alright I'll have a look at it.

I have to warn you that the license of the library has been changed from GPLv3 to Apache 2.0, if you don't agree to that change please close the PR. Otherwise you can add your name to the authors file and copyright if you haven't already.

[CaiMiao]

has been changed from GPLv3 to Apache 2.0

I'm always ok with that

[DarkMatterCore]

Would it be feasible to implement some sort of compile-time flag to make Borealis load all fonts, even if the console region isn't Chinese or Korean, and let the developer decide? Let me elaborate.

I'm currently looking into implementing Borealis in a project where listing available titles is a must. Due to the region-free nature of the Switch, it could be possible to import Chinese/Korean games and use them with a unit from a different region.

Most of the time, these games only hold Chinese/Korean name entries as part of their application control data, which in turn wouldn't be properly displayed under consoles with a different locale because not all fonts would be loaded.

Defaulting to only load a subset of the shared fonts based on the configured locale seems alright, but letting the developer decide if all fonts should be loaded may also be benefitial for cases like this.

[CaiMiao]

got sometime trying to revive but GetSharedFontInOrderOfPriority is not implemented in libnx yet

[CaiMiao]

Necromancer approaching - I think i am close, but i have no clue about building fallback chain completely with GetSharedFontInOrderOfPriority at the moment though....

[CaiMiao]

I might have done my work, but if you need any sorta collab guildline compliance etc. , just leave comment...