locale, factor, shuf: Example of deferred locale loading for clap

[ChrisDryden]

A few month ago I attempted the same thing but now I've got a much better understanding of the project to attempt it again. The change here is that there is a parse_deferred handler now that attempts to parse the clap arguments and it it fails it will reattempt it with the locale files loaded. This combined with changes in the default locale initialization to take the util_name at the beginning from the arguments allows us to not have to load the locale file until the translation is actually required.

This requires migrating each utility to this new format, but I already started the process for factor and shuf to start.

This change is also required for the ls/stat-free-symlink gnu test and it should make the valgrind tests much more stable since it runs twice as fast in local testing and less likely to timeout.

[ChrisDryden]

Whoops didn't realize, uudoc calls uuapp directly, should be a simple fix to swap the names of the two uuapp functions and it should work backwards compatible. Just waiting to see the perf numbers and the gnu test results

[github-actions]

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/symlink (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/misc/io-errors is no longer failing!
Congrats! The gnu test tests/pr/bounded-memory is no longer failing!
Note: The gnu test tests/printf/printf-surprise is now being skipped but was previously passing.
Congrats! The gnu test tests/unexpand/bounded-memory is now passing!

[codspeed-hq]

Merging this PR will degrade performance by 4.94%

⚡ 122 improved benchmarks
❌ 4 regressed benchmarks
✅ 158 untouched benchmarks
⏩ 38 skipped benchmarks¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	wc_bytes_synthetic[500]	194.7 µs	86.9 µs	×2.2
⚡	Simulation	wc_chars_large_line_count[100000]	1,023 µs	912.4 µs	+12.12%
⚡	Simulation	wc_lines_variable_length[(50, 500)]	3.5 ms	3.3 ms	+3.19%
⚡	Simulation	wc_lines_extreme_line_lengths[(100000, 200)]	1.6 ms	1.5 ms	+6.32%
⚡	Simulation	wc_lines_large_line_count[500000]	2.9 ms	2.8 ms	+3.94%
⚡	Simulation	b64_decode_synthetic	171.7 µs	60.1 µs	×2.9
⚡	Simulation	b64_decode_ignore_garbage_synthetic	168.7 µs	62 µs	×2.7
⚡	Simulation	b64_encode_synthetic	167.7 µs	58.6 µs	×2.9
⚡	Simulation	cksum_blake3	216.9 µs	113.3 µs	+91.43%
❌	Simulation	sort_ascii_c_locale	18.2 ms	18.8 ms	-3.16%
⚡	Simulation	sort_ascii_utf8_locale	18 ms	17.2 ms	+4.47%
⚡	Simulation	sort_long_line[160000]	1.4 ms	1.3 ms	+6.8%
⚡	Memory	b64_decode_synthetic	46.2 KB	12.1 KB	×3.8
⚡	Memory	b64_encode_synthetic	46.2 KB	8.6 KB	×5.4
⚡	Memory	b64_decode_ignore_garbage_synthetic	46.2 KB	12.1 KB	×3.8
⚡	Memory	file_tz_abbreviations	333.4 KB	288.4 KB	+15.59%
⚡	Memory	complex_relative_date	46.1 KB	16.9 KB	×2.7
⚡	Memory	single_date_now	46.1 KB	15.6 KB	×3
⚡	Memory	file_custom_format	46.2 KB	20.1 KB	×2.3
⚡	Memory	file_iso_dates	46.2 KB	19.5 KB	×2.4
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

---

_{Comparing ChrisDryden:deferred-locale-loading (7e70533) with main (dec633c)}

Footnotes

1. 38 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[ChrisDryden]

This should also unblock us from doing translations in all of the shared libraries because there will no longer be the performance penalty hit from adding a bunch of translations into the shared locale file

[github-actions]

GNU testsuite comparison:

Skip an intermittent issue tests/tail/symlink (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/factor/t33 is no longer failing!
Note: The gnu test tests/cut/bounded-memory is now being skipped but was previously passing.
Note: The gnu test tests/tail/tail-n0f is now being skipped but was previously passing.
Congrats! The gnu test tests/printf/printf-surprise is now passing!

[sylvestre]

maybe remove _util_name if we don't use it anymore?

[sylvestre]

i wonder if we follow the same standard everywhere ?

[lordeji]

Hi ! I've also tried to work a bit on the localization system and your defer implementation is really game changer.

I just have a few remarks :

• I've observed that each uupackage have some differences in their localization system. Some manually set help and version some don't. Some use uucore::clap_localization::configure_localized_command while others manually set .help_template(uucore::localized_help_template(uucore::util_name())).
  This system would need to reformat all uupackages and if you need help I can work with you for this tedious task.

• You're doing a no-op when no localization ID is found but I think that it's too much forgettable. Maybe forcing the existence of every arg ID, even when it's an empty string, would lead to an easier way to convert flt files and also making developpers know how they should format the flt file and "enforce" this without relying on documentation that people won't really read. Seems more draconian but easier on the long run.

• Why using crate::locale::get_message instead of translate! ?