Custom language hooks

[Speiger]

o/

I know i asked for a while if we could allow for prefixes for language files, and it was said for the modloader it wouldn't make much of sense. Which is understandable.

What i think is a more reasonable request is if there was a proper way to hook into the language loading system, so when a language gets loaded that you could inject at least your own key/value pairs.

This is mainly a QoL feature where you could implement special loaders that could handle permutations (looking at #1264) a lot easier, or allow to separate large files from each others. (Wikis are the first thing that come to mind)

I am not asking for you to provide these systems to handle the cases.
All I am asking is, provide a global hook where appending of translations can be done, not even overriding existing translations.
So devs can include their own solutions is wanted.

[HenryLoenwind]

I'm not quite sure I understand all the use cases this would enable. Is it just about being able to provide multiple lang files for one mod?

At the moment, the code tries to load exactly one String.format("lang/%s.json", langName) file for every clientResources.getNamespaces() (i.e. known modid).

Would your suggestion add additional modids, make langName a list, fire an event to load arbitrary additional files, fire an event to inject additional lang key-value pairs, add a hook to preempt the lookup of keys, or add a hook to look up keys if they don't exist?

Edit: Or add a way to specify "if the langName has no file, try that langName" (e.g. " if there's no es-mx, try es-es, then es")?

[Speiger]

@HenryLoenwind
That is one of the cases. You could for example split your lang files if that is desirable, without applying it to all namespaces.
But it would also allow for custom lang formats, since all i am asking is a post process hook that allows you to do addLangEntry(String key, String value)

2 examples i can think of is.
First one would be allowing in code key chaining.

{
	"minecraft.item": {
		"dye": {
			"red": "Red Dye",
			"green": "Green Dye"
		}
	}
}

Which would result in: minecraft.item.dye.green and minecraft.item.dye.green

Second example would be that you could import translations from non json files too.
This is especially useful if you have entries that are multiple hundred/thousand words long and they contain formatting/newlines/etc in them. Which is horrible to edit in a single line json translation key. (Believe me i did that for a bit before writing a F3 reload hook that preprocesses the files and then allows langasset reloads to happen)

Ofc you could have preprocessor resourcepacks that basically does that automatically (because F3 support and all).
But that doesn't feel right IMO... Especially since you need to add fakeout modIds to have additional files present, which increases loadtimes for everything.

[HenryLoenwind]

Ok, I leave evaluating that injection to the maintainers. Just one thing; you can inject your own language handling with:

Language.inject(MyLangauge.wrap(Language.getInstance()));

in a ReloadListener. That way, you can do whatever you want. If this were a common thing for mods to do, it would result in a horrible wrapping chain, but it isn't, so I see no harm in doing so.

[lukebemish]

You can already split up your language files with ease -- it loads one per namespace, not one per mod ID. Just prefix/suffix your mod ID to make different namespaces, and boom, you've split up your language files.

[Speiger]

@lukebemish so how many new name spaces should i introduce? A few hundred? A few thousand?
With the secondary feature i use as an example i would have to introduce 1 namespace per file i want to load.
With my current preprocessor script which is dev only i have roughly 700 files. So 700 name spaces.... And next time that might be more than that as the system gets more advanced?
The only thing my mind just came up with after i wrote everything else was: Maybe add a preprocessor into the resource pack itself that just merges all files and does custom logic as the "lang file" gets loaded.... Buuuut that wouldn't account for custom resourcepacks provided by the user. Or cause a recursive loop.... Mixins it is then.

@HenryLoenwind eh sadly that wouldn't work well because forge actually has its own internal language loader included.
What i am doing atm is hooking into "ClientLanguages#loadfrom" function just before the return value and just load the registered prefixes for that specific mod.
But since that only covers ClientSided translations and the server also needs it I am also hooking into "LanguageHook#loadLanguage" doing the same there like the client side.

just doing 1 side is not enough sadly.
And trying to bypass the forge/neoforge system isn't as easily done as you think.

I am asking for a loading hook so i don't have to mixin into neo/forge internals to get things going ^^"

[lukebemish]

There's no downside to adding more namespaces. Splitting it up into multiple files within one namespace, and splitting it into multiple namespaces, are really equivalent.

[Speiger]

@lukebemish it has a impact on anything that does a wildcard scan (wildcard => any namespace) though depending on how extremely that is done the impact is low. I give you that.

Still doesn't solve the limitations of breaking compat with basically user generated resource packs not being supported.

[lukebemish]

Not sure how user generated resource packs are any different here or how that would cause any issues at all? All lang files -- regardless of namespace -- are merged together in resource pack order. The namespace is purely for organization within a pack.

[Speiger]

@lukebemish if i create a inMemory resource pack that converts for example 100 files into 1 en_us because i don't want to have values that are hundreds/thousands of characters long with text formatting included (newline/coloring/etc) so its easier to edit for anyone then i can not support that for user resource packs because they are only working for in memory resource packs.

In other words to make stuff work like you suggest it forces me to do folder spamming where i have hundreds of folders with hundreds of en_us.json files with only a single entry that is hundreds of characters long with text formatting etc in them.
Not solving anything.

Or to clarify at that point i can just use MyClass.class.getResourceAsStream() because these files won't work with user resource packs anyways and is 100% incompatible with it anyways.

What i am asking for is to allow loading data with the resource pack system still supported.

At this point i explained it to death.
What i am suggesting is:
Provide a hook where custom lang entries can be added to the language. Without forcing neo/forge to implement support for custom parsers.
This hook would allow developers to provide their own parsers if that is desired and doesn't force them to write mixin/asm hooks into neo/forges internal code to make it work.

[lukebemish]

You could also just have your in memory resource pack loader or whatever support multiple namespaces? I'm not seeing any issue you've mentioned that's not solvable with current systems

[Speiger]

@lukebemish ok let me put that out in a step by step action.

• 1: Create In Memory Resource pack that redirects the file "en_us.json" to "test_en_us.json" of the mods resources.
• 2: Create a ResourcePack.zip that has a "test_en_us.json" in it.
• 3: Load said resource pack using the vanilla "resourcepacks folder"
• 4: Realize that the resource packs "test_en_us.json" isn't working because the InMemory ResourcePack can't access it.

This is because the in memory resource pack only contains the mods resources.
(Because resource packs shouldn't load other resourcepacks)

This applies to any solution you suggest.

[lukebemish]

Why are you trying to redirect a single file? Either (a) make an in memory resource pack generated by existing resource packs, which is perfectly possible, (b) use a different reload listener instead of an in memory resource pack, or (c) if the processing is applying formatting, do that in code to the components instead.

[Speiger]

@lukebemish

• A: so i should make a resourcepack that contains all other resource packs (including mods resource packs because addons that already exist) to load a single file pattern i am searching for? love the overhead on language loading then :3
  Also the moment someone else does it you have a infinite loop since.... 2x all resources packs being iterated...
• B: Again, a Mixin would be better then because then i can hook into it directly and get a better performance/feature support out of it, at the cost of reliability. (Thats what i am doing right now, but i would like an official hook)
• C: Text formatting can also include "newLines" and "tabs". Ignoring that. With a wiki parser you can't just make for every single change a component. That's why mc supports in text formatting too... (That's the example with the thousands of letters per file)

Anyways. All i am asking is for an official hook to make "mixins/asm hooks" not required to expand support.
Without having to import ALL RESOURCE PACKS TWICE.

Anyways i explained it to death...
Either close the issue and say: "Not in scope" or make it a planned feature.

I am done arguing with you specifically.

[Speiger]

Fuck it.
I am done arguing this to death.
It is much easier/simpler to hook into their core and do changes i need then.
And if these changes include bugfixes so be it.

Just to classify the level of support i asked: public record LanguageLoadEvent(String lang, ResourceManager manager, BiConsumer<String, String> adder){}

Good luck.

[TelepathicGrunt]

if i create a inMemory resource pack that converts for example 100 files into 1 en_us because i don't want to have values that are hundreds/thousands of characters long with text formatting included (newline/coloring/etc) so its easier to edit for anyone then i can not support that for user resource packs because they are only working for in memory resource packs.

Is the crux of the problem here just the giant lang entries? If so, maybe those should not be in a lang file. Someone suggested that if you were trying to convert an entire massive wikipedia page into a single lang file entry, that is more of a design problem with the mod. Instead, it would be easier for everyone involved, (modder and players), if you do something like wiki/<lang code>/fileX.md and load the markdown file yourself and display that in game. With markdown format, it'll be so much easier to do all the formatting and handling of massive amount of text. The vanilla lang files are much better suited for small amounts of text.

[Speiger]

@TelepathicGrunt Its not only long lang entires, but multiple reasons.
If you have thousands of entries, or want to make it more organized in general you can't because the lang loader doesn't allow that by default. Asking for supporting custom lang loaders is a bit to much.
Or if you want to generate permutations automatically from a few key elements and a permutation generator or have different parser format (see my dye example) or create a have prefixes in your lang files that get merged upon load without causing a deadlock loop if not every resource pack doesn't have loop checks in them that does that.

About the wiki.
Assuming it is a markdown renderer i would agree. But that is not the case. At most you have something like <header> or <image:path:bounds> or <pageEnd> or <doublePageEnd> which is rather small. Making it a whole json system is overkill. And it is anyways less than 5% of the overall lang data itself, we only count wiki entries to inflate it. On top of that i have seen longer description entries compared to how much text we have per page on the sane ones at least. The average of the 90% was like 150 characters. Still reasonable for lang files.

[Speiger]

Or let's put it in plain terms:
Without asm/mixin hooks into neoforges core atm you can't do anything custom at all...
And if you try to do it with resourcepacks you risk in causing an incompat because with the lang system you want to support overrides like anything else.

What I am asking: Please provide a bit of flexibility because atm there is none...

You simply have to go the mc way or you don't have lang support at all...