Rewarding translators

[nomennescio]

In Codewars the author of a Kata can get awarded by users for their experience with the Kata, by upvotes or downvotes.
Translators are currently not rewarded. I request the introduction of a similar feature to let users award translators too.

Translating a Kata requires quite some effort, first by writing a solution in a language of choice, but also to write all testcode.
Typically the quality of translation is not consistent between languages for the same katas, which shows there are "good" and "bad" translations. The user's experience of solving the kata is tightly coupled to the (translated) language he solves it in. By rewarding (or punishing) the author only, the role of the quality of the translation gets fully attributed to the Kata author only, although the translator plays a significant role too.

Introducing such a feature requires some careful thoughts and a matching design. I will elaborate.

Note that it might be helpful to think of author and translator not as two different persons, but to discern between two concepts:

1. quality of the kata experience: how well was the description, the challenge, the abstract concept, the story etc.?
2. the quality of the implementation of the language of choice: how well where the tests written, could the sample code be easily studied to understand detailed requirements, how well were language specific details explained, how well were errors detected, how relevant were the test vectors?

In such a way, the author of the kata is both responsible for 1 and for 2 in language of the first implementation. He might also translate to more languages.

Incentives both for "authors"/1 and "translators"/2 act as a stimulant to give good katas to users. Having a fair distribution of incentives between authors and translators will stimulate more translations and better translations of katas, which will improve the overall Codewars experience for users. However, how to determine what incentives scheme will result in such a better experience, while avoiding negative effects of such a scheme that might deter translators to produce more and better translations?

A careful consideration should be made how different incentive schemes impact . Also, how are users going to show their appreciation, and how can they tell how much they like the kata idea ("author"/1) and how much they like the kata implementation ("translator"/2) ? Should that be two concepts they can vote on? Or a single combined one?
If a combined one, a more sophisticated mechanism could be used to "calculate" which part of could be attributed to the "author" and which part to the "translator" e.g. by comparing votes in different languages of the same kata and see if there are significant differences which might show difference in quality of translations.

Also next to distribution of rewards, how will a "punishment" impact the overall amount and quality of translations?

And last I would also like to mention the role of translation approvals, and incentives to help that (e.g. #1518), which is somewhat related to the grander scheme of how incentives can help improve the overall Codewars experience

[Kacarott]

Aside from being able to reward/punish translators, I think that reworking the post-completion survey slightly could have other nice benefits too. Consider the following mockup example below. This essentially ranks the points 1. and 2. as nomen mentioned in his post, and would give us a good insight into what kata people are enjoying/disliking, and why. This would make it much easier for maintainers to identify and fix problems, as well as being able to identify authors/translators who are able to consistently make high quality translations, even if the kata they are translating are considered bad/boring to many.

Another option might be to leave the current "Are you satisfied", but add an additional (optional) "Give detailed feedback" drop down menu which could ask for ratings on every specific part (description, concept, task, code set up, sample tests, full tests) as well as perhaps a spot for written feedback that would be visible to the author/translator responsible.

[hobovsky]

As far as I can see, votes can be cast only on Solutions tab? If so, then context of a language is already there, and the "Javascript" part of your mock-up would not have to be a dropdown?

[Kacarott]

? No votes can be cast from the discourse page too. Notice that in my mock-screenshot, I am on the discourse page (where I messed around with the html elements)

[hobovsky]

Uh, right, I was probably blind when I was checking.

[hobovsky]

I generally like the idea, one potential problem I see are not transparent, not always intuitive, and probably not the most fortunate rules of "being a translator".
Currently, when approving translations, the author of a root ancestor (i.e. the first published version) of an approved fork gets a reward. This can be considered not always deserved, because initial versions of translations can be sometimes bad, and need to be redone anyway. Awarding an author of a most recent fork also does not seem the best option, because this could allow hijacking ownership of translations with minor edits (especially by menders who'd self-approve forks with fixes). Maybe awarding an author of a first approved fork would be more fair?

[Blind4Basics]

it's more and less fair at the same time. It's actually the usual problem.

• a bad translator shit post a language
• someone fixes the language
• the bad translator gets the points?

The opposite strategy also has its problem, because currently, a "not satisfied" vote actually make the author lose honor. So what about menders fixing tiny issues, but not the whole thing? They shouldn't lose points for this.

So I guess your suggestion is "safer", rather than "more fair".
(not sure there is an actual good solution to this problem)

[Kacarott]

I think the idea was more like:

• If the translation is almost good, then the mender can fix a tiny issue, and approve. In this case, the original translator gets the credit.
• If the translation needs a lot of work to fix, then it can be rejected, and a new fork can be created to "redo" it. In this case, the author of the new fork gets credit.

This way, people who are fixing translations will only get credit (or lose points) if they are prepared to rewrite the whole thing properly. Presumably they will only be doing this if the original translator doesnt seem to have an interest in fixing their translation.

[Blind4Basics]

but how do you technically differentiate the two kind of forks? they both go through the same thing, no?

[Kacarott]

The difference would be whether the parent fork was outdated, or rejected (I think). I not remembering how exactly it works to fork a rejected translation though...

[hobovsky]

If the translation needs a lot of work to fix, then it can be rejected, and a new fork can be created to "redo" it. In this case, the author of the new fork gets credit.

Except our hoarder mentality prevents us from doing this :) But yes, I agree that the practice of rejecting translations should be more common. There was a proposal of providing an easy access/overview to rejected translations of the same language in kumite editor or somewhere (edit: it's #2796).

[nomennescio]

Basically this boils down to when should a new translation replace an old translation, or when should it considered to be just minor fixes. As soon as a translation is official, all rewards from then on should go to the current translation. In the end that means that the translation approval should somehow evaluate how much someone contributed to a translation, and decide how much of the original translator still deserves credit.

One way to handle this, is to distribute rewards to contributers also; as I have argued elsewhere for translation approvers. This might be as simple as giving a fixed amount of points to small fixes, similar to how simple katas are valued. One way to discern could be in the approval process, a more automated process could be in calculating how much the contribution did change the code.

But this is extending upon the original idea, which is rewarding translators next to rewarding authors.

[ejini6969]

Despite using the rejection of parent fork as a criteria, it still brings issues especially when menders make tiny fixes to kata like adding test / solution import in sample tests of Python. Who should be responsible of the main idea of the translation then?

I like the core idea, but it is not so feasible from current system's POV, even though all translators or language "forkers" are given respective panel to vote for satisfaction, it can lead to a huge fuzz with a lot of beginners potentially trolling votes, specifically those that do not understand how the test framework and assertions is setup.

Or another occasion --> "owh, translator does not include specific fixed tests like empty strings or invalid data types, DOWNVOTE option 2!!! But I like the kata's idea and workings, so UPVOTE option 1." whilst the description mentions that all inputs will be valid and it is the voter's fault of not reading description thoroughly. There will be lots of cases of targeting downvotes too for unreasonable justifications, specifically for those who translated a lot and who authored a lot as well (specifically G964).... Then it'll become an endless rabbit hole to maintain IMO!

[nomennescio]

I agree all options should be careful considered for its overall impact. Including having no downvotes to avoid trolling.