TF: clearer model variable naming

[gante]

This issue is part of our Great Code Cleanup 2022. If you're interested in helping out, take a look at this thread, or come join us on Discord and talk with other contributors!

As introduced in #15908 and implemented in #15907, we now have a new @unpack_inputs decorator to unpack TensorFlow model call() arguments. In essence, if we apply the decorator, we can replace inputs["foo"] with foo, making the code for the layer/model much shorter and clearer.

This issue is a call for contributors, to implement the new decorator in the architectures below. If you wish to contribute, reply in this thread which architectures you'd like to take :)

Guide to contributing:

1. Ensure you've read our contributing guidelines 📜
2. Claim your architecture(s) in this thread (confirm no one is working on it) 🎯
3. Implement the changes as in TF: Unpack model inputs through a decorator  #15907 (see the diff on the model architectures for a few examples) 💪
   • The file you want to look at is in src/transformers/models/[model_name]/modeling_tf_[model_name].py
   • In functions that have an input_processing call, remove it and add the @unpack_inputs decorator
   • Replace any use of the inputs variable in those functions (e.g. inputs["foo"] -> foo)
4. Test your changes on the architecture with RUN_SLOW=1 py.test -vv tests/[model_name]/test_modeling_tf_[model_name].py 💻
5. Open the PR and tag me in it (don't forget to run make fixup before your final commit) 🎊
   • Note that some code is copied across our codebase. If you see a line like # Copied from transformers.models.bert..., this means that the code is copied from that source, and our scripts will automatically keep that in sync. If you see that, you should not edit the copied method! Instead, edit the original method it's copied from, and run make fixup to synchronize that across all the copies. Be sure you installed the development dependencies with pip install -e ".[dev]", as described in the contributor guidelines above, to ensure that the code quality tools in make fixup can run.

Models updated:

• (the templates)
• albert
• auto
• bart
• bert
• blenderbot
• blenderbot_small
• camembert
• clip
• convbert
• convnext
• ctrl
• deberta
• deberta_v2
• distilbert
• dpr
• electra
• encoder_decoder
• flaubert
• funnel
• gpt2
• hubert
• layoutlm
• led
• longformer
• lxmert
• marian
• mbart
• mobilebert
• mpnet
• mt5
• openai
• pegasus
• rag
• rembert
• roberta
• roformer
• speech_to_text
• t5
• tapas
• transfo_xl
• vision_encoder_decoder
• vit
• wave2vec2 👉 lgnore this one for now, looking into the issue @infinite-Joy raised
• xlm
• xlm_roberta
• xlnet

[osanseviero]

My favorite sport is deleting code 🦾

I would be interested in taking bart 🚀

[silvererudite]

Hi..would love to work on encoder_decoder

[cakiki]

I'd like to work on gpt2!

[johnnygreco]

I'll take longformer!

[bhavika]

I can take CLIP and OpenAI.

[robotjellyzone]

Hi,
I would like to work on vision_encoder_decoder & distilbert

[vumichien]

I'll take mobilebert and layoutlm

[Abdelrhman-Hosny]

I'll take mbart and t5

[johnnv1]

I'll work on vit

[tanmoyio]

I'll work on albert

[kamalkraj]

electra, tapas, deberta v1 and v2, pegasus, xlnet

[Rocketknight1]

@Abdelrhman-Hosny The user doing the t5 type annotations for TF said they're going to do the decorator too! I see you closed that PR already, but don't worry, it's being handled!

[vumichien]

I have just made PR for mpnet, rag, roformer

[johko]

I'll do convnext.

Also I found that the hubert and camembert (which inherits from roberta) models do not seem to have any input_processing call, so probably those can be marked as done

[bhavika]

I can also do led and lxmert.

[Tegzes]

I would like to work on xlm_roberta

[infinite-Joy]

working on wave2vec2

[louisowen6]

I would like to work on xlm.

[saglamutku]

working on blenderbot

[saglamutku]

working on blenderbot_small

[johko]

I'll do marian next

[elusenji]

I'll work on convbert.

[johko]

auto does not seem to have any input_processing methods. I'll take ctrl and don't know if any models after that are left (I could not find any)

[louisowen6]

Created PR for XLM #16247

[jmwoloso]

@robotjellyzone are you still intending to do DistilBERT? If not, i'd like to take that one, please.

[robotjellyzone]

@robotjellyzone are you still intending to do DistilBERT? If not, i'd like to take that one, please.

Hi @jmwoloso , actually I am done with the modifications in the distilbert but got busy in my job. Tomorrow will going to push the changes.
But one thing, if you need something then you can take that vision_encoder_decoder model that one I have not touched yet

[silvererudite]

Hi @gante I don't seem to see any inputs methods in the encoder_decoder model can you pls help me in detecting where it needs changing?

[silvererudite]

Hi @bhavika since you already worked on two models if you don't mind please I'd like to submit PR for lxmert

[gante]

@silvererudite it is possible that it doesn't have the function, I listed all TF models :)

[bhavika]

Hi @bhavika since you already worked on two models if you don't mind please I'd like to submit PR for lxmert

Go ahead @silvererudite

[silvererudite]

@silvererudite it is possible that it doesn't have the function, I listed all TF models :)

@gante I checked and there does not seem to be any inputs function in the model file, seems identical...I think encoder_decoder can be marked done then

[gante]

Everyone, thank you so much for contributing to this issue 🧡 Together, we deleted >4000 lines of code, making the TF codebase more readable and easier to maintain 💪

Tomorrow I will be updating any remaining architecture and closing this issue -- I'm assuming that claimed architectures will not be worked on this week. If you've claimed an architecture and are still interested in working on it, please let me know :)

Finally, if you've enjoyed this experience, there are other similar initiatives going on! Check issues with the Good First Issue (like this one), or venture out in the harder Good Second Issue-labeled issues 🚀