Questions about the use of the model

[ScottishFold007]

Is the above code a direct run out of a model that can be used directly? Or does it need tuning or further training?Like

from transformers import pipeline
from transformers import AutoModelForMaskedLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("new")
model = AutoModelForMaskedLM.from_pretrained("new")


unmasker = pipeline("fill-mask",model=model,tokenizer=tokenizer,device = 0)
from pprint import pprint

pprint(unmasker(f"I come for {unmasker.tokenizer.mask_token} in last time！"))

Will the above application code come out with a good result? Based on your experience?

[bminixhofer]

Hi, thanks for your interest!

In general, WECHSEL just acts as good initialization, you will need further training to get good results.

You can try the zero-shot case, and it should be better than random, but I don't expect results will be usable for any real-world application.