Token indices sequence length is longer than the specified maximum sequence length for this model (812 > 600)

[Kahdeg-15520487]

i got this error when running this on windows

my setup:
vga 3090,
openai api: text-generation-web-ui
model: https://huggingface.co/bartowski/sparsetral-16x7B-v2-SPIN_iter1-exl2 6.5 quant

`G:\lll\liva [venv]> python .\main.py --url http://localhost:5000/v1/ --model-id mistral:instruct --stt-model openai/whisper-base.en
User:
Using microphone: Microphone (Razer Seiren Mini)
G:\lll\liva\venv\lib\site-packages\transformers\models\whisper\modeling_whisper.py:697: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.)
attn_output = torch.nn.functional.scaled_dot_product_attention(
youu

LIVA:
Hello there! How can I brighten up your day? Would you like to hear a fascinating fact, or are you in need of suggestions for a movie or book? Let me know! Or, perhaps you'd like some help figuring out what to cook for dinner? Or just need a brief conversation in a different language? I'm up for it all! Please tell me what you need or care for a little chat?
User:
Using microphone: Microphone (Razer Seiren Mini)
What can you do??

LIVA:
I can gather and analyze information to provide solutions, make recommendations, and answer inquiries across a wide range of topics. I can perform tasks connected to high-level topic analysis, those that require extensive reading, web research, and complex clerical tasks, such as drafting emails, collecting internal and external links, writing English translations, etc.
Another important skill I have is the ability to organize knowledge in an easy-to-understand way, through various formats like text, graphics cards or even as a voice assistant.
While I don't have feelings I can certainly empathize and interact in a natural and emotional human-like way based on patterns in our conversations, input data and programmed interactions. I'm also constantly learning and adapting to provide better responses.

Token indices sequence length is longer than the specified maximum sequence length for this model (812 > 600). Running this sequence through the model will result in indexing errors
Traceback (most recent call last):
File "G:\lll\liva\main.py", line 79, in
main()
File "G:\lll\liva\main.py", line 75, in main
speech = audio_model.synthesize_audio(response)
File "G:\lll\liva\speech_generator.py", line 15, in synthesize_audio
speech = self.synthesiser(text,
File "G:\lll\liva\venv\lib\site-packages\transformers\pipelines\text_to_audio.py", line 182, in call
return super().call(text_inputs, **forward_params)
File "G:\lll\liva\venv\lib\site-packages\transformers\pipelines\base.py", line 1162, in call
return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)
File "G:\lll\liva\venv\lib\site-packages\transformers\pipelines\base.py", line 1169, in run_single
model_outputs = self.forward(model_inputs, **forward_params)
File "G:\lll\liva\venv\lib\site-packages\transformers\pipelines\base.py", line 1068, in forward
model_outputs = self._forward(model_inputs, **forward_params)
File "G:\lll\liva\venv\lib\site-packages\transformers\pipelines\text_to_audio.py", line 143, in _forward
output = self.model.generate(**model_inputs, **forward_params)
File "G:\lll\liva\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "G:\lll\liva\venv\lib\site-packages\transformers\models\speecht5\modeling_speecht5.py", line 2834, in generate
return _generate_speech(
File "G:\lll\liva\venv\lib\site-packages\transformers\models\speecht5\modeling_speecht5.py", line 2486, in _generate_speech
encoder_out = model.speecht5.encoder(
File "G:\lll\liva\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "G:\lll\liva\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "G:\lll\liva\venv\lib\site-packages\transformers\models\speecht5\modeling_speecht5.py", line 1447, in forward
hidden_states = self.prenet(input_values)
File "G:\lll\liva\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "G:\lll\liva\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "G:\lll\liva\venv\lib\site-packages\transformers\models\speecht5\modeling_speecht5.py", line 787, in forward
inputs_embeds = self.encode_positions(inputs_embeds)
File "G:\lll\liva\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "G:\lll\liva\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "G:\lll\liva\venv\lib\site-packages\transformers\models\speecht5\modeling_speecht5.py", line 417, in forward
emb = emb + self.alpha * self.pe[:, : emb.size(1)]
RuntimeError: The size of tensor a (812) must match the size of tensor b (600) at non-singleton dimension 1`

[LuciAkirami]

Hey @Kahdeg-15520487 , thanks for pointing this out. The text to speech model (speech5_tts) can only generate audio upto 600 tokens. The text passed to the speech generator is 812 tokens which is exceeding 600, hence the error. I plan on solving this by dividing the generated text into chunks if it exceeds 600 tokens and then generate separate audio files