encoding before measuring byte length + extra try clause

[ATheorell]

Solution to #773 crash bug. Addressing the issue by making sure that length is measured in bytes in utf-8 encoding.
Secondly, added additional try/except clause to make sure that, in case the truncation still fails, the program doesn't crash.

[sweep-ai]

Apply Sweep Rules to your PR?

• Apply: Ensure all new functions and classes have very clear, concise and up-to-date docstrings. Take gpt_engineer/ai.py as a good example.