Token Count Failure in Claude-3.5-Sonnet

[nyanyacoffee]

The token count for Claude-3.5-sonnet has failed, remaining at 0, while the GPT-4o model functions normally. The tiktoken_core has been properly installed and the health check shows no issues.

Additionally, during installation, build = "make tiktoken" was configured, and the compiled artifacts can be found. However, the health check still reports that tiktoken_core is missing. Why might this be?

[pidgeon777]

I also have the same problem (token count equal to 0 when using Claude 3.5 Sonnet).

Also checkhealth reports:

- WARNING tiktoken_core: missing, optional for accurate token counting. See README for installation instructions.

[jellydn]

Thanks all. Just tag @gptlang to follow up on this token count.