[Text Generation] Turn off the (currently) inefficient external KV cache logic when internal KV cache management enabled

[dbogunowicz]

Having an external KV cache loop running in parallel to the internal KV cache logic (if enabled) does not make much sense.

1. It slows down our most efficient pipeline mode
2. It could be potentially used for testing. But as of now, the cache values returned by the internal kv cache are nonsensical, so we do not have ground truth to compare the external cache with.

This PR has been tested using our internal (modest) LLM testing harness.

In general, this PR roughly accelerates inference by 25%

import time
from deepsparse import Pipeline

opt = Pipeline.create(task="opt",
                      model_path="/home/ubuntu/damian/sparseml/deployment_opt",
                      max_generated_tokens = 512,
                      prompt_processing_sequence_length = 3,
                      use_deepsparse_cache =False, # or True
)
counter = 0
for n in range(100):
    start = time.time()
    output = opt(sequences=prompts[0])
    counter += (time.time() - start)
print(prompts[0] + output.sequences[0])
print(f"Time taken: {counter/n} seconds")

Time taken: 21.29 seconds # use_deepsparse_case = False

Time taken: 15.99 seconds # use_deepsparse_case = True

Edit: confirmed by a more exhaustive investigation

[rahul-tuli]

LGTM!

The base branch was changed.

[rahul-tuli]

(n+1)st time is the charm 🚀