Skip to content

Commit

Permalink
use block_size as max_seq_len during profile_run
Browse files Browse the repository at this point in the history
  • Loading branch information
tae-su-kim committed Jul 19, 2024
1 parent 089eace commit fec2fd2
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion vllm/worker/habana_model_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -1034,7 +1034,7 @@ def profile_run(self) -> None:
num_layers = self.model_config.get_num_layers(self.parallel_config)
kv_caches = [None] * num_layers
max_batch_size = self.prompt_bs_bucket_cfg[-1]
max_seq_len = self.prompt_seq_bucket_cfg[-1]
max_seq_len = self.block_size

self.warmup_scenario(max_batch_size, max_seq_len, True, kv_caches)

Expand Down

0 comments on commit fec2fd2

Please sign in to comment.