Skip to content

Commit

Permalink
Fix windows dtype bug of neural search (#3182)
Browse files Browse the repository at this point in the history
* Fix windows dtype bug of neural search

* Fix windows dtype bug of neural search

Co-authored-by: ε΄ι«˜ε‡ <[email protected]>
  • Loading branch information
kztao and w5688414 authored Sep 7, 2022
1 parent 9b51a64 commit 12c605a
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions applications/neural_search/recall/simcse/inference.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,10 @@ def convert_example(example, tokenizer, max_seq_length=512, do_evalute=False):
max_seq_length=max_seq_length)

batchify_fn = lambda samples, fn=Tuple(
Pad(axis=0, pad_val=tokenizer.pad_token_id), # text_input
Pad(axis=0, pad_val=tokenizer.pad_token_type_id), # text_segment
Pad(axis=0, pad_val=tokenizer.pad_token_id, dtype="int64"
), # text_input
Pad(axis=0, pad_val=tokenizer.pad_token_type_id, dtype="int64"
), # text_segment
): [data for data in fn(samples)]

pretrained_model = AutoModel.from_pretrained(model_name_or_path)
Expand Down

0 comments on commit 12c605a

Please sign in to comment.