Code

Rejection Sampling on GSM8k#78

Code

Merged

AlexPiche merged 104 commits intomainServiceNow/TapeAgents:mainfrom llama70b_gsm8kServiceNow/TapeAgents:llama70b_gsm8kCopy head branch name to clipboardNov 20, 2024

+695-311

Commits

Commits on Oct 31, 2024

• run 70b

  AlexPiche

  committed
  0044902

  Copy full SHA for 0044902

• clean up debug code

  AlexPiche

  committed
  09f7fb2

  Copy full SHA for 09f7fb2

• conf_dir as str

  AlexPiche

  committed
  0537e02

  Copy full SHA for 0537e02

• chunked prefill vllm

  AlexPiche

  committed
  4b9407a

  Copy full SHA for 4b9407a

• iteration as str

  AlexPiche

  committed
  04279b0

  Copy full SHA for 04279b0

• no testing when test_every_n_iterations is -1

  AlexPiche

  committed
  537f9a4

  Copy full SHA for 537f9a4

• no testing with -1

  AlexPiche

  committed
  819a161

  Copy full SHA for 819a161

• print finetuning output

  AlexPiche

  committed
  b772d97

  Copy full SHA for b772d97

• rm seq length tokens

  AlexPiche

  committed
  4317fcb

  Copy full SHA for 4317fcb

• fp8 quant

  AlexPiche

  committed
  d781b74

  Copy full SHA for d781b74

• use deepspeed

  AlexPiche

  committed
  8657767

  Copy full SHA for 8657767

• better vllm logging

  AlexPiche

  committed
  d050cf4

  Copy full SHA for d050cf4

• deepspeed training

  AlexPiche

  committed
  6b0bc26

  Copy full SHA for 6b0bc26

• update accelerate version

  AlexPiche

  committed
  2a3f0ee

  Copy full SHA for 2a3f0ee

Commits on Nov 1, 2024

• negative rewards for too many steps

  AlexPiche

  committed
  1503884

  Copy full SHA for 1503884

• discounting

  AlexPiche

  committed
  5600f85

  Copy full SHA for 5600f85

Commits on Nov 2, 2024

• try to get lora working

  AlexPiche

  committed
  743a3e6

  Copy full SHA for 743a3e6

• step norm

  AlexPiche

  committed
  5e11b32

  Copy full SHA for 5e11b32

• penalize 20 steps tape

  AlexPiche

  committed
  b42a785

  Copy full SHA for b42a785

• step discount

  AlexPiche

  committed
  e8872c3

  Copy full SHA for e8872c3

Commits on Nov 3, 2024

• discoutn and max steps

  AlexPiche

  committed
  7be219b

  Copy full SHA for 7be219b

Commits on Nov 4, 2024

• fix discount typo

  AlexPiche

  committed
  f46c0ea

  Copy full SHA for f46c0ea

• merge 8b branch

  AlexPiche

  committed
  5d87a2a

  Copy full SHA for 5d87a2a

Commits on Nov 5, 2024

• use adafactor with accelerate

  AlexPiche

  committed
  4e79c5f

  Copy full SHA for 4e79c5f

• implicit kl

  AlexPiche

  committed
  1dfe78d

  Copy full SHA for 1dfe78d

• implicit kl

  AlexPiche

  committed
  7a2961c

  Copy full SHA for 7a2961c

• better variable names

  AlexPiche

  committed
  c737e72

  Copy full SHA for c737e72

• update docstring

  AlexPiche

  committed
  d9857c5

  Copy full SHA for d9857c5

• update docstring

  AlexPiche

  committed
  4273428

  Copy full SHA for 4273428

• get log probs worker

  AlexPiche

  committed
  efc38f5

  Copy full SHA for efc38f5

• WIP commit for browsing math tapes

  rizar

  committed
  2df67bd

  Copy full SHA for 2df67bd

• samples -> problems

  rizar

  committed
  a2427fe

  Copy full SHA for a2427fe

• dataset stats logging

  AlexPiche

  committed
  e805a33

  Copy full SHA for e805a33

• refactoring of the rl code

  AlexPiche

  committed
  575df66

  Copy full SHA for 575df66

• conf/accelerate

  AlexPiche

  committed
  2e332a7

  Copy full SHA for 2e332a7

• clean up conf

  AlexPiche

  committed
  540e078

  Copy full SHA for 540e078

• clean up conf

  AlexPiche

  committed
  2d6ff26

  Copy full SHA for 2d6ff26

• better default hps

  AlexPiche

  committed
  e240688

  Copy full SHA for e240688

• clean up

  AlexPiche

  committed
  db225e0

  Copy full SHA for db225e0

• bigger batch size

  AlexPiche

  committed
  fc204eb

  Copy full SHA for fc204eb

• small refactor

  AlexPiche

  committed
  7679984

  Copy full SHA for 7679984

• dataset stats on main process only

  AlexPiche

  committed
  193e94b

  Copy full SHA for 193e94b

Commits on Nov 6, 2024

• fix dataset stats when training is resumed

  AlexPiche

  committed
  4bdf653

  Copy full SHA for 4bdf653

Commits on Nov 7, 2024

• try RS

  AlexPiche

  committed
  5ac8f14

  Copy full SHA for 5ac8f14

• typo in config

  AlexPiche

  committed
  ebab454

  Copy full SHA for ebab454

• rs

  AlexPiche

  committed
  d09d7ff

  Copy full SHA for d09d7ff

Commits on Nov 8, 2024

• read 100 llm calls at a time

  rizar

  committed
  1696a51

  Copy full SHA for 1696a51

• Merge branch 'llama70b_gsm8k' into browse_math_tapes

  rizar

  committed
  43b7636

  Copy full SHA for 43b7636

• changeable port for tape_browser

  rizar

  committed
  127c7f3

  Copy full SHA for 127c7f3

• tape browser launch script and json gathering script

  rizar

  committed
  631ebb0

  Copy full SHA for 631ebb0

• better way to save tapes

  rizar

  committed
  961a429

  Copy full SHA for 961a429

• cute little renaming

  rizar

  committed
  98656ea

  Copy full SHA for 98656ea

Commits on Nov 12, 2024

• Merge pull request #92 from ServiceNow/browse_math_tapes

  Show description for 3af4b2b
  AlexPiche

  authored
  3af4b2b

  Copy full SHA for 3af4b2b

• simple rl gsm8k

  AlexPiche

  committed
  056592c

  Copy full SHA for 056592c

Commits on Nov 13, 2024

• produce more tokens

  AlexPiche

  committed
  8605fe5

  Copy full SHA for 8605fe5

• icnrease max tokens

  AlexPiche

  committed
  7ecc197

  Copy full SHA for 7ecc197

• clena up agent architecture

  AlexPiche

  committed
  196c9b4

  Copy full SHA for 196c9b4

Commits on Nov 14, 2024

• simpler agent

  AlexPiche

  committed
  bb96ae8

  Copy full SHA for bb96ae8

• rm loop

  AlexPiche

  committed
  7499c8f

  Copy full SHA for 7499c8f

• clean string

  AlexPiche

  committed
  ac02145

  Copy full SHA for ac02145

• hack simple agent

  AlexPiche

  committed
  c241214

  Copy full SHA for c241214

• improve simple agent

  AlexPiche

  committed
  000ade0

  Copy full SHA for 000ade0

• clean up string

  AlexPiche

  committed
  9ce7a5b

  Copy full SHA for 9ce7a5b

• 2000 tokens

  AlexPiche

  committed
  64ac725

  Copy full SHA for 64ac725

• better logging

  AlexPiche

  committed
  3379173

  Copy full SHA for 3379173

• fix browser to read new tapes

  AlexPiche

  committed
  294d2bf

  Copy full SHA for 294d2bf

Commits on Nov 15, 2024

• assert its the right completion

  AlexPiche

  committed
  54dc1b7

  Copy full SHA for 54dc1b7

• rm assert

  AlexPiche

  committed
  73ad382

  Copy full SHA for 73ad382

• relu weights

  AlexPiche

  committed
  f8e24d8

  Copy full SHA for f8e24d8

• klcoef for reinforce

  AlexPiche

  committed
  a30dbc6

  Copy full SHA for a30dbc6

Commits on Nov 16, 2024

• revert to adam

  AlexPiche

  committed
  bbabd3e

  Copy full SHA for bbabd3e

• typo

  AlexPiche

  committed
  4cfa51e

  Copy full SHA for 4cfa51e

• rm bos token

  AlexPiche

  committed
  53ac342

  Copy full SHA for 53ac342

Commits on Nov 17, 2024

• clean up nodes

  AlexPiche

  committed
  7e903c0

  Copy full SHA for 7e903c0

Commits on Nov 18, 2024

• clipping loss to 0

  AlexPiche

  committed
  a795ac1

  Copy full SHA for a795ac1

• better handling of bos token

  AlexPiche

  committed
  4854cdc

  Copy full SHA for 4854cdc

• Merge pull request #101 from ServiceNow/simple_rl_gsm8k

  Show description for e0f8203
  AlexPiche

  authored
  e0f8203

  Copy full SHA for e0f8203

• rm BOS from tests

  AlexPiche

  committed
  f7b66bf

  Copy full SHA for f7b66bf

• Merge remote-tracking branch 'origin/simple_rl_gsm8k' into llama70b_gsm8k

  AlexPiche

  committed
  94f7c21

  Copy full SHA for 94f7c21

• fix typo

  AlexPiche

  committed
  8608726

  Copy full SHA for 8608726

• clean up example

  AlexPiche

  committed
  e97ed41

  Copy full SHA for e97ed41

• Merge remote-tracking branch 'origin/main' into llama70b_gsm8k

  AlexPiche

  committed
  8e19489

  Copy full SHA for 8e19489

• Merge remote-tracking branch 'origin/fix_test' into llama70b_gsm8k

  AlexPiche

  committed
  3b09425

  Copy full SHA for 3b09425

• rm run training in process

  AlexPiche

  committed
  1303602

  Copy full SHA for 1303602

• rm if rl start from base model

  AlexPiche

  committed
  73ee354

  Copy full SHA for 73ee354

Commits on Nov 19, 2024

• clean up

  AlexPiche

  committed
  35cb007

  Copy full SHA for 35cb007

• typo

  AlexPiche

  committed
  03174da

  Copy full SHA for 03174da

• clean up

  AlexPiche

  committed
  6ffc6b0

  Copy full SHA for 6ffc6b0

• better docs

  AlexPiche

  committed
  397e0b8

  Copy full SHA for 397e0b8

• clean up

  AlexPiche

  committed
  8b60ba6

  Copy full SHA for 8b60ba6

• Update tapeagents/observe.py

  Show description for 0200482
  

  AlexPiche

  and

  rizar

  authored
  0200482

  Copy full SHA for 0200482

• dima changes

  AlexPiche

  committed
  4c4e71a

  Copy full SHA for 4c4e71a

• clean up

  AlexPiche

  committed
  5618143

  Copy full SHA for 5618143

• update readme

  AlexPiche

  committed
  6268098

  Copy full SHA for 6268098

• reverse change

  AlexPiche

  committed
  9dd4cf7

  Copy full SHA for 9dd4cf7

• improve doc

  AlexPiche

  committed
  a59aaad

  Copy full SHA for a59aaad

• improve doc

  AlexPiche

  committed
  251ce78

  Copy full SHA for 251ce78

Commits on Nov 20, 2024

• rm debug code

  AlexPiche

  committed
  cbae73f

  Copy full SHA for cbae73f

• fix logging of max min dataset len

  AlexPiche

  committed
  94109b0

  Copy full SHA for 94109b0

• fix min seq length logging

  AlexPiche

  committed
  6ef2ea3

  Copy full SHA for 6ef2ea3

• fix naming of variables

  AlexPiche

  committed
  2d698cd

  Copy full SHA for 2d698cd

• typo

  AlexPiche

  committed
  18b1b02

  Copy full SHA for 18b1b02

• fix naming

  AlexPiche

  committed
  1291be2

  Copy full SHA for 1291be2

• hf dataset

  AlexPiche

  committed
  90b080f

  Copy full SHA for 90b080f