Discussions

• 1 You must be logged in to vote

  🙏

  How would you use shard scalar weight using ZeRO Stage 3?

  ggoggam asked Jan 31, 2025 in Q&A · Unanswered

  

  

  0
• 4 You must be logged in to vote

  💬

  Some random russian guy got Triton 2.0.0 to build on windows

  PleezDeez started Jan 12, 2023 in General · Closed

  

  

  

  

  12
• 9 You must be logged in to vote

  💬

  Can't get Deepspeed to work on Ubuntu 22.04

  AlpsAficionado started May 12, 2023 in General · Closed

  

  

  

  

  3
• 1 You must be logged in to vote

  🙏

  Cann't pip install deepspeed==0.11.1

  nothing7744 asked Sep 14, 2024 in Q&A · Closed · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Problem about async_io is uncompatible

  niebowen666 asked Sep 21, 2024 in Q&A · Answered

  

  

  

  

  2
• 1 You must be logged in to vote

  🙏

  build_win.bat 报错

  songyu1980 asked Apr 15, 2023 in Q&A · Answered

  

  

  

  

  4
• 1 You must be logged in to vote

  🙏

  I want to use two GPUs by starting a process, one for model A and the other for model B. Is there a viable way?

  Wallejd asked Jan 7, 2025 in Q&A · Answered

  

  

  1
• 4 You must be logged in to vote

  💬

  DeepSpeed MPI error

  Sabiha1225 started Mar 16, 2024 in General

  

  

  1
• 1 You must be logged in to vote

  🙏

  Why buffers are not synchronized in zero3?

  kiradiso asked Dec 6, 2024 in Q&A · Unanswered

  

  

  3
• 1 You must be logged in to vote

  🙏

  Why ZeRO-offload parameter update after model backward? can do them pipeline?

  Zijie-Tian asked Apr 29, 2024 in Q&A · Answered

  

  

  

  

  3
• 6 You must be logged in to vote

  💬

  Invalidate trace cache @ step 13: expected module xx, but got module xxx

  mrluin started Aug 3, 2023 in General

  

  

  

  

  13
• 1 You must be logged in to vote

  🙏

  How to learn deepspeed?

  ming-shy asked Oct 30, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  DeepSpeed on 3Detr

  yxteo2 started Oct 28, 2024 in General · Closed

  

  

  1
• 1 You must be logged in to vote

  🙏

  How can I lock the global batch size?

  timpal0l asked Oct 23, 2024 in Q&A · Unanswered

  

  

  5
• 2 You must be logged in to vote

  🙏

  Deepspeed ZeRo3 Error 'RuntimeError: 'weight must be 2-D' Please help

  shing100 asked Oct 18, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Why DeepSpeed ZeRO paper says "reduce-scatter (or all-gather) has data movement of Ψ elements"?

  civat asked Oct 1, 2024 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  Tensor / Model parallelism with deepspeed

  ansar-ayan asked Sep 30, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Some questions about the running environment without GPU.

  niebowen666 asked Sep 14, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  deepspeed ignoring steps_per_print

  ansar-ayan asked Sep 11, 2024 in Q&A · Closed · Unanswered

  

  

  4
• 1 You must be logged in to vote

  🙏

  Saving checkpoints in deepspeed

  base-y asked May 1, 2022 in Q&A · Unanswered

  

  

  

  

  10
• 1 You must be logged in to vote

  🙏

  tensor parallel for customized model

  zzxslp asked Sep 2, 2024 in Q&A · Unanswered

  

  

  1
• 2 You must be logged in to vote

  🙏

  How to use InferenceEngineV2?

  Arashimu asked Aug 26, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  When using torch.no_grad(), will deepspeed *not* perform memory optimization on some parameters of no_grad?

  ojipadeson asked Jul 29, 2024 in Q&A · Unanswered

  

  

  1
• 5 You must be logged in to vote

  🙏

  Global_steps taking too much space

  pratikkumar018 asked Sep 12, 2023 in Q&A · Answered

  

  

  

  

  5
• 4 You must be logged in to vote

  💬

  Is deepspeed zero stage 3 MP or PP. Same as FSDP?

  andrasiani started May 6, 2023 in General

  

  

  

  

  6