Code

[飞桨多模态大模型套件PaddleMIX开发大赛] rfc：过滤重复文本#908

Code

Open

Liyulingyue wants to merge 3 commits intoPaddlePaddle:developPaddlePaddle/PaddleMIX:developfrom Liyulingyue:patch-1Liyulingyue/PaddleMIX:patch-1Copy head branch name to clipboard

+133

Commits

Commits on Dec 23, 2024

• Create 基于Embedding过滤重复文本.md

  Liyulingyue

  authored
  39c1f94

  Copy full SHA for 39c1f94

• Update and rename 基于Embedding过滤重复文本.md to 过滤重复文本.md

  Liyulingyue

  authored
  bf5d914

  Copy full SHA for bf5d914

Commits on Jan 6, 2025

• Create preprocess.py

  Liyulingyue

  authored
  d8d72b7

  Copy full SHA for d8d72b7