【master】fix paddlenlp (#1463)

* Revert "delete --privileged,test=document_fix (#1052)" This reverts commit f177e92. * add docker home path, test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * add seg case,update seg repo,test=document_fix * test add seg case,test=document_fix * test_1104 * merge upstream master * Revert "merge upstream master" This reverts commit a25523b. * fix mylog ,test=document_fix * delete models,test=document_fix * delete models,test=document_fix * update nlp repo,test=document_fix * update clas repo * update clas repo,test=document_fix * update clas repo,test=document_fix * delete senta,test=document_fix * delete senta,test=document_fix * change http_proxy,test=document_fix * change http_proxy,test=document_fix * change delete build paddle,test=document_fix * Revert "change delete build paddle,test=document_fix" This reverts commit 53965e7. * add script to test phi op (#1410) * add script to test phi op * add script to test phi op * add script to test phi op * add script to test phi op * add script to test phi op * add script to test phi op test=document_fix Update the map of directory name to apis of conv. (#1424) * Update the map of directory name to apis of conv. test=document_fix * Update tril_triu. test=document_fix Delete the v2 suffix, test=document_fix (#1425) Add a argument to allow profile from start for debuging purpose. (#1423) delete senta,test=document_fix add mod op benchmark, test=document_fix (#1427) delete senta,test=document_fix 【master】下线动态图senta wavenet (#1426) * Revert "delete --privileged,test=document_fix (#1052)" This reverts commit f177e92. * add docker home path, test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * add seg case,update seg repo,test=document_fix * test add seg case,test=document_fix * test_1104 * merge upstream master * Revert "merge upstream master" This reverts commit a25523b. * fix mylog ,test=document_fix * delete models,test=document_fix * delete models,test=document_fix * update nlp repo,test=document_fix * update clas repo * update clas repo,test=document_fix * update clas repo,test=document_fix * delete senta,test=document_fix * delete senta,test=document_fix change http_proxy,test=document_fix change http_proxy,test=document_fix change delete build paddle,test=document_fix Revert "change delete build paddle,test=document_fix" This reverts commit 53965e7. 【master】编包换代理 (#1428) * Revert "delete --privileged,test=document_fix (#1052)" This reverts commit f177e92. * add docker home path, test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * add seg case,update seg repo,test=document_fix * test add seg case,test=document_fix * test_1104 * merge upstream master * Revert "merge upstream master" This reverts commit a25523b. * fix mylog ,test=document_fix * delete models,test=document_fix * delete models,test=document_fix * update nlp repo,test=document_fix * update clas repo * update clas repo,test=document_fix * update clas repo,test=document_fix * delete senta,test=document_fix * delete senta,test=document_fix * change http_proxy,test=document_fix * change http_proxy,test=document_fix 【master】删除编包操作，使用跟PDC任务中一样的包 (#1429) * Revert "delete --privileged,test=document_fix (#1052)" This reverts commit f177e92. * add docker home path, test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * test add seg case,test=document_fix * add seg case,update seg repo,test=document_fix * test add seg case,test=document_fix * test_1104 * merge upstream master * Revert "merge upstream master" This reverts commit a25523b. * fix mylog ,test=document_fix * delete models,test=document_fix * delete models,test=document_fix * update nlp repo,test=document_fix * update clas repo * update clas repo,test=document_fix * update clas repo,test=document_fix * delete senta,test=document_fix * delete senta,test=document_fix * change http_proxy,test=document_fix * change http_proxy,test=document_fix * change delete build paddle,test=document_fix * fix paddle version ,test=document_fix * unset http_proxy ,test=document_fix * change for build paddle ,test=document_fix * change for build paddle ,test=document_fix * change for build paddle ,test=document_fix * add FLAGS_enable_eager_mode ,test=document_fix * delete FLAGS_enable_eager_mode ,test=document_fix * cancel build paddle,test=document_fix * test new dynamic_graph,test=document_fix * test old dynamic_graph,test=document_fix * test old dynamic_graph,test=document_fix * test old dynamic_graph,test=document_fix * test old dynamic_graph,test=document_fix * test 2.3,test=document_fix * test 2.3,test=document_fix * test 2.3,test=document_fix * update model repo ,test=document_fix * update fix clas amp_fp16 ,test=document_fix * update fix clas amp_fp16 ,test=document_fix * update fix clas ,test=document_fix * update fix clas ,test=document_fix * fix seg ,test=document_fix * update dynamic_to_static_models.sh ,test=document_fix * fix nlp,test=document_fix * test new dygraph,test=document_fix * test new dygraph,test=document_fix * test old dygraph,test=document_fix * fix video,test=document_fix * fix dali,test=document_fix * fix dali,test=document_fix * test add dy_mask_rcnn_fpn ,test=document_fix * fix nlp,test=document_fix * fix nlp,test=document_fix * fix nlp,test=document_fix Co-authored-by: Divano <[email protected]>
PaddlePaddle · Jun 2, 2022 · afc85e0 · afc85e0
1 parent b31c0f2
commit afc85e0
Show file tree

Hide file tree

Showing 3 changed files with 34 additions and 7 deletions.
diff --git a/scripts/dynamic_graph_models.sh b/scripts/dynamic_graph_models.sh
@@ -142,6 +142,10 @@ dy_bert(){
     cd ${cur_model_path}
     pip install -r requirements.txt
     pip install h5py
+    pip uninstall -y paddlenlp
+    pip install attrdict
+    python setup.py install
+
     cur_model_path=${BENCHMARK_ROOT}/PaddleNLP/examples/language_model/bert/
     cd ${cur_model_path}
     ln -s ${data_path}/Bert/hdf5_lower_case_1_seq_len_512_max_pred_80_masked_lm_prob_0.15_random_seed_12345_dupe_factor_5/wikicorpus_en_seqlen512 ${cur_model_path}/wikicorpus_en_seqlen512 ./data
@@ -240,8 +244,12 @@ dy_ptb_medium(){
 # transformer
 dy_transformer(){
     echo "###########pip install paddlenlp"
-    pip install paddlenlp
+    cd ${BENCHMARK_ROOT}/PaddleNLP
+    pip install -r requirements.txt
+    pip install h5py
+    pip uninstall -y paddlenlp
     pip install attrdict
+    python setup.py install
     cur_model_path=${BENCHMARK_ROOT}/PaddleNLP/examples/machine_translation/transformer
     cd ${cur_model_path}
     # prepare data
@@ -595,14 +603,18 @@ dy_resnet(){
 
 # lac
 dy_lac(){
+    cd ${BENCHMARK_ROOT}/PaddleNLP
+    pip install -r requirements.txt
+    pip uninstall -y paddlenlp
+    pip install attrdict
+    python setup.py install
     cur_model_path=${BENCHMARK_ROOT}/PaddleNLP/examples/lexical_analysis
     cd ${cur_model_path}
 
     # Prepare data
     ln -s ${data_path}/dygraph_data/lac/lexical_analysis_dataset_tiny/ ${cur_model_path}/data
 
     echo "###########pip install paddlenlp"
-    pip install paddlenlp 
     # Running ...
     rm -f ./run_benchmark.sh
     cp ${BENCHMARK_ROOT}/dynamic_graph/lac/paddle/run_benchmark.sh ./
@@ -752,7 +764,8 @@ dy_gpt(){
 }
 
 
-dy_xlnet() {    
+dy_xlnet() {
+    pip uninstall paddlenlp -y
     cur_model_path=${BENCHMARK_ROOT}/PaddleNLP
     cd ${cur_model_path}
     mkdir run_env

diff --git a/scripts/dynamic_to_static_models.sh b/scripts/dynamic_to_static_models.sh
@@ -21,6 +21,10 @@ dy_to_static_bert() {
     cur_model_path=${BENCHMARK_ROOT}/PaddleNLP/
     cd ${cur_model_path}
     pip install -r requirements.txt
+    pip install h5py
+    pip uninstall -y paddlenlp
+    pip install attrdict
+    python setup.py install
     cur_model_path=${BENCHMARK_ROOT}/PaddleNLP/examples/language_model/bert/
     cd ${cur_model_path}
     ln -s ${data_path}/Bert/hdf5_lower_case_1_seq_len_512_max_pred_80_masked_lm_prob_0.15_random_seed_12345_dupe_factor_5/wikicorpus_en_seqlen512 ${cur_model_path}/wikicorpus_en_seqlen512 ./data
@@ -181,8 +185,10 @@ dy_to_st_seg(){
 
 dy_to_st_transformer(){
     echo "###########pip install paddlenlp"
-    pip install paddlenlp
+    cd ${BENCHMARK_ROOT}/PaddleNLP
+    pip uninstall -y paddlenlp
     pip install attrdict
+    python setup.py install
     cur_model_path=${BENCHMARK_ROOT}/PaddleNLP/examples/machine_translation/transformer
     cd ${cur_model_path}
     # prepare data

diff --git a/scripts/static_graph_models.sh b/scripts/static_graph_models.sh
@@ -221,14 +221,18 @@ mask_rcnn(){
 
 #run_bert
 bert(){
+    cd ${BENCHMARK_ROOT}/PaddleNLP
+    pip install -r requirements.txt
+    pip install h5py
+    pip uninstall -y paddlenlp
+    pip install attrdict
+    python setup.py install
     cur_model_path=${BENCHMARK_ROOT}/PaddleNLP/examples/language_model/bert/static
     cd ${cur_model_path}
     ln -s ${data_path}/Bert/hdf5_lower_case_1_seq_len_512_max_pred_80_masked_lm_prob_0.15_random_seed_12345_dupe_factor_5/wikicorpus_en_seqlen512 ${cur_model_path}/wikicorpus_en_seqlen512
     ln -s ${data_path}/Bert/wikicorpus_en_seqlen128 ${cur_model_path}/wikicorpus_en_seqlen128
     cp ${BENCHMARK_ROOT}/static_graph/BERT/paddle/run_benchmark.sh ./run_benchmark.sh
     sed -i '/set\ -xe/d' run_benchmark.sh
-    pip install h5py
-    pip install paddlenlp
 
     model_mode_list=(base large)
     fp_mode_list=(fp32 fp16)
@@ -264,8 +268,12 @@ bert(){
 
 #run_transformer
 transformer(){
-    pip install paddlenlp
+    cd ${BENCHMARK_ROOT}/PaddleNLP
+    pip install -r requirements.txt
+    pip install h5py
+    pip uninstall -y paddlenlp
     pip install attrdict
+    python setup.py install
     cur_model_path=${BENCHMARK_ROOT}/PaddleNLP/examples/machine_translation/transformer/static
     cd ${cur_model_path}
     # prepare data