Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove unnecessary reference to use_runner_v2 experiment in x-lang examples and documentation #22376

Merged
merged 1 commit into from
Jul 28, 2022
Merged

Remove unnecessary reference to use_runner_v2 experiment in x-lang examples and documentation #22376

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions sdks/python/apache_beam/examples/kafkataxi/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,8 +104,7 @@ python -m apache_beam.examples.kafkataxi.kafka_taxi \
--region $REGION \
--num_workers $NUM_WORKERS \
--job_name $JOB_NAME \
--bootstrap_servers $BOOTSTRAP_SERVER \
--experiments=use_runner_v2
--bootstrap_servers $BOOTSTRAP_SERVER
```

## *(Optional)* Running the Example from a Beam Git Clone
Expand Down Expand Up @@ -185,6 +184,5 @@ python -m apache_beam.examples.kafkataxi.kafka_taxi \
--num_workers $NUM_WORKERS \
--job_name $JOB_NAME \
--bootstrap_servers $BOOTSTRAP_SERVER \
--sdk_harness_container_image_overrides ".*java.*,${DOCKER_ROOT}/beam_java8_sdk:latest" \
--experiments=use_runner_v2
--sdk_harness_container_image_overrides ".*java.*,${DOCKER_ROOT}/beam_java8_sdk:latest"
```
3 changes: 1 addition & 2 deletions sdks/python/apache_beam/examples/kafkataxi/kafka_taxi.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,7 @@ def run(
# '--runner', 'DataflowRunner',
# '--temp_location', 'my-temp-location',
# '--region', 'my-region',
# '--num_workers', 'my-num-workers',
# '--experiments', 'use_runner_v2']
# '--num_workers', 'my-num-workers']

window_size = 15 # size of the Window in seconds.

Expand Down
5 changes: 2 additions & 3 deletions sdks/python/apache_beam/examples/sql_taxi.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,8 @@
the data in 15s windows using SqlTransform, and writes the output to
a user-defined PubSub topic.

Java 8 must be available to run this pipeline, and the
--experiments=use_runner_v2 flag must be passed when running on Dataflow.
Docker must also be available to run this pipeline locally.
A Java version supported by Beam must be installed locally to run this pipeline.
Additionally, Docker must also be available to run this pipeline locally.
"""

# pytype: skip-file
Expand Down
5 changes: 2 additions & 3 deletions sdks/python/apache_beam/examples/wordcount_xlang_sql.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,8 @@

"""A word-counting workflow that uses the SQL transform.

Java 8 must be available to run this pipeline, and the
--experiments=use_runner_v2 flag must be passed when running on Dataflow.
Docker must also be available to run this pipeline locally.
A Java version supported by Beam must be installed locally to run this pipeline.
Additionally, Docker must also be available to run this pipeline locally.
"""

import argparse
Expand Down
3 changes: 1 addition & 2 deletions website/www/site/content/en/documentation/sdks/java-multi-language-pipelines.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,8 +149,7 @@ export PYTHON_VERSION=<version>
./gradlew :examples:multi-language:pythonDataframeWordCount --args=" \
--runner=DataflowRunner \
--output=gs://${OUTPUT_BUCKET}/count \
--region=${GCP_REGION} \
--experiments=use_runner_v2"
--region=${GCP_REGION}"
```

The pipeline outputs a file with the results to
Expand Down
2 changes: 0 additions & 2 deletions website/www/site/content/en/documentation/sdks/python-multi-language-pipelines.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -205,15 +205,13 @@ export NUM_WORKERS="1"

gsutil rm gs://$GCS_BUCKET/javaprefix/*

# --experiments=use_runner_v2 is optional for Beam 2.32.0 and later
python addprefix.py \
--runner DataflowRunner \
--temp_location $TEMP_LOCATION \
--project $GCP_PROJECT \
--region $GCP_REGION \
--job_name $JOB_NAME \
--num_workers $NUM_WORKERS \
--experiments=use_runner_v2 \
--input "gs://dataflow-samples/shakespeare/kinglear.txt" \
--output "gs://$GCS_BUCKET/javaprefix/output"
```