Skip to content

Commit

Permalink
Merge pull request #22376: Remove unnecessary reference to use_runner…
Browse files Browse the repository at this point in the history
…_v2 experiment in x-lang examples and documentation
  • Loading branch information
chamikaramj authored Jul 28, 2022
2 parents dafa380 + 033c304 commit f4f1de3
Show file tree
Hide file tree
Showing 6 changed files with 8 additions and 16 deletions.
6 changes: 2 additions & 4 deletions sdks/python/apache_beam/examples/kafkataxi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,8 +104,7 @@ python -m apache_beam.examples.kafkataxi.kafka_taxi \
--region $REGION \
--num_workers $NUM_WORKERS \
--job_name $JOB_NAME \
--bootstrap_servers $BOOTSTRAP_SERVER \
--experiments=use_runner_v2
--bootstrap_servers $BOOTSTRAP_SERVER
```

## *(Optional)* Running the Example from a Beam Git Clone
Expand Down Expand Up @@ -185,6 +184,5 @@ python -m apache_beam.examples.kafkataxi.kafka_taxi \
--num_workers $NUM_WORKERS \
--job_name $JOB_NAME \
--bootstrap_servers $BOOTSTRAP_SERVER \
--sdk_harness_container_image_overrides ".*java.*,${DOCKER_ROOT}/beam_java8_sdk:latest" \
--experiments=use_runner_v2
--sdk_harness_container_image_overrides ".*java.*,${DOCKER_ROOT}/beam_java8_sdk:latest"
```
3 changes: 1 addition & 2 deletions sdks/python/apache_beam/examples/kafkataxi/kafka_taxi.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,7 @@ def run(
# '--runner', 'DataflowRunner',
# '--temp_location', 'my-temp-location',
# '--region', 'my-region',
# '--num_workers', 'my-num-workers',
# '--experiments', 'use_runner_v2']
# '--num_workers', 'my-num-workers']

window_size = 15 # size of the Window in seconds.

Expand Down
5 changes: 2 additions & 3 deletions sdks/python/apache_beam/examples/sql_taxi.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,8 @@
the data in 15s windows using SqlTransform, and writes the output to
a user-defined PubSub topic.
Java 8 must be available to run this pipeline, and the
--experiments=use_runner_v2 flag must be passed when running on Dataflow.
Docker must also be available to run this pipeline locally.
A Java version supported by Beam must be installed locally to run this pipeline.
Additionally, Docker must also be available to run this pipeline locally.
"""

# pytype: skip-file
Expand Down
5 changes: 2 additions & 3 deletions sdks/python/apache_beam/examples/wordcount_xlang_sql.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,8 @@

"""A word-counting workflow that uses the SQL transform.
Java 8 must be available to run this pipeline, and the
--experiments=use_runner_v2 flag must be passed when running on Dataflow.
Docker must also be available to run this pipeline locally.
A Java version supported by Beam must be installed locally to run this pipeline.
Additionally, Docker must also be available to run this pipeline locally.
"""

import argparse
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -149,8 +149,7 @@ export PYTHON_VERSION=<version>
./gradlew :examples:multi-language:pythonDataframeWordCount --args=" \
--runner=DataflowRunner \
--output=gs://${OUTPUT_BUCKET}/count \
--region=${GCP_REGION} \
--experiments=use_runner_v2"
--region=${GCP_REGION}"
```

The pipeline outputs a file with the results to
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -205,15 +205,13 @@ export NUM_WORKERS="1"
gsutil rm gs://$GCS_BUCKET/javaprefix/*
# --experiments=use_runner_v2 is optional for Beam 2.32.0 and later
python addprefix.py \
--runner DataflowRunner \
--temp_location $TEMP_LOCATION \
--project $GCP_PROJECT \
--region $GCP_REGION \
--job_name $JOB_NAME \
--num_workers $NUM_WORKERS \
--experiments=use_runner_v2 \
--input "gs://dataflow-samples/shakespeare/kinglear.txt" \
--output "gs://$GCS_BUCKET/javaprefix/output"
```

0 comments on commit f4f1de3

Please sign in to comment.