Skip to content

Commit

Permalink
Remove unnecessary reference to use_runner_v2 experiment in x-lang ex…
Browse files Browse the repository at this point in the history
…amples and documentation
  • Loading branch information
chamikaramj committed Jul 21, 2022
1 parent 4821e03 commit 033c304
Show file tree
Hide file tree
Showing 6 changed files with 8 additions and 16 deletions.
6 changes: 2 additions & 4 deletions sdks/python/apache_beam/examples/kafkataxi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,8 +104,7 @@ python -m apache_beam.examples.kafkataxi.kafka_taxi \
--region $REGION \
--num_workers $NUM_WORKERS \
--job_name $JOB_NAME \
--bootstrap_servers $BOOTSTRAP_SERVER \
--experiments=use_runner_v2
--bootstrap_servers $BOOTSTRAP_SERVER
```

## *(Optional)* Running the Example from a Beam Git Clone
Expand Down Expand Up @@ -185,6 +184,5 @@ python -m apache_beam.examples.kafkataxi.kafka_taxi \
--num_workers $NUM_WORKERS \
--job_name $JOB_NAME \
--bootstrap_servers $BOOTSTRAP_SERVER \
--sdk_harness_container_image_overrides ".*java.*,${DOCKER_ROOT}/beam_java8_sdk:latest" \
--experiments=use_runner_v2
--sdk_harness_container_image_overrides ".*java.*,${DOCKER_ROOT}/beam_java8_sdk:latest"
```
3 changes: 1 addition & 2 deletions sdks/python/apache_beam/examples/kafkataxi/kafka_taxi.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,7 @@ def run(
# '--runner', 'DataflowRunner',
# '--temp_location', 'my-temp-location',
# '--region', 'my-region',
# '--num_workers', 'my-num-workers',
# '--experiments', 'use_runner_v2']
# '--num_workers', 'my-num-workers']

window_size = 15 # size of the Window in seconds.

Expand Down
5 changes: 2 additions & 3 deletions sdks/python/apache_beam/examples/sql_taxi.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,8 @@
the data in 15s windows using SqlTransform, and writes the output to
a user-defined PubSub topic.
Java 8 must be available to run this pipeline, and the
--experiments=use_runner_v2 flag must be passed when running on Dataflow.
Docker must also be available to run this pipeline locally.
A Java version supported by Beam must be installed locally to run this pipeline.
Additionally, Docker must also be available to run this pipeline locally.
"""

# pytype: skip-file
Expand Down
5 changes: 2 additions & 3 deletions sdks/python/apache_beam/examples/wordcount_xlang_sql.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,8 @@

"""A word-counting workflow that uses the SQL transform.
Java 8 must be available to run this pipeline, and the
--experiments=use_runner_v2 flag must be passed when running on Dataflow.
Docker must also be available to run this pipeline locally.
A Java version supported by Beam must be installed locally to run this pipeline.
Additionally, Docker must also be available to run this pipeline locally.
"""

import argparse
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -149,8 +149,7 @@ export PYTHON_VERSION=<version>
./gradlew :examples:multi-language:pythonDataframeWordCount --args=" \
--runner=DataflowRunner \
--output=gs://${OUTPUT_BUCKET}/count \
--region=${GCP_REGION} \
--experiments=use_runner_v2"
--region=${GCP_REGION}"
```

The pipeline outputs a file with the results to
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -205,15 +205,13 @@ export NUM_WORKERS="1"
gsutil rm gs://$GCS_BUCKET/javaprefix/*
# --experiments=use_runner_v2 is optional for Beam 2.32.0 and later
python addprefix.py \
--runner DataflowRunner \
--temp_location $TEMP_LOCATION \
--project $GCP_PROJECT \
--region $GCP_REGION \
--job_name $JOB_NAME \
--num_workers $NUM_WORKERS \
--experiments=use_runner_v2 \
--input "gs://dataflow-samples/shakespeare/kinglear.txt" \
--output "gs://$GCS_BUCKET/javaprefix/output"
```

0 comments on commit 033c304

Please sign in to comment.