Skip to content

Commit

Permalink
Implement custom code highlighter function (apache#21)
Browse files Browse the repository at this point in the history
  • Loading branch information
kgabryje authored and robertwb committed Apr 30, 2020
1 parent cc4384c commit c4252ef
Show file tree
Hide file tree
Showing 3 changed files with 29 additions and 30 deletions.
16 changes: 4 additions & 12 deletions website/www/site/content/en/blog/stateful-processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -276,9 +276,7 @@ write stateful processing code using Beam's Java SDK. Here is the code for a
stateful `DoFn` that assigns an arbitrary-but-consistent index to each element
on a per key-and-window basis:

{{% classwrapper class="language-java" %}}

```java
{{< highlight java >}}
new DoFn<KV<MyKey, MyValue>, KV<Integer, KV<MyKey, MyValue>>>() {

// A state cell holding a single Integer per key+window
Expand All @@ -295,13 +293,9 @@ new DoFn<KV<MyKey, MyValue>, KV<Integer, KV<MyKey, MyValue>>>() {
index.write(current+1);
}
}
```

{{% /classwrapper %}}
{{< /highlight >}}

{{% classwrapper class="language-py" %}}

```py
{{< highlight py >}}
class IndexAssigningStatefulDoFn(DoFn):
INDEX_STATE = CombiningStateSpec('index', sum)

Expand All @@ -310,9 +304,7 @@ class IndexAssigningStatefulDoFn(DoFn):
current_index = index.read()
yield (value, current_index)
index.add(1)
```

{{% /classwrapper %}}
{{< /highlight >}}

Let's dissect this:

Expand Down
24 changes: 6 additions & 18 deletions website/www/site/content/en/get-started/quickstart-go/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,18 +53,12 @@ required arguments described in the examples.

For example, to run `wordcount`, run:

{{% classwrapper class="runner-direct" %}}

```
{{< highlight class="runner-direct" >}}
$ go install github.com/apache/beam/sdks/go/examples/wordcount
$ wordcount --input <PATH_TO_INPUT_FILE> --output counts
```
{{< /highlight >}}

{{% /classwrapper %}}

{{% classwrapper class="runner-dataflow" %}}

```
{{< highlight class="runner-dataflow" >}}
$ go install github.com/apache/beam/sdks/go/examples/wordcount
# As part of the initial setup, for non linux users - install package unix before run
$ go get -u golang.org/x/sys/unix
Expand All @@ -75,17 +69,11 @@ $ wordcount --input gs://dataflow-samples/shakespeare/kinglear.txt \
--temp_location gs://<your-gcs-bucket>/tmp/ \
--staging_location gs://<your-gcs-bucket>/binaries/ \
--worker_harness_container_image=apache/beam_go_sdk:latest
```

{{% /classwrapper %}}

{{% classwrapper class="runner-nemo" %}}
{{< /highlight >}}

```
{{< highlight class="runner-nemo" >}}
This runner is not yet available for the Go SDK.
```

{{% /classwrapper %}}
{{< /highlight >}}

## Next Steps

Expand Down
19 changes: 19 additions & 0 deletions website/www/site/layouts/shortcodes/highlight.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{{ $content := .Inner }}
{{ $ctx := . }}
{{ with (.Get "class") }}
<div class={{ . }}>
{{ with ($ctx.Get "lang") }}
{{ highlight (trim $content "\n\r") . "" }}
{{ else }}
{{ (printf "```%s```" $content) | markdownify }}
{{ end }}
</div>
{{ else }}
{{ with (.Get 0) }}
<div class={{ printf "language-%s" . }}>
{{ highlight (trim $content "\n\r") . "" }}
</div>
{{ else }}
{{ (printf "```%s```" $content) | markdownify }}
{{ end }}
{{ end }}

0 comments on commit c4252ef

Please sign in to comment.