docs(mpt): updating docs and examples

PIPELINE_TEMPLATES.md

@@ -35,9 +35,10 @@ For usage information, please see [ALPHA-GETTING-STARTED.md](ALPHA-GETTING-START
 
 # Taxonomy
 
-* *Pipeline Template*: An abstract, composable definition of a Spinnaker Pipeline
-* *Pipeline Configuration*: A variable-binding and user-facing configuration
-  format that inherits a Pipeline Template.
+* *Pipeline Template (template)*: An abstract, composable definition of a Spinnaker Pipeline
+* *Pipeline Configuration (configuration)*: A variable-binding and user-facing configuration
+  format that combines with a Pipeline Template to make it runnable.
+* *Spinnaker Pipeline (pipeline)*: (what you see in the UI) a pipeline, represented internally by json, that can be run by orca.
 
 # Key Concepts
 
@@ -46,42 +47,12 @@ For usage information, please see [ALPHA-GETTING-STARTED.md](ALPHA-GETTING-START
   concrete implementations of a Template.
 * Composability of Templates are done via Modules. Configurations can configure
   a module or entirely replace modules if they don't fit.
-* Templates can use Jinja expression syntax for better flow control.
 * Configurations can inject new stages into the final pipeline graph and modify
   individual objects with JSONPath.
+* Templates can use Jinja expression syntax for better flow control.
 
-# Jinja Templating
-
-Templates can use jinjava (Java implementation of Jinja) to offer greater
-productivity while creating and using templates. Jinja templating is only
-allowed in string values so that the JSON transport can always be valid. The
-results of Jinja templates can and often will result in non-string values (even
-object graphs).
-
-For more information about what's possible with Jinja, and specifically jinjava:
-
-* https://github.com/HubSpot/jinjava
-* http://jinja.pocoo.org/docs/2.9/ (Python implementation; good reference)
-
-Jinja is supported in the following areas:
-
-* Stage & module `when` stanzas
-* Module `definition` stanza (or any nested value inside)
-* Stage `config` stanza (or any nested value inside)
-* Stage `name` stanza
-* Template `metadata` values
-
-## Custom Tags & Filters
-
-We're continually extending the Jinja implementation with new Tags and Filters.
-
-* `frigga` filter: Used to parse string values and return Frigga naming
-  convention substrings. `{{ "orca-main"|frigga('stack') }} == "main" }}`
-* `json` filter: Output a value as a JSON object. `{{ myVar|json }}`
-* `base64` filter: Output a value as a Base64 encoded string. `{{ myVar|base64 }}`
-
-**IMPORTANT**: If your Jinja template is intended to return a list, map or
-object graph, you must ensure the output is *valid YAML or JSON*.
+# Explained in Pictures
+![mpt-graphic](MPTGraphic.png "Logo Title Text 1")
 
 # Template & Configuration Schemas
 
@@ -96,7 +67,7 @@ UI. **NOTE:** This stanza, while defined in the spec, is not currently consumed
 by Spinnaker itself.
 
 This section primarily gives explanations of Templates & Configurations. For
-actual reference schemas, please take a look at [schemas/pipeline-templates-v1](schemas/pipeline-templates-v1).
+actual reference schemas, please take a look at [schemas/pipeline-templates-v1](schemas/pipeline-templates-v1). For examples, please take a look at [examples/pipeline-templates-v1](examples/pipeline-templates-v1).
 
 ## Templates
 
@@ -210,6 +181,41 @@ When using the *spinnaker scheme*, it's recommended to namespace your template
 IDs, as they are globally unique, regardless of the scope you provide them while
 publishing.
 
+# Jinja Templating
+
+Templates can use jinjava (Java implementation of Jinja) to offer greater
+productivity while creating and using templates. Jinja templating is only
+allowed in string values so that the JSON transport can always be valid. The
+results of Jinja templates can and often will result in non-string values (even
+object graphs).
+
+For more information about what's possible with Jinja, and specifically jinjava:
+
+* https://github.com/HubSpot/jinjava
+* http://jinja.pocoo.org/docs/2.9/ (Python implementation; good reference)
+
+Jinja is supported in the following areas:
+
+* Stage & module `when` stanzas
+* Module `definition` stanza (or any nested value inside)
+* Stage `config` stanza (or any nested value inside)
+* Stage `name` stanza
+* Template `metadata` values
+
+## Custom Tags & Filters
+
+We're continually extending the Jinja implementation with new Tags and Filters.
+
+* `frigga` filter: Used to parse string values and return Frigga naming
+  convention substrings. `{{ "orca-main"|frigga('stack') }} == "main" }}`
+* `json` filter: Output a value as a JSON object. `{{ myVar|json }}`
+* `base64` filter: Output a value as a Base64 encoded string. `{{ myVar|base64 }}`
+
+**IMPORTANT**: If your Jinja template is intended to return a list, map or
+object graph, you must ensure the output is *valid YAML or JSON*.
+
+
+
 # Variables
 
 Variables are used during the Jinja template rendering phase to build up stages.
@@ -332,6 +338,8 @@ Conditional stages are supported within Partials as well.
 
 # Modules
 
+Modules are used to represent common functionality within stages, like stage configuration.
+
 Modules can be referenced by each other, the Template they are defined in, 
 child Templates and Configurations. Furthermore, they can be replaced by child
 Templates and Configurations as well.

examples/README.md

@@ -0,0 +1,8 @@
+# dcd-examples
+
+Examples of how to use MPT, organized by schema version. 
+
+Templates are linked using the `file://myfile.yml` format. 
+To use templates in a hosted manner, change all the `source:` fields to represent the URL
+ of the raw file or the `spinnaker://` format. 
+See [my gists](https://gist.github.com/emjburns/b769578860ecbb84d64ea72b544e205c) for an example.

examples/pipeline-templates-v1/bake-deploy/README.md

@@ -0,0 +1,9 @@
+# This example shows a bake and deploy pipeline with the option to override to a find image and deploy pipeline
+The UI view for this pipeline is shown in [complex-wait-ui.png](complex-wait-ui.png). Same UI for all three options.
+
+## Option 1: Bake
+`template.yml` shows a sample multi region bake and deploy. `bake-deploy-config.yml` shows the config for that pipeline, providing a stage name for the first stage.
+
+
+## Option 2: Find Image
+`template.yml` remains the same. `findImage-deploy-config.yml` shows replacing the bake stage with a findImage stage. 

examples/pipeline-templates-v1/bake-deploy/bake-deploy-config.yml

@@ -0,0 +1,10 @@
+---
+schema: "1"
+pipeline:
+  application: orca
+  name: Bake and Deploy
+  template:
+    source: file://template.yml
+  variables:
+    myCustomFirstStageName: "Bake"
+stages: []

examples/pipeline-templates-v1/bake-deploy/findImage-deploy-config.yml

@@ -0,0 +1,24 @@
+---
+schema: "1"
+pipeline:
+  application: orca
+  name: Find Image and Deploy
+  template:
+    source: file://template.yml
+  variables:
+    myCustomFirstStageName: "Find Image"
+stages:
+- id: bake1
+  type: findImageFromTags
+  name: "{{ myCustomFirstStageName }}"
+  config:
+    cloudProvider: aws
+    cloudProviderType: aws
+    packageName: "{{ application }}"
+    regions:
+    - us-east-1
+    - us-west-1
+    - us-west-2
+    - eu-west-1
+    tags:
+      stack: test

examples/pipeline-templates-v1/bake-deploy/template.yml

@@ -0,0 +1,114 @@
+---
+schema: "1"
+id: bakeDeploy
+metadata:
+  name: Bake and Deploy
+  description: Example bake and deploy template
+variables:
+- name: myCustomFirstStageName
+stages:
+- id: bake1
+  type: bake
+  name: "{{ myCustomFirstStageName }}"
+  config:
+    baseLabel: release
+    baseOs: trusty
+    cloudProviderType: aws
+    enhancedNetworking: false
+    extendedAttributes: {}
+    overrideTimeout: true
+    package: orca
+    regions:
+    - us-east-1
+    - us-west-1
+    - us-west-2
+    - eu-west-1
+    sendNotifications: true
+    showAdvancedOptions: true
+    stageTimeoutMs: 900000
+    storeType: ebs
+    user: [email protected]
+    vmType: hvm
+- id: deploy2
+  type: deploy
+  dependsOn:
+  - bake1
+  name: Deploy
+  config:
+    clusters:
+    - account: test
+      application: orca
+      availabilityZones:
+        us-west-1:
+        - us-west-1a
+        - us-west-1c
+      capacity:
+        desired: 1
+        max: 1
+        min: 1
+      cloudProvider: aws
+      cooldown: 10
+      copySourceCustomBlockDeviceMappings: true
+      ebsOptimized: false
+      enabledMetrics: []
+      freeFormDetails: demo
+      healthCheckGracePeriod: 600
+      healthCheckType: EC2
+      iamRole: myIAMRole
+      instanceMonitoring: false
+      instanceType: m3.large
+      interestingHealthProviderNames:
+      - Amazon
+      keyPair: keypair
+      loadBalancers:
+      - orca-demo-frontend
+      maxRemainingAsgs: 2
+      preferSourceCapacity: true
+      provider: aws
+      scaleDown: true
+      securityGroups: []
+      stack: test
+      strategy: redblack
+      subnetType: mySubnet
+      suspendedProcesses: []
+      tags: {}
+      targetGroups: []
+      targetHealthyDeployPercentage: 100
+      terminationPolicies:
+      - Default
+      useAmiBlockDeviceMappings: false
+      useSourceCapacity: true
+    - account: test
+      application: orca
+      availabilityZones:
+        us-east-1:
+        - us-east-1c
+        - us-east-1d
+        - us-east-1e
+      capacity:
+        desired: 0
+        max: 0
+        min: 0
+      cloudProvider: aws
+      cooldown: 10
+      ebsOptimized: false
+      freeFormDetails: demo
+      healthCheckGracePeriod: 600
+      healthCheckType: EC2
+      iamRole: myIAMRole
+      instanceMonitoring: false
+      instanceType: m3.large
+      interestingHealthProviderNames:
+      - Amazon
+      keyPair: keypair
+      provider: aws
+      securityGroups: []
+      stack: test
+      strategy: highlander
+      subnetType: mySubnet
+      suspendedProcesses: []
+      tags: {}
+      targetHealthyDeployPercentage: 100
+      terminationPolicies:
+      - Default
+      useSourceCapacity: false

examples/pipeline-templates-v1/complex-wait/README.md

@@ -0,0 +1,12 @@
+# This example shows a complex wait pipeline.   
+The UI view for this pipeline is shown in [complex-wait-ui.png](complex-wait-ui.png). Same UI for all three options.
+
+## Option 1: Inheritance
+`root-template.yml`, `child-template.yml`, and `child-2-template.yml` show an inheritence structure and the use variable. `mypipeline-config.yml` adds another stage at the configuration level.    
+
+
+## Option 2: One template
+`combined-template.yml` shows the same stages represented in one template. `mypipelineCombined-config.yml` shows the config for that template that produces the same pipeline.
+
+## Option 3: All Config
+`only-config.yml` shows how you can create the same pipeline with one config file. You might want to do this if you are only using this logic for one pipeline, or you are working to transition to managed pipeline templates and want to first convert each pipeline to code.

examples/pipeline-templates-v1/complex-wait/child-2-template.yml

@@ -0,0 +1,16 @@
+schema: "1"
+id: child-2-template
+source: file://child-template.yml
+variables:
+- name: childWaitTime
+  description: pause time for another wait
+metadata:
+  name: A Second Child template
+  description: A second child template
+stages:
+- id: waitChild2
+  type: wait
+  dependsOn: 
+  - wait1
+  config:
+    waitTime: "{{ childWaitTime }}"

examples/pipeline-templates-v1/complex-wait/child-template.yml

@@ -0,0 +1,13 @@
+schema: "1"
+id: child-template
+source: file://root-template.yml   # Indicates that this template inherits from the root-template
+metadata:
+  name: Child template
+  description: A child template
+stages:
+- id: waitChild1
+  type: wait
+  dependsOn:
+  - wait1                   # Depending on a stage from the root-template
+  config:
+    waitTime: "{{ waitTime }}"  # Using a variable from the root-template

examples/pipeline-templates-v1/complex-wait/combined-template.yml

@@ -0,0 +1,38 @@
+# "Combined" template, exact same as child-2-template.yml +
+# the stage defined in the config
+
+schema: "1"
+id: combined-template
+metadata:
+  name: Less simple wait template
+  description: A template with many waits
+variables:
+- name: waitTime
+  description: The time a wait stage should pause
+  type: int
+- name: childWaitTime
+  description: pause time for another wait
+stages:
+- id: wait1
+  type: wait
+  config:
+    waitTime: "{{ waitTime }}"
+- id: waitChild1
+  type: wait
+  dependsOn:
+  - wait1
+  config:
+    waitTime: "{{ waitTime }}"
+- id: waitChild2
+  type: wait
+  dependsOn:
+  - wait1
+  config:
+    waitTime: "{{ childWaitTime }}"
+- id: finalWait
+  type: wait
+  dependsOn:
+  - waitChild1
+  - waitChild2
+  config:
+    waitTime: 10