Add analyzer docs - step 3 of walkthrough (#16)

docs/step-03/03-example-analysis.ex

@@ -0,0 +1,4 @@
+defmodule ElixirAnalyzer.ExerciseTest.Example do
+  @dialyzer generated: true
+  use ElixirAnalyzer.ExerciseTest
+end

docs/step-03/03-example-module.ex

@@ -0,0 +1,6 @@
+defmodule Example do
+
+  def hello(name) do
+    "Hello, #{name}!"
+  end
+end

docs/step-03/step-03.md

@@ -0,0 +1,237 @@
+# A Guide to Writing an Elixir Analyzer Extension
+
+## Step 3: Adding the first feature test
+
+### module
+
+Remember, this is the simple module that we want to analyze:
+
+```elixir
+defmodule Example do
+
+  def hello(name) do
+    "Hello, #{name}!"
+  end
+end
+```
+
+It contains a simple function which returns a string created with the string interpolation syntax and the variable provided.
+
+### approach
+
+So how do we go about analyzing this?  Let's think about what patterns we want to find in the ideal solution, and what patterns we don't want to find in the ideal solution.
+
+> Remember, analysis is not a replacement for the test unit, and testing the presence and output of public functions is better and more clearly tested using appropriate testing.
+
+Some discrete things that we want to find:
+
+- a parameter named `name`
+- the use of string interpolation
+
+Some discrete things that we don't want to find:
+
+- binary concatenation
+
+### analyzer extension
+
+So now let's revisit the analyzer extension from step 2, and write an analyzer test to check if the name of the parameter is `name`.  Our analyzer extension module so far:
+
+```elixir
+defmodule ElixirAnalyzer.ExerciseTest.Example do
+  @dialyzer generated: true
+  use ElixirAnalyzer.ExerciseTest
+end
+```
+
+We are going to add a call to the the macro `feature/1` which we gained use of by including `use ElixirAnalyzer.ExerciseTest` on line 2.
+
+#### 1. Add a feature block with a string name
+
+```diff
+defmodule ElixirAnalyzer.ExerciseTest.Example do
+  @dialyzer generated: true
+  use ElixirAnalyzer.ExerciseTest
+
++ feature "has a parameter called 'name'" do
++ end
+end
+```
+
+Now we have the to add in some details that describe the feature and what it should do if it fails to match our description.
+
+#### 2. Add the form block to our feature test
+
+The form block is how we describe what we want elixir to pattern match on. Pattern matching is an important concept which we are going to expand on.  Similar to pattern matching on a nested list/tuple structure, we are going to pattern match on the AST (abstract syntax tree) generated by the elixir compiler.  Let's look at the AST of the function in our module:
+
+```shell
+iex> ast = quote do
+...>   def hello(name) do
+...>     "Hello, #{name}!"
+...>   end
+...> end
+{:def, [context: Elixir, import: Kernel],
+ [
+   {:hello, [context: Elixir], [{:name, [], Elixir}]},
+   [
+     do: {:<<>>, [],
+      [
+        "Hello, ",
+        {:"::", [],
+         [
+           {{:., [], [Kernel, :to_string]}, [], [{:name, [], Elixir}]},
+           {:binary, [], Elixir}
+         ]},
+        "!"
+      ]}
+   ]
+ ]}
+```
+
+This is the AST of our function that we want to pattern match on.  We can experiment with this and see that we can get a match on subsets of this by using `_` as a placeholder:
+
+```elixir
+# Matches
+{:def, _, _} = ast
+{:def, _,[_|_]} = ast
+{:def, _, [{:hello, _, _}, _]} = ast
+```
+
+And we can match on a subset of this tree, if we traverse this structure using an algorithm like `Macro.prewalk/3` to find matches. But pattern matching manually would be a chore to do by hand, and likely prone to error, so let's make use of the functions in `ElixirAnalyzer.ExerciseTest` and let our DSL (domain specific language) define this for us.  So what we need to worry about is finding a piece of syntax that will generate an AST snippet that we can match on:
+
+```elixir
+# `form` is a special word in this DSL to define a pattern to look for
+form do
+  def hello(name) do
+    ...
+  end
+end
+```
+
+So we know we want a function named hello and with a parameter named `name` to be matched, but we haven't looked at how to ignore all the other stuff that make it complicated.  In this DSL, we use the keyword `_ignore` inside of the form block which is transformed into `_` when our pattern is compiled:
+
+```diff
+form do
+  def hello(name) do
++   _ignore
+  end
+end
+```
+
+Now we have the pattern to match, let's insert it into our extension:
+
+```diff
+defmodule ElixirAnalyzer.ExerciseTest.Example do
+  @dialyzer generated: true
+  use ElixirAnalyzer.ExerciseTest
+
+  feature "has a parameter called 'name'" do
++   form do
++     def hello(name) do
++       _ignore
++     end
++   end
+  end
+end
+```
+
+#### 3. Add attributes to the matching process
+
+So far we have defined a feature to test, and a pattern to match, but we need to further specify how we want it to match and what it should do when it doesn't.  We need to add the `find`, `on_fail`, and `comment` attributes:
+
+##### `find`
+
+`find` tells our analyzer how we want it to match the patterns we define for the test (we will soon look at defining multiple patterns for a single test).  It has a few different accepted arguments:
+
+- `:all` - the analyzer must match all of the patterns defined for the test to pass
+- `:any` - the analyzer must match any of the patterns defined for the test to pass
+- `:one` - the analyzer must only match one of the patterns defined for the test to pass
+- `:none` - the analyzer must not match any of the patterns defined for the test to pass
+
+Because we only have one pattern, `:any`, `:all`, or `:one` would all produce the same output, so let's just choose `:all`
+
+```diff
+defmodule ElixirAnalyzer.ExerciseTest.Example do
+  @dialyzer generated: true
+  use ElixirAnalyzer.ExerciseTest
+
+  feature "has a parameter called 'name'" do
++   find :all
+
+    form do
+      def hello(name) do
+        _ignore
+      end
+    end
+  end
+end
+```
+
+##### `on_fail`
+
+`on_fail` tells our analyzer how to treat the solution when it fails the test.  There are 3 accepted actions:
+
+- `:info` - append the comment associated with this test
+- `:disapprove` - disapprove the submission if the test fails, append the comment associated with this test
+- `:refer` - the analyzer is unable to determine if the submission should be approved or disapproved, flag that it requires manual attention.
+
+For our test, let's disapprove of the solution if the pattern doesn't match the submission:
+
+```diff
+defmodule ElixirAnalyzer.ExerciseTest.Example do
+  @dialyzer generated: true
+  use ElixirAnalyzer.ExerciseTest
+
+  feature "has a parameter called 'name'" do
+    find :all
++   on_fail :disapprove
+
+    form do
+      def hello(name) do
+        _ignore
+      end
+    end
+  end
+end
+```
+
+##### `comment`
+
+Analysis of the solution is coordinated by a central service in exercism's infrastructure. Delivery of the report to the student is through the website.  The website copy would be cumbersome to add into this test framework, so a string to locate the copy in another repo is used.  It takes the form: `elixir.__exercise_slug__.__comment_to_display__` in snake case format.
+
+At this moment, the comments are found in the [exercism/website-copy][website-copy] repo.
+
+so lets add that to our analyzer as well:
+
+```diff
+defmodule ElixirAnalyzer.ExerciseTest.Example do
+  @dialyzer generated: true
+  use ElixirAnalyzer.ExerciseTest
+
+  feature "has a parameter called 'name'" do
+    find :all
+    on_fail :disapprove
++   comment "elixir.example.use_name_parameter"
+
+    form do
+      def hello(name) do
+        _ignore
+      end
+    end
+  end
+end
+```
+
+### 4. Now test
+
+So this next part will likely take a bit of testing to isolate the exact pattern you wish to isolate, but the workflow is generally:
+
+1. make changes to the analyzer function
+2. rebuild the escript
+3. run the analyzer on the file via CLI
+4. determine if the analyze.json has to appropriate output
+5. (repeat)
+
+If you have made it this far, great!  You are doing awesome.  Now to learn about some of the other things we can do in a match in [step 4][step-4]
+
+[website-copy]: https://github.com/exercism/website-copy/tree/master/automated-comments
+[step-4]: ../step-04/step-04.md

docs/writing-an-analyzer.md

@@ -3,7 +3,9 @@
 1. [Setting up the analyzer locally][step-1]
 2. [Anatomy of an extension][step-2]
 3. [Adding a feature test][step-3]
+4. [What else can we do with a test][step-4]
 
 [step-1]: ./step-01/step-01.md
 [step-2]: ./step-02/step-02.md
 [step-3]: ./step-03/step-03.md
+[step-4]: ../step-04/step-04.md