Skip to content

Commit

Permalink
Update documentation (#149)
Browse files Browse the repository at this point in the history 
* Rewrite the horribly outdated CONTRIBUTING.md

* Reformat README

* Mention asdf

* Add missing link
  • Loading branch information
@angelikatyborska
angelikatyborska authored Sep 13, 2021
1 parent d448bdf commit fda10a5
Show file tree
Hide file tree
Showing 3 changed files with 41 additions and 166 deletions.
165 changes: 14 additions & 151 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
@@ -1,168 +1,31 @@
# Contributing

Automated Mentoring support for the Elixir track is a work in progress, and
it's not uncommon that people discover incorrect analysis, have a better
approach on detecting certain code paths, report missing edge cases, factual
errors, logical errors and develop new analysers.
Thank you so much for contributing! 🎉

We welcome contributions of all sorts and sizes, from reporting issues to
submitting patches, as well as joining the current [💬 discussions][issue-discussion].

-----

This guide covers several common scenarios pertaining to **improving the
Elixir analyzer**. There are several other guides about contributing to
other parts of the Exercism ecosystem.

* [The Elixir track][contributing-elixir]
* [Project Automated Mentoring Support][contributing-automated-mentoring-support]
* [The Elixir Analyzer][contributing-elixir-analyzer]
Unprompted (without an issue) PRs with small bugfixes are welcome, but if you want to propose bigger changes, make sure to open an issue first so that we can discuss it.

## Code of Conduct

Help us keep Exercism welcoming. Please read and abide by the [Code of Conduct][coc].

## Analyzers

Before contributing code to any existing analyzer or any new analyzer, please
have a thorough look at the current analyzers and dive into [the interface][docs-interface]
and general [analyzer guidelines][docs-guidelines].

### New exercise / analyzer

Let's say you want to write a new analyzer, because you've noticed that this
exercise could benefit from automated mentoring support, or just because you
find this interesting, the first step is to [check for an open issue][issue-new-exercise].
If it's there, make sure no one is working on it, and most of all that there is
not an open Pull Request towards this exercise.

If there is no such issue, you may open one. Take [this problem spec][sample-twofer]
as a baseline of what should be in it. This serves as a starting point for
all discussion, as we now must establish a few things before starting to write
code.

#### Project Track Anatomy

If the exercise has undergone at least phase 1 of Project Track Anatomy, it is
likely that there are [mentoring notes][mentor-notes]. These are likely to
contain what the _maintainers_ of the JavaScript track _or_ the dedicated people
from the track anatomy project deem to be the set of reasonable solutions.

#### Abstract Syntax Approach

These solutions will be your baseline for `approve`, one of the expected
outputs. The Elixir analyzer takes an abstract syntax tree approach to look for
features in the problem submission to then return feedback to determine whether
it should be approved or disapproved.

#### Discussion 💬 and Samples

In your issue, write out how you're going to tackle the initial set of
solutions, and work on a prototype. If there are no `fixtures` yet, we will
supply you with a set of fixtures (500, if there are at least 500 submissions)
so you can run your analyzer using `batch.sh slug` on these fixtures.

It can be very fruitful to ask current maintainers and previous authors of
analyzers how to tackle various approaches, especially since there are so many
ways to write equivalent code in Elixir.

#### Writing comments

Comments are the output of the Analyzer. You don't need to be a great writer in
order to contribute to the analyzers. How this works inside the repository is
a guide on its own: read more about [📝 Comments][docs-comments].

#### Tests

While no formal CI testing is done, would welcome a framework to integrate it in
to the workflow. Most exercism CI is done with TravisCI.

### Sync with exercise

<!-- Explain that syncs in problem-descriptions need to be synced with the analyzers,
establish the set of rules how to update, but wait until there is proper
versioning and how that is given at runtime -->

This section will be written once there is consensus on how the input will be
versioned. Until then, the steps are the same as for writing a new analyzer,
except that you may skip creating an issue.

### Add new behaviour

<!-- Adding new tests is mandatory -->

This section will be written once there is consensus on coverage and tests in
general.

### Commentary copy

This section will be written once there is consensus on how the commentary is
structured in `website-copy` and what the format for parameters/templates is
like.

Find all _technical_ details in [the doc about 📝 Comments][docs-comments].

## Tools

We use various tools to maintain this repository and this analyzer. In order
to contribute to the _code_ of this track, you'll need Elixir 1.9.1 or higher.

### `analyze` (.sh, .bat)

```shell
# not yet built, but plan for this to be the
./bin/analyze.sh two-fer ~/folder/to/solution
```

This runs the analyzer using `two-fer` as exercise and a path to a solution.
Most scripts, including this one, accept a wide range of flags to change or
enhance the behaviour, as coded in [`elixir_analyzer/cli.ex`][file-execution-options].

<!-- Run with the `-h` / `--help` flag to get a list of flags and their description.
```shell
./bin/analyze.sh --help
``` -->

<!--
### `remote` (.sh, .bat)
```shell
./bin/remote.sh https://exercism.io/tracks/javascript/exercises/two-fer/solutions/df3bb5d7131c44ea9c62206cc8d6c225 -dcp --dry
```
You need the [`exercism` cli][cli] in order for this to work. It takes an
_exercism solution url_. and downloads it using the `exercism` cli. It then
runs the analyzer on it.
## How does this project work?

You'll most likely want `-dcp --dry` (`--debug`, `--pretty`, `--console` and
`dry run`) during development, which enables console output (instead of
`stdout`/`stderr`), shows `logger.log` as well as `logger.error` and
`logger.fatal`, pretty prints the JSON output and disables writing the output
to `analysis.json`.
To learn how the Elixir Analyzer works:

You can pass the following type of URLs:
- Read [the guide to writing an Elixir Analyzer extension](https://github.com/exercism/elixir-analyzer/blob/main/docs/writing-an-analyzer.md).
- Browse [recipes](https://github.com/exercism/elixir-analyzer/blob/main/docs/recipes.md).
- Browse [existing exercise extensions](https://github.com/exercism/elixir-analyzer/tree/main/lib/elixir_analyzer/test_suite).

- Published solutions: `/tracks/javascript/exercises/<slug>/<id>`
- Mentor solutions: `/mentor/solutions/<id>`
- Your solutions: `/my/solutions/<id>`
- Private solutions: `/solutions/<id>`
### Writing comments

If you wish to _preview_ the actual messages, pass in `--noTemplates` to use
the analyzer `Comment`Factories to generate actual messages. If the comment
factories are kept in-sync with `website-copy`, it will be the exact same
output as on the site.
All the Analyzer comments for all tracks are stored together in [exercism/website-copy][analyzer-comments]. When adding a new Analyzer check with a new comment, you need to open a corresponding PR in [exercism/website-copy][analyzer-comments] with that comment's content.

-->
See [the Analyzer comment copy guidelines][analyzer-comments-guidelines] for guidance on how to write friendly and constructive Analyzer comments.

[cli]: https://github.com/exercism/cli
[contributing-elixir]: https://github.com/exercism/elixir/blob/master/CONTRIBUTING.md
[contributing-elixir-analyzer]: https://github.com/exercism/elixir-analyzer/blob/master/CONTRIBUTING.md
[contributing-automated-mentoring-support]: https://github.com/exercism/automated-mentoring-support/
[coc]: https://exercism.io/code-of-conduct
[docs-interface]: https://github.com/exercism/automated-mentoring-support/blob/master/docs/interface.md
[docs-guidelines]: https://github.com/exercism/automated-mentoring-support/blob/master/docs/guidelines.md
[issue-discussion]: https://github.com/exercism/elixir-analyzer/issues?q=is%3Aopen+is%3Aissue+label%3A%22discussion%22%3Aspeech_balloon%3A
[issue-new-exercise]: https://github.com/exercism/elixir-analyzer/labels/new%20exercise%20%3Asparkles%3A
[mentor-notes]: https://github.com/exercism/website-copy/tree/master/tracks/javascript/exercises
[coc]: https://github.com/exercism/elixir/blob/main/CODE_OF_CONDUCT.md
[analyzer-comments]: https://github.com/exercism/website-copy/tree/main/analyzer-comments
[analyzer-comments-guidelines]: https://github.com/exercism/docs/blob/main/building/tooling/analyzers/comments.md#comment-copy-guidelines
[issue-discussion]: https://github.com/exercism/elixir-analyzer/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3A%22discussion+%F0%9F%92%AC%22
36 changes: 23 additions & 13 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,29 +10,25 @@ See the project docs: https://github.com/exercism/docs/tree/main/building/toolin

`ElixirAnalyzer.ExerciseTest` is able to generate static analysis tests and the analyze function loads the features to be tested.

## How to run the prototype
## How to run

### via CLI

#### Fast start

```shell
> git clone https://github.com/exercism/elixir-analyzer.git
[git output]
> cd elixir_analyzer

> ./bin/build
$ ./bin/build
[output of CLI being build]
> ./bin/elixir_analyzer <exercism slug for exercise> <root folder of solution to be analyzed> <folder to output contents>
$ ./bin/elixir_analyzer <exercise-slug> <path the folder containing the solution> <path to folder for output>
```

#### Running the analyzer

running `bin/elixir_analyzer` on a system with elixir/erlang/otp installed
Running `bin/elixir_analyzer` on a system with Elixir/Erlang/OTP installed

```text
Usage:
$ elixir_analyzer <exercise-name> <path the folder containing the solution> <path to folder for output> [options]
$ elixir_analyzer <exercise-slug> <path the folder containing the solution> <path to folder for output> [options]
You may also pass the following options:
--skip-analysis flag skips running the static analysis
Expand All @@ -45,12 +41,24 @@ running `bin/elixir_analyzer` on a system with elixir/erlang/otp installed

### via IEX

`iex -S mix`, then calling `ElixirAnalyzer.analyze_exercise("slug-name", "/path/to/solution/", "/path/to/output/")`.
`iex -S mix`, then calling:
```elixir
ElixirAnalyzer.analyze_exercise("exercise-slug", "/path/to/solution/", "/path/to/output/")
```

This assumes the solution has the file of the proper name and also a test unit by the proper name.

At this time "two-fer" is the only solution implemented at a most basic level. All that gets validated is if it passes the test unit completely. Goal will be to able to recognize the optimal solution through ast search for specific patterns.
To check that it works without preparing a custom exercise solution, you can run it on one of the text fixtures:

```elixir
ElixirAnalyzer.analyze_exercise("two-fer", "./test_data/two_fer/imperfect_solution/", "./test_data/two_fer/imperfect_solution/")
```

## Tests

As a demonstration, once iex loads with `iex -S mix` you can type `ElixirAnalyzer.analyze_exercise("two-fer", "./test_data/two_fer/passing_solution/", "./test_data/two_fer/passing_solution/")`.
The tests are run with `mix test`.

However, there are also tests tagged as `:external` which are excluded by default. Those tests check if all of the comments used in this repository exist in [`exercism/website-copy`][website-copy-comments]. To run all tests, use the `--include external` flag.

## Design

Expand Down Expand Up @@ -106,7 +114,7 @@ This module contains macros for a DSL to be able to compare ideal solution featu

### `ElixirAnalyzer.Constants`

Contains macro to generate a function returning the comment path on the `exercism/website-copy` repository. The current plan is to centralize these so that they can be easily tested and changed
Contains macro to generate a function returning the comment path on the [`exercism/website-copy`][website-copy-comments] repository.

### `ElixirAnalyzer.CLI`

Expand All @@ -115,3 +123,5 @@ This module is a module for the CLI escript to parse the command line arguments
### `ElixirAnalyzer.ExerciseTest.________`

These modules are for describing the tests for which the analyzer is able to determine the state of the solution based on style, syntax. They use `ElixirAnalyzer.Exercise` for macro generation of code.

[website-copy-comments]: https://github.com/exercism/website-copy/tree/main/analyzer-comments
6 changes: 4 additions & 2 deletions docs/step-01/step-01.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@

Requirements:

- See `mix.exs` for the required version of Elixir
- you may comment this out and run on an alternate version, but you may encounter problems
- See `mix.exs` for the required version of Elixir.
- You may comment this out and run on an alternate version, but you may encounter problems.
- Alternatively use [asdf][adsf].

1. Clone the repo to your desired location.
2. Make sure that you can build the _escript_ with `mix escript.build`.
3. Continue to [step 2][step-2]

[step-2]: ../step-02/step-02.md
[asdf]: https://asdf-vm.com/

0 comments on commit fda10a5

Please sign in to comment.