Skip to content

Commit

Permalink
Merge pull request #611 from AutoResearch/docs/remove-pooler-sampler-…
Browse files Browse the repository at this point in the history
…distinction

docs: remove distinction between poolers and samplers in experimentalist.md
  • Loading branch information
musslick authored Jun 5, 2024
2 parents 8e3ed8a + 5411b47 commit 8ae9326
Showing 1 changed file with 7 additions and 63 deletions.
70 changes: 7 additions & 63 deletions docs/contribute/modules/experimentalist.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,6 @@ by an experiment runner. Experimentalists are generally implemented as functions

![Experimentalist Module](../../img/experimentalist.png)

Experimentalists can be implemented as *poolers* or as *samplers*.
- **Poolers** return a pool of candidate experimental conditions, which can be passed to a sampler that selects
a subset of conditions from the pool to be used in the next experiment.
- **Samplers** directly return a subset of experimental conditions from a pool of candidate experimental conditions that already exist.

## Repository Setup

We recommend using the [cookiecutter template](https://github.com/AutoResearch/autora-template-cookiecutter) to set up
Expand All @@ -22,80 +17,30 @@ a repository for your experimentalist. Alternatively, you can use the
cookiecutter https://github.com/AutoResearch/autora-template-cookiecutter
```

Make sure to select the `experimentalist` option when prompted. You may also select whether you want to implement an experimentalist as a sampler, pooler, or custom function. You can skip all other prompts pertaining to other modules
Make sure to select the `experimentalist` option when prompted. You can skip all other prompts pertaining to other modules
(e.g., experiment runners) by pressing enter.

## Implementation

Irrespective of whether you are implementing a pooler or a sampler,
you should implement a function that returns a set of experimental conditions. This set may be
For an experimentalist, you should implement a function that returns a set of experimental conditions. This set may be
a numpy array, iterator variable or other data format.

!!! hint
We generally **recommend using 2-dimensional numpy arrays as outputs** in which
each row represents a set of experimental conditions. The columns of the array correspond to the independent variables.

### Implementing Poolers

Once you've created your repository, you can implement your experimentalist pooler by editing the `init.py` file in
``src/autora/experimentalist/pooler/name_of_your_experimentalist/``.
Once you've created your repository, you can implement your experimentalist by editing the `init.py` file in
``src/autora/experimentalist/name_of_your_experimentalist/``.
You may also add additional files to this directory if needed.
It is important that the `init.py` file contains a function called `name_of_your_experimentalist`
which returns a pool of experimental conditions (e.g., as an iterator object or numpy array).

The following example ``init.py`` illustrates the implementation of a simple experimentalist pooler
that generates a grid of samples within the specified bounds of each independent variable (IV):

```python

"""
Example Experimentalist Pooler
"""

from itertools import product
from typing import List
from autora.variable import IV


def grid_pool(ivs: List[IV]):
"""
Creates exhaustive pool from discrete values using a Cartesian product of sets
Arguments:
ivs {List[IV]}: List of independent variables
which returns a set of experimental conditions (e.g., as a numpy array).

Returns:
pool: An iterator over all possible combinations of IV values
"""

l_iv_values = []
for iv in ivs:
assert iv.allowed_values is not None, (
f"gridsearch_pool only supports independent variables with discrete allowed values, "
f"but allowed_values is None on {iv=} "
)
l_iv_values.append(iv.allowed_values)

# Return Cartesian product of all IV values
return product(*l_iv_values)


```

### Implementing Samplers

Once you've created your repository, you can implement your experimentalist sampler by editing the `init.py` file in
``src/autora/experimentalist/sampler/name_of_your_experimentalist/``.
You may also add additional files to this directory if needed.
It is important that the `init.py` file contains a function called `name_of_your_experimentalist`
which returns a set of experimental conditions (e.g., as a numpy array) given a pool of candidate experimental conditions.

The following example ``init.py`` illustrates the implementation of a simple experimentalist sampler
The following example ``init.py`` illustrates the implementation of a simple experimentalist
that uniformly samples without replacement from a pool of candidate conditions.

```python
"""
Example Experimentalist Sampler
Example Experimentalist
"""

import random
Expand All @@ -120,7 +65,6 @@ random_sample(conditions: Union[Iterable, Sequence], n: int = 1):
return samples
```


## Next Steps: Testing, Documentation, Publishing

For more information on how to test, document, and publish your experimentalist, please refer to the
Expand Down

0 comments on commit 8ae9326

Please sign in to comment.