From be82c72af481fa31cc18589093e7734b8ada2d57 Mon Sep 17 00:00:00 2001 From: Edward Cormany Date: Tue, 6 Aug 2024 22:13:10 -0400 Subject: [PATCH 01/12] adding csv parameters --- api/docs/v2/parameters/defining.rst | 23 ++++++++++++++++++++++- api/docs/v2/parameters/using_values.rst | 2 ++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/api/docs/v2/parameters/defining.rst b/api/docs/v2/parameters/defining.rst index 6b596ec8a0a..bbbaac0db94 100644 --- a/api/docs/v2/parameters/defining.rst +++ b/api/docs/v2/parameters/defining.rst @@ -71,7 +71,9 @@ Within this function definition, call methods on ``parameters`` to define parame Types of Parameters =================== -The API supports four types of parameters: Boolean (:py:class:`bool`), integer (:py:class:`int`), floating point number (:py:class:`float`), and string (:py:class:`str`). It is not possible to mix types within a single parameter. +The API supports four types of parameters that correspond to Python built-in types: Boolean (:py:class:`bool`), integer (:py:class:`int`), floating point number (:py:class:`float`), and string (:py:class:`str`). It is not possible to mix types within a single parameter. + +In addition, starting in version 2.20, the API supports CSV files as parameters. All data contained in CSV parameters, including numeric data, is initially interpreted as strings. See :ref:`using-rtp-types` for more information on manipulating CSV values. Boolean Parameters ------------------ @@ -179,3 +181,22 @@ A common use for string display names is to provide an easy-to-read version of a During run setup, the technician can choose from a menu of the provided choices. .. versionadded:: 2.18 + +CSV Parameters +-------------- + +CSV parameters accept any valid comma-separated file. You don't need to specify the format of the data. Due to this flexibility, they do not have default values. + +Briefly describe the purpose of your CSV parameter when defining it. + +.. code-block:: + + parameters.add_csv_file( + variable_name="cherrypicking_wells", + display_name="Cherrypicking wells", + description="Table of labware, wells, and volumes to transfer." + ) + +Separately provide standard operating procedures or template files to the scientists and technicians who will create the tabular data your protocol relies on. + +.. versionadded:: 2.20 diff --git a/api/docs/v2/parameters/using_values.rst b/api/docs/v2/parameters/using_values.rst index 754300347c9..bd61e7af8b8 100644 --- a/api/docs/v2/parameters/using_values.rst +++ b/api/docs/v2/parameters/using_values.rst @@ -28,6 +28,8 @@ Then ``params`` will gain three attributes: ``params.dry_run``, ``params.sample_ You can also save parameter values to variables with names of your choosing. +.. _using-rtp-types: + Parameter Types =============== From 88dc07cfb70ca965a03b91ef150fff3d8baa6dfa Mon Sep 17 00:00:00 2001 From: Edward Cormany Date: Tue, 6 Aug 2024 22:20:20 -0400 Subject: [PATCH 02/12] how to choose a csv file --- api/docs/v2/parameters/defining.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/api/docs/v2/parameters/defining.rst b/api/docs/v2/parameters/defining.rst index bbbaac0db94..f1baa21449a 100644 --- a/api/docs/v2/parameters/defining.rst +++ b/api/docs/v2/parameters/defining.rst @@ -28,7 +28,7 @@ Depending on the :ref:`type of parameter `, you'll need to specify so - An optional longer explanation of what the parameter does, or how its values will affect the execution of the protocol. - Maximum 100 characters. * - ``default`` - - + - - The value the parameter will have if the technician makes no changes to it during run setup. * - ``minimum`` and ``maximum`` - @@ -185,7 +185,7 @@ During run setup, the technician can choose from a menu of the provided choices. CSV Parameters -------------- -CSV parameters accept any valid comma-separated file. You don't need to specify the format of the data. Due to this flexibility, they do not have default values. +CSV parameters accept any valid comma-separated file. You don't need to specify the format of the data. Due to this flexibility, they do not have default values. Separately provide standard operating procedures or template files to the scientists and technicians who will create the tabular data your protocol relies on. Briefly describe the purpose of your CSV parameter when defining it. @@ -197,6 +197,6 @@ Briefly describe the purpose of your CSV parameter when defining it. description="Table of labware, wells, and volumes to transfer." ) -Separately provide standard operating procedures or template files to the scientists and technicians who will create the tabular data your protocol relies on. +During run setup, the technician can use the Flex touchscreen to select from files already stored on the robot or on an attached USB drive. Or in the Opentrons App, they can choose any file on their computer. .. versionadded:: 2.20 From a9f678532d48c450c107202ad4853f1a7c9eb63e Mon Sep 17 00:00:00 2001 From: Edward Cormany Date: Tue, 6 Aug 2024 23:18:55 -0400 Subject: [PATCH 03/12] API reference entries --- api/docs/v2/new_protocol_api.rst | 4 ++++ api/src/opentrons/protocols/parameters/types.py | 1 - 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/api/docs/v2/new_protocol_api.rst b/api/docs/v2/new_protocol_api.rst index 0fd8deb4afb..d0501557b13 100644 --- a/api/docs/v2/new_protocol_api.rst +++ b/api/docs/v2/new_protocol_api.rst @@ -91,6 +91,10 @@ Useful Types .. automodule:: opentrons.types :members: PipetteNotAttachedError, Point, Location, Mount +.. autoclass:: opentrons.protocols.parameters.types.CSVParameter + :members: + + .. autodata:: opentrons.protocol_api.OFF_DECK :no-value: diff --git a/api/src/opentrons/protocols/parameters/types.py b/api/src/opentrons/protocols/parameters/types.py index 0e248d8e1c0..bfd41401b36 100644 --- a/api/src/opentrons/protocols/parameters/types.py +++ b/api/src/opentrons/protocols/parameters/types.py @@ -2,7 +2,6 @@ from .csv_parameter_interface import CSVParameter - PrimitiveAllowedTypes = Union[str, int, float, bool] AllAllowedTypes = Union[str, int, float, bool, bytes, None] UserFacingTypes = Union[str, int, float, bool, CSVParameter] From 017bbe5f544340c1ab4586962a2d8e4c437f316e Mon Sep 17 00:00:00 2001 From: Edward Cormany Date: Tue, 13 Aug 2024 16:38:00 -0400 Subject: [PATCH 04/12] accessing CSV data --- api/docs/v2/parameters/using_values.rst | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/api/docs/v2/parameters/using_values.rst b/api/docs/v2/parameters/using_values.rst index bd61e7af8b8..6c9558b1833 100644 --- a/api/docs/v2/parameters/using_values.rst +++ b/api/docs/v2/parameters/using_values.rst @@ -33,7 +33,7 @@ You can also save parameter values to variables with names of your choosing. Parameter Types =============== -Each attribute of ``params`` has the type corresponding to its parameter definition. Keep in mind the parameter's type when using its value in different contexts. +Each attribute of ``params`` has the type corresponding to its parameter definition (except CSV parameters; see :ref:`rtp-csv-data` below). Keep in mind the parameter's type when using its value in different contexts. Say you wanted to add a comment to the run log, stating how many samples the protocol will process. Since ``sample_count`` is an ``int``, you'll need to cast it to a ``str`` or the API will raise an error. @@ -45,6 +45,28 @@ Say you wanted to add a comment to the run log, stating how many samples the pro Also be careful with ``int`` types when performing calculations: dividing an ``int`` by an ``int`` with the ``/`` operator always produces a ``float``, even if there is no remainder. The :ref:`sample count use case ` converts a sample count to a column count by dividing by 8 — but it uses the ``//`` integer division operator, so the result can be used for creating ranges, slicing lists, and as ``int`` argument values without having to cast it in those contexts. +.. _rtp-csv-data: + +Manipulating CSV Data +===================== + +CSV parameters have their own :py:class:`~opentrons.protocols.parameters.types.CSVParameter` type, since they don't correspond to a built-in Python type. This class has properties and methods that let you access the CSV data in one of three ways: as a file handler, as a string, or as nested lists. + +The :py:obj:`.CSVParameter.file` parameter provides a `file handler object `_ that points to your CSV data. You can pass this object to functions of the built-in :py:obj:`csv` module, or to other modules you import, such as ``pandas``. + +The :py:obj:`.CSVParameter.contents` parameter returns the entire contents of the CSV file as a single string. You then need to parse the data yourself to extract the information you need. + +The :py:meth:`.CSVParameter.parse_as_csv` method returns CSV data in a structured format. Specifically, it is a list of list of strings. This lets you access any "cell" of your tabular data by row and column index. This example parses a runtime parameter named ``csv_data``, stores the parsed data as ``parsed_csv``, and then accesses different portions of the data:: + + parsed_csv = protocol.params.csv_data + parsed_csv[0] # first row (header, if present) + parsed_csv[1][2] # second row, third column + [row[1] for row in parsed_csv] # second column + +.. versionadded:: 2.20 + +Remember that, like all Python lists, the lists representing your CSVs are zero-indexed. + Limitations =========== From b6bdb1e2ece655a5950eeda5aada76fa79951d69 Mon Sep 17 00:00:00 2001 From: Edward Cormany Date: Thu, 15 Aug 2024 09:30:34 -0400 Subject: [PATCH 05/12] typo --- api/docs/v2/parameters/using_values.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/api/docs/v2/parameters/using_values.rst b/api/docs/v2/parameters/using_values.rst index 6c9558b1833..52a7aea2ab5 100644 --- a/api/docs/v2/parameters/using_values.rst +++ b/api/docs/v2/parameters/using_values.rst @@ -56,7 +56,7 @@ The :py:obj:`.CSVParameter.file` parameter provides a `file handler object Date: Thu, 15 Aug 2024 12:07:30 -0400 Subject: [PATCH 06/12] how to pick cherries --- .../v2/parameters/use_case_cherrypicking.rst | 173 ++++++++++++++++++ api/docs/v2/runtime_parameters.rst | 2 + 2 files changed, 175 insertions(+) create mode 100644 api/docs/v2/parameters/use_case_cherrypicking.rst diff --git a/api/docs/v2/parameters/use_case_cherrypicking.rst b/api/docs/v2/parameters/use_case_cherrypicking.rst new file mode 100644 index 00000000000..0323517111d --- /dev/null +++ b/api/docs/v2/parameters/use_case_cherrypicking.rst @@ -0,0 +1,173 @@ +:og:description: How to use a CSV parameter to perform cherrypicking in an Opentrons Python protocol. + +.. _use-case-cherrypicking: + +********************************** +Parameter Use Case – Cherrypicking +********************************** + +Cherrypicking is a common liquid handling task: pipetting liquid from only certain wells on a source plate and placing them in order on a destination plate. This use case demonstrates how to use a CSV runtime parameter to automate this process and to customize it on every run — without having to modify the Python protocol itself. + +In this simple example, the CSV will only control: + + - Source slot + - Source well + - Volume to transfer + +The destination labware and well order will remain fixed, to focus on using these three pieces of data with the :py:meth:`.transfer` function. In actual use, you can further customize pipetting behavior by adding more runtime parameters or by adding columns to your CSV file. + +Preparing the CSV +================= + +First, we need to set up the CSV parameter. The data format we expect for this protocol is simple enough to fully explain in the parameter's description. + +.. code-block:: python + + def add_parameters(parameters): + + parameters.add_csv_file( + variable_name="cherrypicking_wells", + display_name="Cherrypicking wells", + description=( + "Table with three columns:" + " source slot, source well," + " and volume to transfer in µL." + ) + ) + +Here is an example of a CSV file that fits this format, specifying three wells across two plates: + +.. code-block:: text + + source slot,source well,volume + D1,A1,50 + D1,C4,30 + D2,H1,50 + +The technician would select this, or another file with the same structure, during run setup. + +Our protocol will use the information contained in the selected CSV for loading labware in the protocol and the cherrypicking transfers themselves. We'll rely on the data being structured exactly this way, with a header row and the three columns in this order. + +Parsing the CSV +=============== + +To get the most out of the CSV data, we'll use the Python API's :py:meth:`.parse_as_csv` method to allow easy access to different portions of the data at different points in the protocol:: + + def run(protocol): + + well_data = protocol.params.cherrypicking_wells.parse_as_csv() + +Now ``well_data`` is a list with four elements, one for each row in the file. We'll use the rows in a ``for`` loop later in the protocol, when it's time to transfer liquid. + +Loading Source Labware +====================== + +First, we need to load the source labware. Let's assume that we always use Opentrons Tough PCR plates for both source and destination plates. Then we need to determine the locations for loading source plates from the first column of the CSV. This will have three steps: + + - Using a list comprehension to get data from the ``source slot`` column. + - Deduplicating the items in the column. + - Looping over the unique items to load the plates. + +First, we'll get all of the data from the first column of the CSV, using a list comprehension. Then we'll take a slice of the resulting list to remove the header:: + + source_slots = [row[0] for row in well_data][1::] + # ['D1', 'D1', 'D2'] + +Next, we'll get the unique items in the list by converting it to a :py:obj:`set` and back to a list:: + + unique_source_slots = list(set(source_slots)) + # ['D1', 'D2'] + +Finally, we'll loop over those slot names to load labware:: + + for slot in unique_source_slots:: + protocol.load_labware( + load_name="opentrons_96_wellplate_200ul_pcr_full_skirt", + location=slot + ) + +Note that loading labware in a loop like this doesn't assign each labware instance to a variable. That's fine, because we'll use :py:obj:`.ProtocolContext.deck` to refer to them by slot name later on. + +The entire start of the ``run()`` function, including a pipette and fixed labware (i.e., labware not affected by the CSV runtime parameter) will look like this:: + + from opentrons import protocol_api + + requirements = {"robotType": "Flex", "apiLevel": "2.20"} + + def add_parameters(parameters): + + parameters.add_csv_file( + variable_name="cherrypicking_wells", + display_name="Cherrypicking wells", + description=( + "Table with three columns:" + " source slot, source well," + " and volume to transfer in µL." + ) + ) + + def run(protocol: protocol_api.ProtocolContext): + well_data = protocol.params.cherrypicking_wells.parse_as_csv() + source_slots = [row[0] for row in well_data][1::] + unique_source_slots = list(set(source_slots)) + + # load tip rack in deck slot C1 + tiprack = protocol.load_labware( + load_name="opentrons_flex_96_tiprack_1000ul", location="C1" + ) + # attach pipette to left mount + pipette = protocol.load_instrument( + instrument_name="flex_1channel_1000", + mount="left", + tip_racks=[tiprack] + # load trash bin + trash = protocol.load_trash_bin("A3") + ) + # load destination plate in deck slot C2 + dest_plate = protocol.load_labware( + load_name="opentrons_96_wellplate_200ul_pcr_full_skirt", location="C2" + ) + # load source plates based on CSV data + for slot in unique_source_slots: + protocol.load_labware( + load_name="opentrons_96_wellplate_200ul_pcr_full_skirt", + location=slot + ) + +Picking the Cherries +==================== + +Now it's time to transfer liquid based on the data in each row of the CSV. + +Once again we'll start by slicing off the header row of ``well_data``. Each remaining row has the source slot, source well, and volume data that we can directly pass to :py:meth:`.transfer`. + +We also need to specify the destination well. We want the destinations to proceed in order according to :py:meth:`.Labware.wells`. To track this all in a single loop, we'll wrap our CSV data in an :py:obj:`.enumerate` item that will provide an index that increments each time through the loop. All together, the transfer loop looks like this:: + + for index, row in enumerate(well_data[1::]): + # get source location from CSV + source_slot = row[0] + source_well = row[1] + source_location = protocol.deck[source_slot][source_well] + + # get volume as a number + transfer_volume = float(row[2]) + + # get destination location from loop index + dest_location = dest_plate.wells()[index] + + # perform parameterized transfer + pipette.transfer( + volume=transfer_volume, + source=source_location, + dest=dest_location + ) + +Let's unpack this. For each time through the loop, we first build the source location from the first (``row[0]``) and second (``row[1]``) item in the row list. We then construct a complete location reference with reference to ``protocol.deck``. + +Next, we get the volume for the transfer. All CSV data is treated as strings, so we have to cast it to a floating point number. + +The last piece of information needed is the destination well. We take the index of the current iteration through the loop, and use that same index with respect to the ordered list of all wells on the destination plate. + +With all the information gathered and stored in variables, all that's left is to pass that information as the arguments of ``transfer()``. With our example file, this will execute three transfers. By using a different CSV at run time, this code could complete up to 96 transfers (at which point it would run out of both tips and destination wells). + +For more complex transfer behavior — such as setting location within the well — you could extend the CSV format and the associated code to work with additional data. And check out the `verified cherrypicking protocol `_ in the Opentrons Protocol Library for further automation based on CSV data, including loading different types of plates, automatically loading tip racks, and more. \ No newline at end of file diff --git a/api/docs/v2/runtime_parameters.rst b/api/docs/v2/runtime_parameters.rst index 71689eedb50..48d61aead14 100644 --- a/api/docs/v2/runtime_parameters.rst +++ b/api/docs/v2/runtime_parameters.rst @@ -12,6 +12,7 @@ Runtime Parameters parameters/using_values parameters/use_case_sample_count parameters/use_case_dry_run + parameters/use_case_cherrypicking parameters/style Runtime parameters let you define user-customizable variables in your Python protocols. This gives you greater flexibility and puts extra control in the hands of the technician running the protocol — without forcing them to switch between lots of protocol files or write code themselves. @@ -26,4 +27,5 @@ It continues with a selection of use cases and some overall style guidance. When - :ref:`Use case – sample count `: Change behavior throughout a protocol based on how many samples you plan to process. Setting sample count exactly saves time, tips, and reagents. - :ref:`Use case – dry run `: Test your protocol, rather than perform a live run, just by flipping a toggle. +- :ref:`Use case – cherrypicking `: Use a CSV file to specify locations for a simple cherrypicking protocol. - :ref:`Style and usage `: When you're a protocol author, you write code. When you're a parameter author, you write words. Follow this advice to make things as clear as possible for the technicians who will run your protocol. From b6a40aa0145bd48700848c5ee5731061a838c726 Mon Sep 17 00:00:00 2001 From: Edward Cormany Date: Thu, 15 Aug 2024 14:26:32 -0400 Subject: [PATCH 07/12] cleanup after docstrings move/conflict --- api/docs/v2/new_protocol_api.rst | 2 +- api/docs/v2/parameters/using_values.rst | 2 +- .../parameters/csv_parameter_interface.py | 24 +++++++++++++++---- 3 files changed, 21 insertions(+), 7 deletions(-) diff --git a/api/docs/v2/new_protocol_api.rst b/api/docs/v2/new_protocol_api.rst index d0501557b13..179cb3cba18 100644 --- a/api/docs/v2/new_protocol_api.rst +++ b/api/docs/v2/new_protocol_api.rst @@ -91,7 +91,7 @@ Useful Types .. automodule:: opentrons.types :members: PipetteNotAttachedError, Point, Location, Mount -.. autoclass:: opentrons.protocols.parameters.types.CSVParameter +.. autoclass:: opentrons.protocol_api.CSVParameter :members: diff --git a/api/docs/v2/parameters/using_values.rst b/api/docs/v2/parameters/using_values.rst index 52a7aea2ab5..94b95371494 100644 --- a/api/docs/v2/parameters/using_values.rst +++ b/api/docs/v2/parameters/using_values.rst @@ -50,7 +50,7 @@ Also be careful with ``int`` types when performing calculations: dividing an ``i Manipulating CSV Data ===================== -CSV parameters have their own :py:class:`~opentrons.protocols.parameters.types.CSVParameter` type, since they don't correspond to a built-in Python type. This class has properties and methods that let you access the CSV data in one of three ways: as a file handler, as a string, or as nested lists. +CSV parameters have their own :py:class:`.CSVParameter` type, since they don't correspond to a built-in Python type. This class has properties and methods that let you access the CSV data in one of three ways: as a file handler, as a string, or as nested lists. The :py:obj:`.CSVParameter.file` parameter provides a `file handler object `_ that points to your CSV data. You can pass this object to functions of the built-in :py:obj:`csv` module, or to other modules you import, such as ``pandas``. diff --git a/api/src/opentrons/protocols/parameters/csv_parameter_interface.py b/api/src/opentrons/protocols/parameters/csv_parameter_interface.py index 20627322547..2c48cbc3203 100644 --- a/api/src/opentrons/protocols/parameters/csv_parameter_interface.py +++ b/api/src/opentrons/protocols/parameters/csv_parameter_interface.py @@ -16,7 +16,10 @@ def __init__(self, contents: Optional[bytes], api_version: APIVersion) -> None: @property def file(self) -> TextIO: - """Returns the file handler for the CSV file.""" + """Returns the file handler for the CSV file. + + The file is treated as read-only, UTF-8-encoded text. + """ if self._file is None: text = self.contents temporary_file = NamedTemporaryFile("r+") @@ -30,7 +33,7 @@ def file(self) -> TextIO: @property def file_opened(self) -> bool: - """Return if a file handler has been opened for the CSV parameter.""" + """Returns ``True`` if a file handler is open for the CSV parameter.""" return self._file is not None @property @@ -45,10 +48,21 @@ def contents(self) -> str: def parse_as_csv( self, detect_dialect: bool = True, **kwargs: Any ) -> List[List[str]]: - """Returns a list of rows with each row represented as a list of column elements. + """Parses the CSV data and returns a list of lists. + + Each item in the parent list corresponds to a row in the CSV file. + If the CSV has a header, that will be the first row in the list: ``.parse_as_csv()[0]``. + + Each item in the child lists corresponds to a single cell within its row. + The data for each cell is represented as a string, even if it is numeric in nature. + Cast these strings to integers or floating point numbers, as appropriate, to use + them as inputs to other API methods. - If there is a header for the CSV that will be the first row in the list (i.e. `.rows()[0]`). - All elements will be represented as strings, even if they are numeric in nature. + :param detect_dialect: If ``True``, examine the file and try to assign it a + :py:class:`csv.Dialect` to improve parsing behavior. + :param kwargs: For advanced CSV handling, you can pass any of the + `formatting parameters `_ + accepted by :py:func:`csv.reader` from the Python standard library. """ rows: List[List[str]] = [] if detect_dialect: From dc3acb52e607ba0a451c4779389f7f52935f213e Mon Sep 17 00:00:00 2001 From: Edward Cormany Date: Thu, 15 Aug 2024 15:24:25 -0400 Subject: [PATCH 08/12] wack line breaks --- api/docs/v2/new_protocol_api.rst | 1 - api/src/opentrons/protocols/parameters/types.py | 1 + 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/api/docs/v2/new_protocol_api.rst b/api/docs/v2/new_protocol_api.rst index 179cb3cba18..a71ad5cf4a2 100644 --- a/api/docs/v2/new_protocol_api.rst +++ b/api/docs/v2/new_protocol_api.rst @@ -94,7 +94,6 @@ Useful Types .. autoclass:: opentrons.protocol_api.CSVParameter :members: - .. autodata:: opentrons.protocol_api.OFF_DECK :no-value: diff --git a/api/src/opentrons/protocols/parameters/types.py b/api/src/opentrons/protocols/parameters/types.py index bfd41401b36..0e248d8e1c0 100644 --- a/api/src/opentrons/protocols/parameters/types.py +++ b/api/src/opentrons/protocols/parameters/types.py @@ -2,6 +2,7 @@ from .csv_parameter_interface import CSVParameter + PrimitiveAllowedTypes = Union[str, int, float, bool] AllAllowedTypes = Union[str, int, float, bool, bytes, None] UserFacingTypes = Union[str, int, float, bool, CSVParameter] From 349606eb97fa1185142542400a4316db761a76a5 Mon Sep 17 00:00:00 2001 From: Edward Cormany Date: Thu, 15 Aug 2024 22:06:46 -0400 Subject: [PATCH 09/12] line edit; add default data tip --- api/docs/v2/parameters/defining.rst | 5 +++- .../v2/parameters/use_case_cherrypicking.rst | 24 +++++++++++-------- api/docs/v2/parameters/using_values.rst | 20 ++++++++++++++-- api/docs/v2/runtime_parameters.rst | 2 +- 4 files changed, 37 insertions(+), 14 deletions(-) diff --git a/api/docs/v2/parameters/defining.rst b/api/docs/v2/parameters/defining.rst index f1baa21449a..47907cc5dda 100644 --- a/api/docs/v2/parameters/defining.rst +++ b/api/docs/v2/parameters/defining.rst @@ -73,7 +73,7 @@ Types of Parameters The API supports four types of parameters that correspond to Python built-in types: Boolean (:py:class:`bool`), integer (:py:class:`int`), floating point number (:py:class:`float`), and string (:py:class:`str`). It is not possible to mix types within a single parameter. -In addition, starting in version 2.20, the API supports CSV files as parameters. All data contained in CSV parameters, including numeric data, is initially interpreted as strings. See :ref:`using-rtp-types` for more information on manipulating CSV values. +In addition, starting in version 2.20, the API supports CSV files as parameters. All data contained in CSV parameters, including numeric data, is initially interpreted as strings. See :ref:`rtp-csv-data` for more information. Boolean Parameters ------------------ @@ -199,4 +199,7 @@ Briefly describe the purpose of your CSV parameter when defining it. During run setup, the technician can use the Flex touchscreen to select from files already stored on the robot or on an attached USB drive. Or in the Opentrons App, they can choose any file on their computer. +.. note:: + The touchscreen and app currently limit you to selecting one CSV file per run. To match this limitation, the API will raise an error if you define more than one CSV parameter. + .. versionadded:: 2.20 diff --git a/api/docs/v2/parameters/use_case_cherrypicking.rst b/api/docs/v2/parameters/use_case_cherrypicking.rst index 0323517111d..4e7083e1d10 100644 --- a/api/docs/v2/parameters/use_case_cherrypicking.rst +++ b/api/docs/v2/parameters/use_case_cherrypicking.rst @@ -6,7 +6,7 @@ Parameter Use Case – Cherrypicking ********************************** -Cherrypicking is a common liquid handling task: pipetting liquid from only certain wells on a source plate and placing them in order on a destination plate. This use case demonstrates how to use a CSV runtime parameter to automate this process and to customize it on every run — without having to modify the Python protocol itself. +A common liquid handling task is `cherrypicking`: pipetting liquid from only certain wells on a source plate and placing them in order on a destination plate. This use case demonstrates how to use a CSV runtime parameter to automate this process and to customize it on every run — without having to modify the Python protocol itself. In this simple example, the CSV will only control: @@ -44,14 +44,14 @@ Here is an example of a CSV file that fits this format, specifying three wells a D1,C4,30 D2,H1,50 -The technician would select this, or another file with the same structure, during run setup. +The protocol will rely on the data being structured exactly this way, with a header row and the three columns in this order. The technician would select this, or another file with the same structure, during run setup. -Our protocol will use the information contained in the selected CSV for loading labware in the protocol and the cherrypicking transfers themselves. We'll rely on the data being structured exactly this way, with a header row and the three columns in this order. +Our protocol will use the information contained in the selected CSV for loading labware in the protocol and performing the cherrypicking transfers. Parsing the CSV =============== -To get the most out of the CSV data, we'll use the Python API's :py:meth:`.parse_as_csv` method to allow easy access to different portions of the data at different points in the protocol:: +We'll use the Python API's :py:meth:`.parse_as_csv` method to allow easy access to different portions of the CSV data at different points in the protocol:: def run(protocol): @@ -88,11 +88,14 @@ Finally, we'll loop over those slot names to load labware:: Note that loading labware in a loop like this doesn't assign each labware instance to a variable. That's fine, because we'll use :py:obj:`.ProtocolContext.deck` to refer to them by slot name later on. -The entire start of the ``run()`` function, including a pipette and fixed labware (i.e., labware not affected by the CSV runtime parameter) will look like this:: +The entire start of the ``run()`` function, including a pipette and fixed labware (i.e., labware not affected by the CSV runtime parameter) will look like this: + +.. code-block:: python + :substitutions: from opentrons import protocol_api - requirements = {"robotType": "Flex", "apiLevel": "2.20"} + requirements = {"robotType": "Flex", "apiLevel": "|apiLevel|"} def add_parameters(parameters): @@ -125,7 +128,8 @@ The entire start of the ``run()`` function, including a pipette and fixed labwar ) # load destination plate in deck slot C2 dest_plate = protocol.load_labware( - load_name="opentrons_96_wellplate_200ul_pcr_full_skirt", location="C2" + load_name="opentrons_96_wellplate_200ul_pcr_full_skirt", + location="C2" ) # load source plates based on CSV data for slot in unique_source_slots: @@ -141,7 +145,7 @@ Now it's time to transfer liquid based on the data in each row of the CSV. Once again we'll start by slicing off the header row of ``well_data``. Each remaining row has the source slot, source well, and volume data that we can directly pass to :py:meth:`.transfer`. -We also need to specify the destination well. We want the destinations to proceed in order according to :py:meth:`.Labware.wells`. To track this all in a single loop, we'll wrap our CSV data in an :py:obj:`.enumerate` item that will provide an index that increments each time through the loop. All together, the transfer loop looks like this:: +We also need to specify the destination well. We want the destinations to proceed in order according to :py:meth:`.Labware.wells`. To track this all in a single loop, we'll wrap our CSV data in an :py:obj:`.enumerate` object to provide an index that increments each time through the loop. All together, the transfer loop looks like this:: for index, row in enumerate(well_data[1::]): # get source location from CSV @@ -162,7 +166,7 @@ We also need to specify the destination well. We want the destinations to procee dest=dest_location ) -Let's unpack this. For each time through the loop, we first build the source location from the first (``row[0]``) and second (``row[1]``) item in the row list. We then construct a complete location reference with reference to ``protocol.deck``. +Let's unpack this. For each time through the loop, we build the source location from the first (``row[0]``) and second (``row[1]``) item in the row list. We then construct a complete location with respect to ``protocol.deck``. Next, we get the volume for the transfer. All CSV data is treated as strings, so we have to cast it to a floating point number. @@ -170,4 +174,4 @@ The last piece of information needed is the destination well. We take the index With all the information gathered and stored in variables, all that's left is to pass that information as the arguments of ``transfer()``. With our example file, this will execute three transfers. By using a different CSV at run time, this code could complete up to 96 transfers (at which point it would run out of both tips and destination wells). -For more complex transfer behavior — such as setting location within the well — you could extend the CSV format and the associated code to work with additional data. And check out the `verified cherrypicking protocol `_ in the Opentrons Protocol Library for further automation based on CSV data, including loading different types of plates, automatically loading tip racks, and more. \ No newline at end of file +For more complex transfer behavior — such as adjusting location within the well — you could extend the CSV format and the associated code to work with additional data. And check out the `verified cherrypicking protocol `_ in the Opentrons Protocol Library for further automation based on CSV data, including loading different types of plates, automatically loading tip racks, and more. \ No newline at end of file diff --git a/api/docs/v2/parameters/using_values.rst b/api/docs/v2/parameters/using_values.rst index 94b95371494..2556a6b3949 100644 --- a/api/docs/v2/parameters/using_values.rst +++ b/api/docs/v2/parameters/using_values.rst @@ -58,14 +58,30 @@ The :py:obj:`.CSVParameter.contents` parameter returns the entire contents of th The :py:meth:`.CSVParameter.parse_as_csv` method returns CSV data in a structured format. Specifically, it is a list of lists of strings. This lets you access any "cell" of your tabular data by row and column index. This example parses a runtime parameter named ``csv_data``, stores the parsed data as ``parsed_csv``, and then accesses different portions of the data:: - parsed_csv = protocol.params.csv_data + parsed_csv = protocol.params.csv_data.parse_as_csv() parsed_csv[0] # first row (header, if present) parsed_csv[1][2] # second row, third column [row[1] for row in parsed_csv] # second column .. versionadded:: 2.20 -Remember that, like all Python lists, the lists representing your CSVs are zero-indexed. +Like all Python lists, the lists representing your CSVs are zero-indexed. + +.. tip:: + + Remember that CSV parameters don't have default values. Accessing CSV data in any of the above ways will prevent protocol analysis from completing until you select a CSV file and confirm all runtime parameter values during run setup. + + You can use a try–except block to work around this and provide the data needed for protocol analysis. First, add ``from opentrons.protocol_api import RuntimeParameterRequiredError`` at the top of your protocol. Then catch the error like this:: + + try: + parsed_csv = protocol.params.csv_data.parse_as_csv() + except RuntimeParameterRequiredError: + parsed_csv = [ + ["source slot", "source well", "volume"], + ["D1", "A1", "50"], + ["D2", "B1", "50"], + ] + Limitations =========== diff --git a/api/docs/v2/runtime_parameters.rst b/api/docs/v2/runtime_parameters.rst index 48d61aead14..810902eb729 100644 --- a/api/docs/v2/runtime_parameters.rst +++ b/api/docs/v2/runtime_parameters.rst @@ -27,5 +27,5 @@ It continues with a selection of use cases and some overall style guidance. When - :ref:`Use case – sample count `: Change behavior throughout a protocol based on how many samples you plan to process. Setting sample count exactly saves time, tips, and reagents. - :ref:`Use case – dry run `: Test your protocol, rather than perform a live run, just by flipping a toggle. -- :ref:`Use case – cherrypicking `: Use a CSV file to specify locations for a simple cherrypicking protocol. +- :ref:`Use case – cherrypicking `: Use a CSV file to specify locations and volumes for a simple cherrypicking protocol. - :ref:`Style and usage `: When you're a protocol author, you write code. When you're a parameter author, you write words. Follow this advice to make things as clear as possible for the technicians who will run your protocol. From f24c13e7d8da385c7c64bc66fc9a41eff0e71501 Mon Sep 17 00:00:00 2001 From: Edward Cormany Date: Fri, 16 Aug 2024 14:00:02 -0400 Subject: [PATCH 10/12] Jeremy's feedback --- api/docs/v2/parameters/defining.rst | 2 +- api/docs/v2/parameters/use_case_cherrypicking.rst | 2 +- .../protocols/parameters/csv_parameter_interface.py | 7 ++++--- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/api/docs/v2/parameters/defining.rst b/api/docs/v2/parameters/defining.rst index 47907cc5dda..ea22e4b09a3 100644 --- a/api/docs/v2/parameters/defining.rst +++ b/api/docs/v2/parameters/defining.rst @@ -197,7 +197,7 @@ Briefly describe the purpose of your CSV parameter when defining it. description="Table of labware, wells, and volumes to transfer." ) -During run setup, the technician can use the Flex touchscreen to select from files already stored on the robot or on an attached USB drive. Or in the Opentrons App, they can choose any file on their computer. +During run setup, the technician can use the Flex touchscreen to choose a CSV file. They can choose from files on an attached USB drive, or from files already associated with the protocol and stored on the robot. Or in the Opentrons App, they can choose any file on their computer. .. note:: The touchscreen and app currently limit you to selecting one CSV file per run. To match this limitation, the API will raise an error if you define more than one CSV parameter. diff --git a/api/docs/v2/parameters/use_case_cherrypicking.rst b/api/docs/v2/parameters/use_case_cherrypicking.rst index 4e7083e1d10..dcfdef32edd 100644 --- a/api/docs/v2/parameters/use_case_cherrypicking.rst +++ b/api/docs/v2/parameters/use_case_cherrypicking.rst @@ -70,7 +70,7 @@ First, we need to load the source labware. Let's assume that we always use Opent First, we'll get all of the data from the first column of the CSV, using a list comprehension. Then we'll take a slice of the resulting list to remove the header:: - source_slots = [row[0] for row in well_data][1::] + source_slots = [row[0] for row in well_data][1:] # ['D1', 'D1', 'D2'] Next, we'll get the unique items in the list by converting it to a :py:obj:`set` and back to a list:: diff --git a/api/src/opentrons/protocols/parameters/csv_parameter_interface.py b/api/src/opentrons/protocols/parameters/csv_parameter_interface.py index 2c48cbc3203..a1b9e7b4df7 100644 --- a/api/src/opentrons/protocols/parameters/csv_parameter_interface.py +++ b/api/src/opentrons/protocols/parameters/csv_parameter_interface.py @@ -54,9 +54,10 @@ def parse_as_csv( If the CSV has a header, that will be the first row in the list: ``.parse_as_csv()[0]``. Each item in the child lists corresponds to a single cell within its row. - The data for each cell is represented as a string, even if it is numeric in nature. - Cast these strings to integers or floating point numbers, as appropriate, to use - them as inputs to other API methods. + The data for each cell is represented as a string. You may need to trim whitespace + or otherwise validate string contents before passing them as inputs to other API methods. + For numeric data, cast these strings to integers or floating point numbers, + as appropriate. :param detect_dialect: If ``True``, examine the file and try to assign it a :py:class:`csv.Dialect` to improve parsing behavior. From e7a64110997b9fed39a8badd7151b06fc2c3233f Mon Sep 17 00:00:00 2001 From: Edward Cormany Date: Fri, 16 Aug 2024 14:12:23 -0400 Subject: [PATCH 11/12] Joe's feedback --- api/docs/v2/parameters/defining.rst | 2 +- api/docs/v2/parameters/use_case_cherrypicking.rst | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/api/docs/v2/parameters/defining.rst b/api/docs/v2/parameters/defining.rst index ea22e4b09a3..eb8caf4c957 100644 --- a/api/docs/v2/parameters/defining.rst +++ b/api/docs/v2/parameters/defining.rst @@ -185,7 +185,7 @@ During run setup, the technician can choose from a menu of the provided choices. CSV Parameters -------------- -CSV parameters accept any valid comma-separated file. You don't need to specify the format of the data. Due to this flexibility, they do not have default values. Separately provide standard operating procedures or template files to the scientists and technicians who will create the tabular data your protocol relies on. +CSV parameters accept any valid comma-separated file. You don't need to specify the format of the data. Due to this flexibility, CSV parameters do not have default values. Separately provide standard operating procedures or template files to the scientists and technicians who will create the tabular data your protocol relies on. Briefly describe the purpose of your CSV parameter when defining it. diff --git a/api/docs/v2/parameters/use_case_cherrypicking.rst b/api/docs/v2/parameters/use_case_cherrypicking.rst index dcfdef32edd..75d2345edde 100644 --- a/api/docs/v2/parameters/use_case_cherrypicking.rst +++ b/api/docs/v2/parameters/use_case_cherrypicking.rst @@ -19,7 +19,7 @@ The destination labware and well order will remain fixed, to focus on using thes Preparing the CSV ================= -First, we need to set up the CSV parameter. The data format we expect for this protocol is simple enough to fully explain in the parameter's description. +To get started, let's set up the CSV parameter. The data format we expect for this protocol is simple enough to fully explain in the parameter's description. .. code-block:: python @@ -62,7 +62,7 @@ Now ``well_data`` is a list with four elements, one for each row in the file. We Loading Source Labware ====================== -First, we need to load the source labware. Let's assume that we always use Opentrons Tough PCR plates for both source and destination plates. Then we need to determine the locations for loading source plates from the first column of the CSV. This will have three steps: +We'll use the data from the ``source slot`` column as part of loading the source labware. Let's assume that we always use Opentrons Tough PCR plates for both source and destination plates. Then we need to determine the locations for loading source plates from the first column of the CSV. This will have three steps: - Using a list comprehension to get data from the ``source slot`` column. - Deduplicating the items in the column. From 502bbc6d044ecc8821767d2330977ed8717eebfc Mon Sep 17 00:00:00 2001 From: Edward Cormany Date: Tue, 20 Aug 2024 14:33:04 -0400 Subject: [PATCH 12/12] final review feedback --- api/docs/v2/parameters/using_values.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/api/docs/v2/parameters/using_values.rst b/api/docs/v2/parameters/using_values.rst index 2556a6b3949..c3265ae1ecd 100644 --- a/api/docs/v2/parameters/using_values.rst +++ b/api/docs/v2/parameters/using_values.rst @@ -52,7 +52,7 @@ Manipulating CSV Data CSV parameters have their own :py:class:`.CSVParameter` type, since they don't correspond to a built-in Python type. This class has properties and methods that let you access the CSV data in one of three ways: as a file handler, as a string, or as nested lists. -The :py:obj:`.CSVParameter.file` parameter provides a `file handler object `_ that points to your CSV data. You can pass this object to functions of the built-in :py:obj:`csv` module, or to other modules you import, such as ``pandas``. +The :py:obj:`.CSVParameter.file` parameter provides a read-only `file handler object `_ that points to your CSV data. You can pass this object to functions of the built-in :py:obj:`csv` module, or to other modules you import, such as ``pandas``. The :py:obj:`.CSVParameter.contents` parameter returns the entire contents of the CSV file as a single string. You then need to parse the data yourself to extract the information you need. @@ -69,7 +69,7 @@ Like all Python lists, the lists representing your CSVs are zero-indexed. .. tip:: - Remember that CSV parameters don't have default values. Accessing CSV data in any of the above ways will prevent protocol analysis from completing until you select a CSV file and confirm all runtime parameter values during run setup. + CSV parameters don't have default values. Accessing CSV data in any of the above ways will prevent protocol analysis from completing until you select a CSV file and confirm all runtime parameter values during run setup. You can use a try–except block to work around this and provide the data needed for protocol analysis. First, add ``from opentrons.protocol_api import RuntimeParameterRequiredError`` at the top of your protocol. Then catch the error like this::