Improve test coverage (#26)

* Install pytest-pydocstyle

* Configure pytest to use pydocstyle

* Lock with DLS PyPi

* Ignore module & package docstrings

* Fix loading docstrings

* Fix configurable docstrings

* Fix topic_naming docstrings

* Fix singleton docstrings

* Fix byte_format docstrings

* Fix source docstrings

* Fix sink docstrings

* Fix state_interface docstrings

* Bump pydocstyle to 6.1.1

* Fix kafka docstrings

* Fix internal docstrings

* Fix slave docstrings

* Fix master docstrings

* Fix base docstrings

* Fix ticker docstrings

* Fix device_simulation docstrings

* Fix system_simulation docstrings

* Fix component docstrings

* Fix typedefs docstrings

* Fix lifetime_runnable docstrings

* Fix device docstrings

* Fix adapter docstrings

* Fix tcp docstrings

* Fix regex_command docstrings

* Fix command_interpreter docstrings

* Fix composed docstrings

* Fix remote_controlled docstrings

* Fix shutter docstrings

* Fix trampoline docstrings

* Fix _version_git docstrings

* Re-lock pipfile.lock

* Add cli docstrings

* Add D418 to pydocstyle ignore

* Fix event_router docstrings

* Refactor epicsadapter

* Update EpicsAdapter adapters

* Update example configs

* Fix base docstrings

* Fix shutter docstrings

* Fix cryostream docstrings

* Fix states docstrings

* Fix status docstrings

* Enable pydocstyle linting

* Add simple docstrings to pneumatic

* Make flake8 complient

* Added docstrings to femto.py; Changed 'records' function in femto, pneumatic and epicsadapter to 'on_db_load' to better reflect what it does; Updated Pipfile to include charset-normalizer

* Fixed some docstring formatting

* Fixed line lengths

* Update tickit/devices/pneumatic/pneumatic.py

Co-authored-by: Garry O'Donnell <[email protected]>

* Removed whitespace

* Changed a couple of docstrings to include args and return values

* Remove charset-normalizer

* Edited set_state function

* Fix typechecking error

* Basic epics adapter sanity checks

* Add tests for cli.py

* Remove unused import

* Add tests for CryostreamBase class

* Improved Exception handling, Improved code readability

Base Exception classes substituted for ValueErrors where appropriate.
Hard coded RunMode values removed in favor of using the RunMode enum.
Made TODO flags fully capitalised to properly integrate with IDE.

* Add check on 'status' attribute before accessing.

Previously there was no check on whether the 'status' attribue was set
before accessing using the 'get_status' method. Now the 'get_status'
method will use the 'set_status_format' method to set the attribute it
wants to access if it is not already set.

* Adding tests for Cryostream and Cryostream Adapter

* Make pydocstyle complient

* Use absolute difference to compare gas and target temperature

* Refactoring and rewriting test

* Change to 'margin of error' style test

* Add comments

* Improve test coverage of CryostreamBase

* Refactoring of attribute and improvements 'set_current' docstring.

Renamed _current attribute to _output_current to better reflect it's
interpretation.

Improved the docstring for the 'set_current' message. Before it seemed
like the 'current' parameter would be stored in the class' current
attribute. Whereas in fact the input parameter is multiplied by the
'gain' of the device first.

* Add tests for the femto class

* Add tests for Pneumatic and PneumaticAdapter

* Rename file and increase test coverage to 100%

* Remove unused imports

* Remove git merging artifacts

* Remove more git merge artifacts

* Improvements to tests for CryostreamBase

* Improve tests by using 'await' instead of 'asyncio.run'

* Fix broken test

* Add tests for EpicsAdapter

* Add attribute in test rather than change class

* Disable eq to make dataclass InputRecord hashable

This is required to be able to use InputRecords as a key to a dict.

* Fix lack of awaiting

Co-authored-by: Garry O'Donnell <[email protected]>

* Clean commented out code

* Make flake8 complient

* Freeze dataclass instead of removing __eq__

* Remove trailing '0' from conditional in CryostreamBase.ramp

A seeming flaw in the logic for ramp where it was impossible to set the gas flow to anything other than 10 was the result of a trailing 0 on the number 9000.

* Change tests for ramp to reflect changes to that method

* Update tests to reflect changes to 'ramp' method

* Make the mocking of softioc explicitly apply to the entire test suite.

Move the mocking of softioc to tests/conftest.py where it will
explicitly apply to all test. Mark femto as xfail for now. We know it
works in isolation but we need to figure out how to get it working when
run along with the other tests.

* Remove outdated comment

* Add typehint

* Patch all softioc methods

* Remove mocking of softioc module

* Undo Garry's meddling

* Refactor for flake8 complience

* Make test skip and add an explaination for the future

* Remove redundant 'xfail'

* Undo pointless tinkering

* Extract method from 'build_ioc'

* Test extracted method instead of old method

* Have mock cryostream get status return correct mock status

* Use assert_awaited for coroutines

* Change test to suppress asyncio warning

* Revert "Change test to suppress asyncio warning"

This reverts commit e085798d3dc729bb14dba0a067d3142abb13ac90.

* Fix CLI tests by mocking tickit.cli.run_all_forever

* Add tests for Source and Sink devices

* Add device and system simulation tests

* Remove merging artifacts

* Mock state producer and consumer

* Declump tests

* Separate testable logic into another function

* Add tests for TcpServer

* Rename fixture and tweak the test to attempt to improve coverage.

* Add docstring to "generate_handle_function"

* Add "asyncio.streams" classes to nitpick_ignore list.

The documentation for these classes was built with an out-of-date
version of Sphinx which may have caused them to become unavailable via
the intersphinx extension.

* Enhance test with more assertions

* Enhance test with more assertions

* Enhance tests with more assertions

* Remove unintensional commenting-out of theme setting.

That setting does not work locally and that change got committed
accidentally.

* Add tests for BaseScheduler

* Refactor for easier testing

* [WIP] Adding tests for MasterScheduler

I'm going to need these files at work next week so don't panic if they
break CI okay? ;)

* Add docstrings

* Substitute set comprehension for list to preserve order

* Add working tests for multiple methods of MasterScheduler

* Remove unneccessary imports

* Change test to reflect the changes made in #16

* Update system simulation tests

* Update tests for Sink device

Changes to how sink object are intantiated calls for updates to these
tests.

* Update to tests for Source device

Updates to how Source devices are instantiated requires that these tests
be updated.

* Add 'Optional' to typing as required by 'mypy'

The origins of this error are unknown but it is a trivial change which
can be included with this PR.

* Add tests for 'SlaveScheduler'

* Rename fixtures

* Split tests for handling interrupt and output messages.

* Rename fixtures

* Changes to 'MasterScheduler' and tests

* Change name of '_run_tick' to '_do_tick'
* Moved call to 'setup' from '_do_initial_tick' to 'run_forever'
* Added test for successful scheduling of interrupts while there are
  queued wakeups

* Mollify MyPy errors

* Replace 'Internal' state interfaces with base classes

* Remove commented-out code

* Enhance testing of logging call

* Make 'generage_handle_function' "private 😏"

* Remove the '_TestBaseScheduler' class and use 'patch.object' instead

* Fix Cli tests for 'skip-lock' installations

* Fix Cli tests for 'skip-lock' installations

* Add tests for 'loading.py'

* Change to reading 'current-monitor.yaml' to avoid import errors.

* Refactor 'run_forever' into an instance method

* Add tests for 'KafkaStateConsumer/Producer'

* Test byte format (#38)

* Added tests for ByteFormat

* Remove concrete classes in favour of Mocks

* Make 'run_forever' method "private"

Co-authored-by: O'Donnell, Garry (DLSLtd,RAL,LSCI) <garry.o'[email protected]>
Co-authored-by: O'Donnell, Garry (DLSLtd,RAL,LSCI) <[email protected]>
Co-authored-by: Ollie Copping <[email protected]>
Co-authored-by: Garry O'Donnell <[email protected]>

docs/conf.py

@@ -61,6 +61,8 @@
     ("py:class", "SimTime"),
     ("py:class", "immutables._map.Map"),
     ("py:class", "_asyncio.Task"),
+    ("py:class", "asyncio.streams.StreamReader"),
+    ("py:class", "asyncio.streams.StreamWriter"),
     ("py:class", "apischema.conversions.conversions.Conversion"),
     ("py:class", "apischema.conversions.conversions.LazyConversion"),
 ]

tests/adapters/servers/test_tcp.py

@@ -0,0 +1,90 @@
+from asyncio import StreamReader, StreamWriter
+from typing import AsyncIterable, Awaitable, Callable
+
+import pytest
+from mock import ANY, MagicMock, Mock, patch
+from mock.mock import AsyncMock, create_autospec
+
+from tickit.adapters.servers.tcp import TcpServer
+
+
+@pytest.fixture
+def tcp_server() -> TcpServer:
+    return TcpServer()
+
+
+@pytest.fixture
+def on_connect() -> Callable[[], AsyncIterable[bytes]]:
+    async def on_connect():
+        yield b"hello"
+
+    return on_connect
+
+
+@pytest.fixture
+def handler() -> Callable[[bytes], Awaitable[AsyncIterable[bytes]]]:
+    async def _handler(message: bytes) -> AsyncIterable[bytes]:
+        async def async_iterable() -> AsyncIterable[bytes]:
+            yield b"hello"
+            yield b""
+            yield b"blue"
+            yield b"world"
+            yield b""
+
+        return async_iterable()
+
+    return _handler
+
+
+@pytest.fixture
+def patch_start_server():
+    with patch("asyncio.start_server") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_stream_reader() -> Mock:
+    mock: MagicMock = create_autospec(StreamReader, instance=True)
+    mock.read = AsyncMock(side_effect=(b"hello", b"world", b""))
+    return mock
+
+
+@pytest.fixture
+def mock_stream_writer() -> Mock:
+    mock: MagicMock = create_autospec(StreamWriter, instance=True)
+    mock.is_closing = Mock(side_effect=[False, False, True])
+    mock.write = Mock()
+    mock.drain = AsyncMock()
+    return mock
+
+
+@pytest.mark.asyncio
+async def test_TcpServer_run_forever_method(
+    tcp_server: TcpServer,
+    on_connect: Callable[[], AsyncIterable[bytes]],
+    handler: Callable[[bytes], AsyncIterable[bytes]],
+    patch_start_server: Mock,
+):
+
+    mock_start_server = patch_start_server
+    mock_start_server.return_value = AsyncMock()
+
+    await tcp_server.run_forever(on_connect, handler)
+
+    mock_start_server.assert_awaited_once_with(ANY, tcp_server.host, tcp_server.port)
+    mock_start_server.return_value.serve_forever.assert_awaited_once()
+
+
+@pytest.mark.asyncio
+async def test_TcpServer_generate_handle_function_method(
+    tcp_server: TcpServer,
+    on_connect: Callable[[], AsyncIterable[bytes]],
+    handler: Callable[[bytes], AsyncIterable[bytes]],
+    mock_stream_reader: Mock,
+    mock_stream_writer: Mock,
+):
+    handle_function = tcp_server._generate_handle_function(on_connect, handler)
+
+    assert handle_function is not None
+
+    await handle_function(mock_stream_reader, mock_stream_writer)

tests/core/components/test_system_simulation.py

@@ -0,0 +1,112 @@
+import asyncio
+from typing import Any, Awaitable, Callable, Iterable
+
+import pytest
+from immutables import Map
+from mock import AsyncMock, Mock, patch
+from mock.mock import create_autospec
+
+from tickit.core.components.component import Component
+from tickit.core.components.system_simulation import (
+    SystemSimulation,
+    SystemSimulationComponent,
+)
+from tickit.core.state_interfaces.state_interface import StateConsumer, StateProducer
+from tickit.core.typedefs import (
+    Changes,
+    ComponentID,
+    ComponentPort,
+    Output,
+    PortID,
+    SimTime,
+)
+from tickit.utils.topic_naming import output_topic
+
+
+class MockStateConsumer:
+    @staticmethod
+    def __call__(callback: Callable[[Any], Awaitable[None]]) -> None:
+        return create_autospec(StateConsumer)
+
+
+class MockStateProducer:
+    @staticmethod
+    def __call__():
+        return create_autospec(StateProducer)
+
+
+@pytest.fixture
+def patch_scheduler() -> Iterable[Mock]:
+    spec = "tickit.core.components.system_simulation.SlaveScheduler"
+    with patch(spec, autospec=True) as mock:
+
+        def on_tick(time, changes):
+            pass
+
+        mock.return_value = AsyncMock()
+        dummy_output = (Changes(Map({PortID("84"): 84})), 3)
+        mock.return_value.on_tick = AsyncMock(spec=on_tick, return_value=dummy_output)
+        mock.return_value.setup
+        yield mock
+
+
+@pytest.fixture
+def system_simulation(patch_scheduler) -> Component:
+    system_simulation_config = SystemSimulation(
+        name=ComponentID("test_system_simulation"),
+        inputs={PortID("24"): ComponentPort(ComponentID("25"), PortID("23"))},
+        components=[],
+        expose={PortID("42"): ComponentPort(ComponentID("43"), PortID("44"))},
+    )
+
+    return system_simulation_config()
+
+
+def test_system_simulation_constructor(system_simulation: SystemSimulation):
+    pass
+
+
+@pytest.fixture
+def patch_asyncio() -> Iterable[Mock]:
+    with patch(
+        "tickit.core.components.system_simulation.asyncio", autospec=True
+    ) as mock:
+        yield mock
+
+
+@pytest.fixture
+def patch_run_all() -> Iterable[Mock]:
+    with patch(
+        "tickit.core.components.system_simulation.run_all", autospec=True
+    ) as mock:
+        mock.return_value = [asyncio.sleep(0)]
+        yield mock
+
+
+@pytest.mark.asyncio
+async def test_system_simulation_methods(
+    system_simulation: SystemSimulationComponent,
+    patch_run_all: Mock,
+):
+    """Test the 'run_forever' and 'on_tick' methods.
+
+    The 'on_tick' method depends on 'run_forever' being called first. Therefore tests
+    for both methods have been bundled together. The test succeeds if the mock state
+    interfaces of the system simulation are awaited once with the correct parameters.
+    """
+    await system_simulation.run_forever(
+        MockStateConsumer(), MockStateProducer()  # type: ignore
+    )
+
+    system_simulation.state_consumer.subscribe.assert_awaited_once()  # type: ignore
+    time = SimTime(0)
+    (
+        expected_changes,
+        expected_call_back,
+    ) = system_simulation.scheduler.on_tick.return_value  # type: ignore
+    await system_simulation.on_tick(time, expected_changes)
+
+    system_simulation.state_producer.produce.assert_awaited_once_with(  # type: ignore
+        output_topic(system_simulation.name),
+        Output(system_simulation.name, time, expected_changes, expected_call_back),
+    )

tests/core/management/schedulers/test_base_scheduler.py

@@ -0,0 +1,112 @@
+from typing import Any, Iterable
+
+import pytest
+from immutables import Map
+from mock import Mock, create_autospec, patch
+
+from tickit.core.management.event_router import Wiring
+from tickit.core.management.schedulers.base import BaseScheduler
+from tickit.core.state_interfaces.state_interface import StateConsumer, StateProducer
+from tickit.core.typedefs import (
+    Changes,
+    ComponentID,
+    Input,
+    Interrupt,
+    Output,
+    PortID,
+    SimTime,
+)
+
+
+@pytest.fixture
+def patch_base_schedule_interrupt_method() -> Iterable[Mock]:
+    with patch.object(BaseScheduler, "schedule_interrupt") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_wiring() -> Mock:
+    return create_autospec(Wiring, instance=True)
+
+
+@pytest.fixture
+def mock_state_consumer_type() -> Mock:
+    return create_autospec(StateConsumer, instance=False)
+
+
+@pytest.fixture
+def mock_state_producer_type() -> Mock:
+    return create_autospec(StateProducer, instance=False)
+
+
+@pytest.fixture
+def patch_ticker() -> Iterable[Mock]:
+    with patch("tickit.core.management.schedulers.base.Ticker", autospec=True) as mock:
+        yield mock
+
+
+@pytest.fixture
+@pytest.mark.asyncio
+async def base_scheduler(
+    mock_wiring: Mock,
+    mock_state_consumer_type,
+    mock_state_producer_type,
+    patch_ticker,
+    patch_base_schedule_interrupt_method,
+) -> BaseScheduler:
+    base_scheduler = BaseScheduler(  # type: ignore
+        mock_wiring, mock_state_consumer_type, mock_state_producer_type
+    )
+    await base_scheduler.setup()
+    return base_scheduler
+
+
+def test_base_scheduler_constructor_and_setup(base_scheduler):
+    pass
+
+
+@pytest.mark.asyncio
+async def test_base_scheduler_update_component_method(
+    base_scheduler: Any,
+):
+    _input = Input(
+        ComponentID("foo"), SimTime(0), Changes(Map({PortID("42"): "hello"}))
+    )
+    await base_scheduler.update_component(_input)
+    base_scheduler.state_producer.produce.assert_awaited_once()
+
+
+@pytest.mark.asyncio
+async def test_base_scheduler_handle_output_message(base_scheduler: Any):
+    message = Output(
+        ComponentID("bar"),
+        SimTime(42),
+        Changes(Map({PortID("42"): "world"})),
+        SimTime(88),
+    )
+    await base_scheduler.handle_message(message)
+    base_scheduler.ticker.propagate.assert_awaited_once_with(message)
+
+
+@pytest.mark.asyncio
+async def test_base_scheduler_handle_interrupt_message(base_scheduler: BaseScheduler):
+    message = Interrupt(ComponentID("foo"))
+    await base_scheduler.handle_message(message)
+    base_scheduler.schedule_interrupt.assert_called_once()  # type: ignore
+
+
+def test_base_scheduler_get_first_wakeups_method(base_scheduler: BaseScheduler):
+    expected_component = ComponentID("foo")
+    expected_when = SimTime(42)
+    base_scheduler.add_wakeup(expected_component, expected_when)
+
+    components, when = base_scheduler.get_first_wakeups()
+
+    assert components == {expected_component}
+    assert when == expected_when
+
+
+def test_base_scheduler_get_empty_wakeups(base_scheduler: BaseScheduler):
+    components, when = base_scheduler.get_first_wakeups()
+    assert components == set()
+    assert when is None