Feature to support setting twin reported properties during simulation (…

…Azure#377) * Introducing IoT Hub dataplane RBAC support + various improvements (Azure#341) * Introducing IoT Hub dataplane RBAC support. * IoT Hub rbac support for most commands. * Removed deprecated nested IoT edge artifacts. * Removed deprecated show-connection-string artifacts. * --auth-type supports configurable defaults. * Improved IoT Hub test infrastructure. * Significant refactor of IoT Hub tests. * Various additional improvements. * All warnings from tests are now shown. * Uamqp integration with jwt auth. * Modify HISTORY.rst * Iotc command versioning (Azure#340) * add support for v1 and preview routes * update history file * address review comments * lint fixes * Add warning for qos deprecation and update contributing guide (Azure#342) * Add warning for qos deprecation and update contributing guide * removing version from deprecate_info * Integration tests command update in Contributing guide * Integrate TQDM to show progress bar when simulator sends d2c messages * remove unused loop varable * Update progress bar descriptionb * Iotc command ga (Azure#348) * remove preview tags * history updates * Use enum value instead of literal str. (Azure#349) * Managed identity support for device-identity import and export (Azure#344) * Fix for new identity parameter format * test updates * better differentiation of storage vars * updated to remove *all* user-identitites after storage tests until CLI core is patched * Update azext_metadata.json (Azure#351) * Increment version to v0.10.13 * Update README.md * Update HISTORY.rst * update twin reported properties during simulation * update unit tests * Add dataplane reset (Azure#352) * pr comments * update dt reset to only support deleting both * Remove unused import * styling updates * C2D messaging improvements. (Azure#354) * Remove hiding of warnings from sample pytest.ini * Digital Twin wait commands (Azure#345) * update to use wait * fix error status code * int testing * update help * Add Identity Storage Account ID param to sentinel values (Azure#355) * Using SDK Listener for Twin properties update * Module identity renew key (Azure#356) * initial changes * update params, help * add missing module to test * Word change * Update README.md * merge from remote * Pipeline updates (Azure#359) * Pipeline updates Added nightly build pipeline Added template to run tests against minimum supported AZ CLI * Pipeline parameter refactoring * Moved deprecated ubuntu images to `ubuntu-latest` Co-authored-by: Paymaun <[email protected]> * Structured mqtt formatting and eliminate dependency on six * remove indent property not needed any more * Check for conditionals before running test jobs (Azure#363) * Update d2c and simulate commands to return errors for non SaS devices - update help to reflect the change (Azure#346) * Add warning for qos deprecation and update contributing guide * removing version from deprecate_info * Integration tests command update in Contributing guide * Update help message for d2c command and mqtt simulation to indicate only sas auth is supported * Checks and error messages when non SAS devices are used for MQTT operations * Indentation update in help file * fix styling * Adding enum and updating test * Addressed comments * Support C2D Message decoding and Add TQDM for HTTP simulation * Update MQTT operations to run on web sockets * Remove duplicate test already in dev branch * Device connection is automatic now * Styling update * Feature to support setting twin reported properties during simulation * Remove extra quote * update twin reported properties during simulation * update unit tests * styling updates * Using SDK Listener for Twin properties update * Support C2D Message decoding and Add TQDM for HTTP simulation * Merging changes Co-authored-by: Paymaun <[email protected]> Co-authored-by: valluriraj <[email protected]> Co-authored-by: Ryan K <[email protected]> Co-authored-by: Paymaun Heidari <[email protected]> Co-authored-by: vilit1 <[email protected]>
avagraw · Jul 28, 2021 · bbd09be · bbd09be
1 parent b078c5e
commit bbd09be
Show file tree

Hide file tree

Showing 7 changed files with 136 additions and 31 deletions.
diff --git a/azext_iot/_help.py b/azext_iot/_help.py
@@ -880,7 +880,7 @@
                    be acknowledged with completion. For http simulation c2d acknowledgement is based on user
                    selection which can be complete, reject or abandon. Additionally, mqtt simulation is only
                    supported for symmetric key auth (SAS) based devices. The mqtt simulation also supports direct
-                   method invocation which can be acknowledged by a response status code and response payload
+                   method invocation which can be acknowledged by a response status code and response payload.
 
                    Note: The command by default will set content-type to application/json and content-encoding
                    to utf-8. This can be overriden.
@@ -893,6 +893,10 @@
       text: az iot device simulate -n {iothub_name} -d {device_id} --method-response-code 201 --method-response-payload '{"result":"Direct method successful"}'
     - name: Basic usage (mqtt) with sending direct method response status code and direct method response payload as path to local file
       text: az iot device simulate -n {iothub_name} -d {device_id} --method-response-code 201 --method-response-payload '../my_direct_method_payload.json'
+    - name: Basic usage (mqtt) with sending the initial state of device twin properties as raw json for the target device
+      text: az iot device simulate -n {iothub_name} -d {device_id} --init-reported-properties '{"reported_prop_1":"val_1", "reported_prop_2":val_2}'
+    - name: Basic usage (mqtt) with sending the initial state of device twin properties as as path to local file for the target device
+      text: az iot device simulate -n {iothub_name} -d {device_id} --init-reported-properties '../my_device_twin_reported_properties.json'
     - name: Basic usage (http)
       text: az iot device simulate -n {iothub_name} -d {device_id} --protocol http
     - name: Basic usage (http) with sending mixed properties

diff --git a/azext_iot/_params.py b/azext_iot/_params.py
@@ -604,6 +604,12 @@ def load_arguments(self, _):
             help="Payload to be returned when direct method is executed on device. Provide file path or raw json. "
             "Optional param, only supported for mqtt.",
         )
+        context.argument(
+            "init_reported_properties",
+            options_list=["--init-reported-properties", "--irp"],
+            help="Initial state of twin reported properties for the target device when the simulator is run. "
+            "Optional param, only supported for mqtt.",
+        )
 
     with self.argument_context("iot device c2d-message") as context:
         context.argument(

diff --git a/azext_iot/operations/_mqtt.py b/azext_iot/operations/_mqtt.py
@@ -13,7 +13,10 @@
 
 
 class mqtt_client(object):
-    def __init__(self, target, device_conn_string, device_id, method_response_code=None, method_response_payload=None):
+    def __init__(
+        self, target, device_conn_string, device_id,
+        method_response_code=None, method_response_payload=None, init_reported_properties=None
+    ):
         self.device_id = device_id
         self.target = target
         # The client automatically connects when we send/receive a message or method invocation
@@ -25,6 +28,7 @@ def __init__(self, target, device_conn_string, device_id, method_response_code=N
         self.device_client.on_twin_desired_properties_patch_received = self.twin_patch_handler
         self.printer = pprint.PrettyPrinter(indent=2)
         self.default_data_encoding = 'utf-8'
+        self.init_reported_properties = init_reported_properties
 
     def send_d2c_message(self, message_text, properties=None):
         message = Message(message_text)
@@ -105,8 +109,19 @@ def twin_patch_handler(self, patch):
             self.device_client.patch_twin_reported_properties(modified_properties)
 
     def execute(self, data, properties={}, publish_delay=2, msg_count=100):
+        from azext_iot.operations.hub import _iot_device_twin_update
         from tqdm import tqdm
+
         try:
+            if self.init_reported_properties:
+                twin_properties = {
+                    "properties": {
+                        "desired": self.init_reported_properties
+                    }
+                }
+
+                _iot_device_twin_update(self.target, self.device_id, twin_properties)
+
             for _ in tqdm(range(msg_count), desc='Device simulation in progress'):
                 self.send_d2c_message(message_text=data.generate(True), properties=properties)
                 sleep(publish_delay)

diff --git a/azext_iot/operations/hub.py b/azext_iot/operations/hub.py
@@ -1740,15 +1740,24 @@ def iot_device_twin_update(
     etag=None,
     auth_type_dataplane=None,
 ):
-    from azext_iot.common.utility import verify_transform
-
     discovery = IotHubDiscovery(cmd)
     target = discovery.get_target(
         hub_name=hub_name,
         resource_group_name=resource_group_name,
         login=login,
         auth_type=auth_type_dataplane,
     )
+    return _iot_device_twin_update(target, device_id, parameters, etag)
+
+
+def _iot_device_twin_update(
+    target,
+    device_id,
+    parameters,
+    etag=None,
+):
+    from azext_iot.common.utility import verify_transform
+
     resolver = SdkResolver(target=target)
     service_sdk = resolver.get_sdk(SdkType.service_sdk)
 
@@ -2500,7 +2509,8 @@ def iot_simulate_device(
     resource_group_name=None,
     login=None,
     method_response_code=None,
-    method_response_payload=None
+    method_response_payload=None,
+    init_reported_properties=None
 ):
     import sys
     import uuid
@@ -2531,6 +2541,8 @@ def iot_simulate_device(
             raise CLIError("'method-response-code' not supported, {} doesn't allow direct methods.".format(protocol_type))
         if method_response_payload:
             raise CLIError("'method-response-payload' not supported, {} doesn't allow direct methods.".format(protocol_type))
+        if init_reported_properties:
+            raise CLIError("'init-reported-properties' not supported, {} doesn't allow setting twin props".format(protocol_type))
 
     properties_to_send = _iot_simulate_get_default_properties(protocol_type)
     user_properties = validate_key_value_pairs(properties) or {}
@@ -2546,6 +2558,11 @@ def iot_simulate_device(
             method_response_payload, argument_name="method-response-payload"
         )
 
+    if init_reported_properties:
+        init_reported_properties = process_json_arg(
+            init_reported_properties, argument_name="init-reported-properties"
+        )
+
     class generator(object):
         def __init__(self):
             self.calls = 0
@@ -2580,7 +2597,8 @@ def http_wrap(target, device_id, generator, msg_interval, msg_count):
                 device_conn_string=device_connection_string,
                 device_id=device_id,
                 method_response_code=method_response_code,
-                method_response_payload=method_response_payload
+                method_response_payload=method_response_payload,
+                init_reported_properties=init_reported_properties
             )
             client_mqtt.execute(data=generator(), properties=properties_to_send, publish_delay=msg_interval, msg_count=msg_count)
         else:

diff --git a/azext_iot/tests/conftest.py b/azext_iot/tests/conftest.py
@@ -29,6 +29,7 @@
     "azext_iot.operations.hub._iot_hub_monitor_events"
 )
 path_iot_device_show = "azext_iot.operations.hub._iot_device_show"
+path_update_device_twin = "azext_iot.operations.hub._iot_device_twin_update"
 hub_entity = "myhub.azure-devices.net"
 
 instance_name = generate_generic_id()
@@ -171,6 +172,11 @@ def fixture_monitor_events_entrypoint(mocker):
     return mocker.patch(path_iot_hub_monitor_events_entrypoint)
 
 
+@pytest.fixture()
+def fixture_update_device_twin(mocker):
+    return mocker.patch(path_update_device_twin)
+
+
 @pytest.fixture()
 def fixture_iot_device_show_sas(mocker):
     device = mocker.patch(path_iot_device_show)

diff --git a/azext_iot/tests/iothub/test_iot_ext_unit.py b/azext_iot/tests/iothub/test_iot_ext_unit.py
@@ -1998,28 +1998,31 @@ def test_generate_sas_token(self):
 
 class TestDeviceSimulate:
     @pytest.fixture(params=[204])
-    def serviceclient(self, mocker, fixture_ghcs, fixture_sas, request, fixture_device, fixture_iot_device_show_sas):
+    def serviceclient(
+        self, mocker, fixture_ghcs, fixture_sas, request, fixture_device, fixture_iot_device_show_sas, fixture_update_device_twin
+    ):
         service_client = mocker.patch(path_service_client)
         service_client.return_value = build_mock_response(mocker, request.param, {})
         return service_client
 
     @pytest.mark.parametrize(
-        "rs, mc, mi, protocol, properties, mrc, mrp",
+        "rs, mc, mi, protocol, properties, mrc, mrp, irp",
         [
-            ("complete", 1, 1, "http", None, None, None),
-            ("reject", 1, 1, "http", None, None, None),
-            ("abandon", 2, 1, "http", "iothub-app-myprop=myvalue;iothub-messageid=1", None, None),
-            ("complete", 1, 1, "http", "invalidprop;content-encoding=utf-16", None, None),
-            ("complete", 1, 1, "http", "iothub-app-myprop=myvalue;content-type=application/text", None, None),
-            ("complete", 3, 1, "mqtt", None, None, None),
-            ("complete", 3, 1, "mqtt", "invalid", None, None),
-            ("complete", 2, 1, "mqtt", "myprop=myvalue;$.ce=utf-16", 201, None),
-            ("complete", 2, 1, "mqtt", "myprop=myvalue;$.ce=utf-16", None, "{'result':'method succeded'}"),
-            ("complete", 2, 1, "mqtt", "myinvalidprop;myvalidprop=myvalidpropvalue", 204, "{'result':'method succeded'}"),
+            ("complete", 1, 1, "http", None, None, None, None),
+            ("reject", 1, 1, "http", None, None, None, None),
+            ("abandon", 2, 1, "http", "iothub-app-myprop=myvalue;iothub-messageid=1", None, None, None),
+            ("complete", 1, 1, "http", "invalidprop;content-encoding=utf-16", None, None, None),
+            ("complete", 1, 1, "http", "iothub-app-myprop=myvalue;content-type=application/text", None, None, None),
+            ("complete", 3, 1, "mqtt", None, None, None, None),
+            ("complete", 3, 1, "mqtt", "invalid", None, None, None),
+            ("complete", 2, 1, "mqtt", "myprop=myvalue;$.ce=utf-16", 201, None, None),
+            ("complete", 2, 1, "mqtt", "myprop=myvalue;$.ce=utf-16", None, "{'result':'method succeded'}", None),
+            ("complete", 2, 1, "mqtt", "myinvalidprop;myvalidprop=myvalidpropvalue", 204, "{'result':'method succeded'}", None),
+            ("complete", 2, 1, "mqtt", "myinvalidprop;myvalidprop=myvalidpropvalue", None, None, "{'rep_1':'val1', 'rep_2':2}"),
         ],
     )
     def test_device_simulate(
-        self, serviceclient, mqttclient, rs, mc, mi, protocol, properties, mrc, mrp
+        self, serviceclient, mqttclient, rs, mc, mi, protocol, properties, mrc, mrp, irp
     ):
         from azext_iot.operations.hub import _iot_simulate_get_default_properties
 
@@ -2033,7 +2036,8 @@ def test_device_simulate(
             protocol_type=protocol,
             properties=properties,
             method_response_code=mrc,
-            method_response_payload=mrp
+            method_response_payload=mrp,
+            init_reported_properties=irp
         )
 
         properties_to_send = _iot_simulate_get_default_properties(protocol)
@@ -2074,19 +2078,22 @@ def test_device_simulate(
             assert serviceclient.call_count == 0
 
     @pytest.mark.parametrize(
-        "rs, mc, mi, protocol, exception, mrc, mrp",
+        "rs, mc, mi, protocol, exception, mrc, mrp, irp",
         [
-            ("complete", 2, 0, "mqtt", CLIError, None, None),
-            ("complete", 0, 1, "mqtt", CLIError, None, None),
-            ("reject", 1, 1, "mqtt", CLIError, None, None),
-            ("abandon", 1, 0, "http", CLIError, None, None),
-            ("complete", 0, 1, "http", CLIError, 201, None),
-            ("complete", 0, 1, "http", CLIError, None, "{'result':'method succeded'}"),
-            ("complete", 0, 1, "http", CLIError, 201, "{'result':'method succeded'}"),
+            ("complete", 2, 0, "mqtt", CLIError, None, None, None),
+            ("complete", 0, 1, "mqtt", CLIError, None, None, None),
+            ("complete", 1, 1, "mqtt", CLIError, 200, "invalid_method_response_payload", None),
+            ("complete", 1, 1, "mqtt", CLIError, None, None, "invalid_reported_properties_format"),
+            ("reject", 1, 1, "mqtt", CLIError, None, None, None),
+            ("abandon", 1, 0, "http", CLIError, None, None, None),
+            ("complete", 0, 1, "http", CLIError, 201, None, None),
+            ("complete", 0, 1, "http", CLIError, None, "{'result':'method succeded'}", None),
+            ("complete", 0, 1, "http", CLIError, 201, "{'result':'method succeded'}", None),
+            ("complete", 0, 1, "http", CLIError, None, None, "{'rep_prop_1':'val1', 'rep_prop_2':'val2'}"),
         ],
     )
     def test_device_simulate_invalid_args(
-        self, serviceclient, rs, mc, mi, protocol, exception, mrc, mrp
+        self, serviceclient, rs, mc, mi, protocol, exception, mrc, mrp, irp
     ):
         with pytest.raises(exception):
             subject.iot_simulate_device(
@@ -2098,8 +2105,8 @@ def test_device_simulate_invalid_args(
                 msg_interval=mi,
                 protocol_type=protocol,
                 method_response_code=mrc,
-                method_response_payload=mrp
-
+                method_response_payload=mrp,
+                init_reported_properties=irp
             )
 
     def test_device_simulate_http_error(self, serviceclient_generic_error):

diff --git a/azext_iot/tests/iothub/test_iot_messaging_int.py b/azext_iot/tests/iothub/test_iot_messaging_int.py
@@ -255,6 +255,55 @@ def test_uamqp_device_messaging(self):
             expect_failure=True,
         )
 
+    def test_mqtt_device_simulation_with_init_reported_properties(self):
+        device_count = 1
+        device_ids = self.generate_device_names(device_count)
+
+        self.cmd(
+            "iot hub device-identity create -d {} -n {} -g {}".format(
+                device_ids[0], LIVE_HUB, LIVE_RG
+            ),
+            checks=[self.check("deviceId", device_ids[0])],
+        )
+
+        from azext_iot.operations.hub import iot_simulate_device
+        from azext_iot._factory import iot_hub_service_factory
+        from azure.cli.core.mock import DummyCli
+
+        cli_ctx = DummyCli()
+        client = iot_hub_service_factory(cli_ctx)
+
+        twin_init_props = {'prop_1': 'val_1', 'prop_2': 'val_2'}
+        twin_props_json = json.dumps(twin_init_props)
+
+        iot_simulate_device(
+            client,
+            device_ids[0],
+            LIVE_HUB,
+            "complete",
+            "Testing init reported twin properties",
+            2,
+            5,
+            "mqtt",
+            None,
+            None,
+            None,
+            None,
+            None,
+            twin_props_json
+        )
+
+        # get device twin
+        result = self.cmd(
+            "iot hub device-twin show -d {} --login {}".format(
+                device_ids[0], self.connection_string
+            )
+        ).get_output_in_json()
+
+        assert result is not None
+        for key in twin_init_props:
+            assert result["properties"]["reported"][key] == twin_init_props[key]
+
     def test_mqtt_device_direct_method_with_custom_response_status_payload(self):
         device_count = 1
         device_ids = self.generate_device_names(device_count)