Feature Request: allow custom client id, different from thing name

[antoine-sac]

Feature Request:

I sometimes run the AWS IoT Device Client alongside other software using another MQTT client. Typically, the AWS IoT Device Client takes care of jobs and secure tunnelling while other bits of software use device shadows directly with their own MQTT client.

This leads me to a situation where I can have 2 MQTT clients for the same thing, which means I need 2 different client ids. Currently I cannot choose the client ID used by the AWS IoT Device client: it is always the thing name (which is a good default).

I would like to be able to change the client ID independently from the thing name, which could be achieved with an optional CLIENT_ID env var, an optional --client-id flag, etc.

For example, --client-id ${thing_name}-aws-iot (assuming that the cert policy allows this name).

The default behaviour should of course still be to use the thing name as client id as it is a very sensible default.

[HarshGandhi-AWS]

Hello @antoine-sac , sorry for the delayed response.

I am not sure if it is a good idea to change the ClientId default from thing name to something else. Mainly because the topic to which device client subscribe to is derived from the thing name. Example: To get notification of the next job in the queue which device subscribes to $aws/things/thingName/jobs/notify-next topic. If we do not subscribe to this topic then we won't get notification for the new jobs created for the thing.

We can make a change to also allow passing thing-name separately so that we still subscribe to all feature required topics correctly. Will that be acceptable to you?

Regards,
Harsh Gandhi

[antoine-sac]

Hello Harsh,

You raise a very good point, indeed it'll be necessary to pass the thing-name too.

Alternatively, what I personally do in my code is pass a thing name and a client id suffix, then I construct the client ID as the thing name + suffix.

It's worth noting that it can be quite complicated to craft the appropriate device policy when you want to use the thing name in a topic filter, because AWS seems to derive the thing name from the client id when evaluating permissions...

The only way I managed to get restricted permissions to work is to actually create a second thing, using the same certificate. This way, each client has its own thing name.

[HarshGandhi-AWS]

Hey @antoine-sac , I am still working with our team to understand if there will be any difficulty/complications having client-id different from thing-name. Please be patient and I will share more updates soon.

I have a question regarding your other client: Device Client also has Shadow feature along with Jobs and Secure Tunneling. What is stopping you from using Device Clients Shadow feature?

[HarshGandhi-AWS]

Hello @antoine-sac , I spoke with our team and found out that the thing policy restricts devices with different thing-name and client-id from making multiple connections to IoT core at the same time. Also as you have mentioned earlier, it can be quite complicated to craft the appropriate device policy so for now we cannot work on this feature request for device client.

I think you have two options right now,

1. use different thing-name and policy for different clients running on your device.
2. use device client shadow feature.

Also it would help us to know what is stopping you from using device client's shadow feature. If you are able to use it, this problem will go away.

Regards,
Harsh Gandhi

[antoine-sac]

Hello @HarshGandhi-AWS,

I have come to the same conclusion: it's impossible to use a client-id different from the thing-name with a production-level device policy. We'll stick to our current approach of provisioning 2 things for the 2 clients

Regarding why we don't use the device client for everything, indeed the sensor publish feature and the shadow feature would cover our 2 use cases for using our own client.

We don't use them because:

1. The device client segfaults in recent builds which doesn't inspire confidence to use it for production-critical workloads. This has already been reported in Segmentation fault when initialize aws iot device client binary and also cannot compile the test binary #428 so I didn't open a ticket but we got a segfault on Debian 22.10 armhf and arm64 (raspberry pi). We actually had to roll back to a commit from last year to get a usable version.

2. Documentation is lacking, for example the only documentation of the device shadow feature is a 10-line paragraph here.. Limitations are not discussed and we didn't want to discover them ourselves. The doc on the sensor publish feature is much better.

Example of basic info we'd need before considering using the shadow feature in production:

• What is the format that should be used in the file ? Having worked with shadows for a while, I can guess, but it's incredible that the info is not there. Do the files contain both desired and reported state, or is there only desired state in output and reported state in input ?
• Is it possible to do delta updates ?
• Does the output file contain delta data or the whole state data, is it similar to the documents topic which contains data + metadata ?
• Are the output and input files really files ? Or are they some type of socket ? Are they persisted if the client reboots ? Are they persisted if the device reboots ?
• Are there any caveats ?
• What's the best way to monitor the output file on a UNIX machine ? Can we attach a callback to handle state updates ? (this last point may be omitted since it's a generic UNIX question, but could still be useful in an FAQ)

It's actually much easier to implement our own shadow service using the shadow MQTT topics because those are extremely well documented.

3. A number of little things don't inspire high confidence in the device client, for example the fact that some remote jobs fail if attributes marked as optional in the AWS UI are not passed to them. For example, RestartApplication does not work if the pathToHandler is not passed, contrary to what the official docs say.

4. It was very easy to implement a simple publisher straight in our code. We had already implemented all the features we needed before considering the device client which is why we're not using the sensor publish feature.

5. Sending a Last Will is not supported or at least not documented and this is an essential feature for us

Lastly, I just want to say that despite the limitations above, we find the AWS IoT Device Client extremely useful and we're grateful that you're maintaining it. It's proven invaluable as a safeguard to allow us to send jobs and open secure tunnels to our devices when we trouble with our own services. You sound like you may be the equivalent of a product owner and there's always insight in feedback which is why I listed all the above.

Cheers
Antoine

[HarshGandhi-AWS]

Hello @antoine-sac , thank you for your feedback. Device Client was built as a sample implementation of client which will run on the device to connect with IoT core and use different IoT functionalities. This software was not intended to run on production device; we expect users to make a clone/fork of this repository and build their own version of the client suitable to their usecase. Building your own version of client should not be difficult if you use Device Client as a base.

Regarding segmentation faults and documentation, feel free to open a ticket and team will look through it and try to resolve it as soon as possible. Also if you have found the problem and know the fix, we encourage you to make an update via pull request and team will be more than happy to review and merge it. This software is built for the IoT community.

Regards,
Harsh Gandhi