Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MQTT receive: Bug fix and avoid payload copy #309

Merged
merged 3 commits into from
Mar 15, 2024

Conversation

eikel
Copy link
Contributor

@eikel eikel commented Mar 14, 2024

See the individual commits for details.

Program size has increased by 488 bytes in my setup (only MQTT_ENABLE enabled in settings-override.h).

Building the dev branch before:

RAM:   [==        ]  18.1% (used 59308 bytes from 327680 bytes)
Flash: [===       ]  31.9% (used 2090733 bytes from 6553600 bytes)

Building with my changes:

RAM:   [==        ]  18.1% (used 59308 bytes from 327680 bytes)
Flash: [===       ]  31.9% (used 2091221 bytes from 6553600 bytes)

I have not yet tested my changes on a real device.

eikel added 3 commits March 14, 2024 23:01
xQueueSend will copy as many bytes to the queue as defined for the items
when the queue was initialized. gRfidCardQueue was initialized with an
item size of cardIdStringSize (= 13). If the MQTT payload would be
shorter, this would previously lead to an out of bounds access.
@tueddy tueddy merged commit 2505a84 into biologist79:dev Mar 15, 2024
1 check failed
@biologist79
Copy link
Owner

@eikel Along with d89bc7c you introduced std::string_view() for Mqtt.cpp. I'd like to know how this works :-)
In the past we've copied payload to heap because I recognized *payload gets orphaned somehow fast. Now when I unterstand string_view() right, there's no copy done. This having said, you have to make somehow sure, that payload out-lives the string_view object.
My guess: it is just luck that the problem didn't re-occur. Maybe because of the movement to ESP32-Arduino2.

Any thoughts on this?

@eikel
Copy link
Contributor Author

eikel commented Sep 19, 2024

You are right, no copy is done, as this is the purpose of a std::string_view. In Mqtt_ClientCallback, receivedString provides a C++ interface for working with the string given as payload and length. More or less, it is only syntactic sugar. I do not see why this extra heap copy was needed in the first place. The payload is only accessed inside Mqtt_ClientCallback and not stored elsewhere. I assume the MQTT framework in use should guarantee that the payload is available until Mqtt_ClientCallback returns, therefore it is safe to use it in that function.

@biologist79
Copy link
Owner

biologist79 commented Sep 19, 2024

As it turns out: your assumption is wrong :-)
https://github.com/knolleary/pubsubclient/blob/master/examples/mqtt_publish_in_callback/mqtt_publish_in_callback.ino#L34

Wasn't 100 % sure about it but just gave the example a view. Even the lib author recommends to make a string copy.

@eikel
Copy link
Contributor Author

eikel commented Sep 20, 2024

Okay, but the problem is the publish, as this modifies the string that you give it. So probably the cases where publishMqtt is called would need to make that copy, or maybe one can change it to something like this (to also use the parameter types to signal what is expected here):

/* Wrapper-functions for MQTT-publish
   The payload buffer will be overwritten whilst constructing the PUBLISH packet,
   therefore accept a std::string parameter by copy here.
*/
bool publishMqtt(const std::string_view topic, std::string payload, bool retained) {
#ifdef MQTT_ENABLE
	if (topic != "" && Mqtt_PubSubClient.connected()) {                        
		Mqtt_PubSubClient.publish(topic.data(), payload.c_str(), retained);
		// delay(100);
		return true;
	}
#endif

	return false;
}

I have not compiled it, but I hope you get the idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants