Guidance on Using Modem Built-in Stacks vs. Zephyr Stacks and Offloaded Sockets Implementation

[zafersn]

Apologies for asking this question here as well, as it was already raised on Discord. However, I wanted to have this question here for easier reference without cluttering other discussions.

I’m working on the HL7812 cellular modem, which provides built-in HTTP, MQTT, and LwM2M stacks. My goal is to use the modem’s stacks instead of Zephyr’s stacks to reduce code size and complexity. From my understanding, including AT commands should be sufficient to utilize these modem stacks.

However, That said, I came across a comment on Discord where someone mentioned that we “shouldn’t need to send MQTT-specific commands because the networking stack should handle all networking calls as long as an interface is up and available.”. This has left me a bit confused, and I’m not entirely sure what it means in practice.

I’m aware of the existing code for the HL7800 modem, which is similar to the HL7812. While it’s a useful reference for basic communication, it doesn’t seem to cover much functionality, and I’m unsure if it’s the best starting point.

Additionally, from a video shared on Discord (https://discord.com/channels/720317445772017664/905450375677636649/1325406611619188787), it seems the HL7800 driver uses offloaded networking rather than offloaded sockets. For my use case, it looks like I need to use offloaded sockets instead.

Overall, I’m not entirely sure where to start. Could anyone clarify the earlier comment regarding MQTT-specific commands and networking calls? Also, could you provide guidance on what I need to know, where to start, and whether there’s a similar modem driver I could look at? Finally, do you think it’s a good idea to rely on the modem’s built-in stacks instead of Zephyr’s stacks? What are the pros and cons of using the modem’s stacks vs Zephyr’s?

From my perspective, the pros seem to outweigh the cons, especially when considering the complexity of Zephyr’s MQTT, TLS stacks, and certificate management implementation, so these are the heavy implementations it requires on a host mcu.

What are your thoughts on this, and are there specific cases where Zephyr’s stacks might be more advantageous than the modem’s built-in stacks?

Thanks for your help!

[ndrs-pst]

@zafersn FYI, regarding a previous discussion, it seems that the keyword is 'MQTT Offloading.'
https://lists.zephyrproject.org/g/devel/topic/rfc_proposal_mqtt/95773752?p=

[zafersn]

@ndrs-pst Yes, I saw that discussion after creating this topic. He mentioned that he already implemented offloaded MQTT, but unfortunately, he hasn’t shared much information that could help me figure it out. I’m still investigating with that keyword.

However, as @rerickson1 also mentioned in his comment, it seems that offloading HTTP/MQTT stacks isn’t practically a great solution, and there doesn’t seem to be much in Zephyr to support it natively.

I’m currently studying how socket offloading works and how to implement socket offloading for MQTT in Zephyr. I recently came across the nRF Serial LTE Modem (SLM) MQTT commands documentation and am trying to understand it: nRF SLM MQTT Commands(https://docs.nordicsemi.com/bundle/ncs-latest/page/nrf/applications/serial_lte_modem/doc/MQTT_AT_commands.html).

[tomi-font]

Regarding the SLM MQTT commands, those are just proprietary AT commands. You won't find them in other modems.

[rerickson1]

@zafersn in my experience the TLS stack in the HL78xx is very limiting. It can only support a small amount of certificates and the certificate size they support is limiting.

Cons:

• HTTP, MQTT, LwM2M offloading are not supported in zephyr. Implementing this would be a lot of work and likely not benefit many others
• You could implement a custom modem APIs to used the modems HTTP, MQTT and LwM2M APIs. It would be custom and not beneficial.
• You rely on the modem firmware and if there are bugs, you are stuck. Using the zephyr stack is open source so you can fix bugs.

[zafersn]

@rerickson1 Thank you so much for your help! Your work on the HL7800 driver is fantastic and has been incredibly helpful to me. I’ve been using it with the HL7812 so far, But I've really hit a wall due to the limited RAM and flash memory on the host MCU.

My application includes several components: multiple sensors, the HL7812, a LoRa LR1121 modem (with LoRaWAN stacks), a bootloader, certificates, and settings partitions. Together, these take up the majority of the flash memory. To tackle this, I’m working to optimize the code and reduce its size. Migrating to the modem's built-in MQTT, HTTPs, and other stacks seemed like the most logical first step.

To be honest, I’ve also encountered some challenges with the driver itself. The most critical issue is that the delta modem update takes 45 minutes, which is unacceptable for my use case. I suspect this is due to the UART handler structure in the HL7800 driver. (In a test app, I hardcoded the delta fw package and was reading from the memory and transfering to the modem) Additionally, I faced another issue recently—after migrating Zephyr from 3.6 to 4.0, MQTT stopped working with the TLS configurations. I created an issue here for that a few days ago.

Additionally, the HL7812 has a 2G fallback RAT mode, and within a few months, Sierra Wireless is expected to release NTN satellite communication as a fourth RAT option for the HL7812. I’m planning to incorporate support for all these features into my implementation as well. Furthermore, I also need to extract more GNSS information, so implementing a GNSS NMEA parser is on my list.

As a result, I’ve decided to write or update the modem driver for the HL7812 to leverage its built-in offloaded MQTT and HTTP stacks while also addressing the UART handling for FOTA.

[ryan4xen]

I have been looking into something similar for the bg95. I noticed that zephyr has the following definitions within mqtt_transport.h

#if defined(CONFIG_MQTT_LIB_CUSTOM_TRANSPORT)
int mqtt_client_custom_transport_connect(struct mqtt_client *client);
int mqtt_client_custom_transport_write(struct mqtt_client *client, const uint8_t *data,
				uint32_t datalen);
int mqtt_client_custom_transport_write_msg(struct mqtt_client *client,
				    const struct msghdr *message);
int mqtt_client_custom_transport_read(struct mqtt_client *client, uint8_t *data,
			       uint32_t buflen, bool shall_block);
int mqtt_client_custom_transport_disconnect(struct mqtt_client *client);
#endif

These appear to allow for externally defined functions to be defined within the application and it simply passes the responsibility to the application/user space to define where these functions point to.

I'd assume that means I can define something like mqtt_client_custom_transport_connect to simply do an AT command for socket open on remote connection etc.

Does anyone see any downsides etc from this?

[ndrs-pst]

Hi @ryan4xen, I'm not sure about the downsides. Wouldn't it be more appropriate to follow this path instead:
drivers/modem/quectel-bg9x.c?

[ryan4xen]

Forgive me. Still quite new to the OFFLOAD concept. Would simply defining something like CONFIG_NET_SOCKETS_OFFLOAD move the socket implementation automatically over to modem without having to do the link manually?

[ndrs-pst]

You may start by looking at this example samples/net/mqtt_publisher.
To enable quectel-bg9x.c, you need to set CONFIG_MODEM_QUECTEL_BG9X=y and it will select NET_SOCKETS_OFFLOAD as per

zephyr/drivers/modem/Kconfig.quectel-bg9x

Lines 6 to 15 in e338657

	config MODEM_QUECTEL_BG9X
	bool "Quectel modem driver"
	select MODEM_CONTEXT
	select MODEM_CMD_HANDLER
	select MODEM_IFACE_UART
	select MODEM_SOCKET
	select NET_SOCKETS_OFFLOAD
	help
	Choose this setting to enable quectel BG9x LTE-CatM1/NB-IoT modem
	driver.

Since I can't find the dedicated example, or maybe @zafersn can suggest one 😃