Merge development into master in prep for Release 1.0.3 (#122)

* fix fopen FILE* unsigned comparison bug

iotc_bsp_io_fs_open() in

src/bsp/platform/posix/iotc_bsp_io_fs_posix.c

sets resource_handle_out to the FILE* returned by fopen() if fopen()
doesn't fail. This in turn gets assigned to the resource manager's
context->resource_handle. Later, iotc_resource_manager_read() in

src/libiotc/io/fs/iotc_resource_manager.c

checks to make sure context->resource_handle is not less the
zero. But there's nothing about fopen() that ensures that the FILE*
returned by fopen() can't be a value less than zero (when used in a
signed comparison).

If iotc_resource_manager_read() is going to assume a value less than
0 indicates a BSP file io open error, then we need to replace the use
of fopen() in iotc_bsp_io_fs_posix.c with open().

* Fix User Guide Links to Headings

We must use all lower case, remove the colons, and substitute spaces
for hypens to get heading links working correctly.

See the code that creates links here:
https://github.com/jch/html-pipeline/blob/master/lib/html/pipeline/toc_filter.rb

* downcases the string
* removes anything that is not a letter, number, space or hyphen (see the source for how Unicode is handled)
* changes any space to a hyphen.
* if that is not unique, add "-1", "-2", "-3",... to make it unique

* Release notes update and in-code version increase to 1.0.3. (#123)

* Release notes update and in-code version increase to 1.0.3.

Co-authored-by: Philip Fisher <[email protected]>
Co-authored-by: Gus Class <[email protected]>
Co-authored-by: Mike V <[email protected]>
Co-authored-by: Arpad Tigyi <[email protected]>

Author: 5 people

RELEASE-NOTES.md

@@ -1,3 +1,13 @@
+# Google Cloud IoT Core Device SDK for Embedded C version  1.0.3
+#### April 28, 2021
+
+- Updated the POSIX BSP to use `open` instead of `fopen` for more consistent error detection, [#100](https://github.com/GoogleCloudPlatform/iot-device-sdk-embedded-c/pull/100) by [@pnfisher](https://github.com/pnfisher).
+- Fixed a non null-terminated issue in message.topic with ‘iotc_user_subscription_callback_’, [#111](https://github.com/GoogleCloudPlatform/iot-device-sdk-embedded-c/issues/111) by [@csobrinho](https://github.com/csobrinho).
+- Added overflow protection to ‘__iotc_calloc()’ calls, [#119](https://github.com/GoogleCloudPlatform/iot-device-sdk-embedded-c/pull/119)
+- Updates to the Zephyr Troubleshooting guide, [#104](https://github.com/GoogleCloudPlatform/iot-device-sdk-embedded-c/pull/104) by [@galz10](https://github.com/galz10).
+- Fixes to links within the User Guide, [#102](https://github.com/GoogleCloudPlatform/iot-device-sdk-embedded-c/pull/102) by [@mikevoyt](https://github.com/mikevoyt).
+- Thank you to all of our contributors!
+
 # Google Cloud IoT Core Device SDK for Embedded C version  1.0.2
 #### February 4, 2020
 

doc/user_guide.md

@@ -204,11 +204,11 @@ To connect to Cloud IoT Core, call **`iotc_connect()`**. This function enqueues
 * TLS handshaking and certificate validation
 * MQTT credential handshaking
 
-Note: the call to connect returns immediately. The connection operation is fulfilled on subsequent calls to the Device SDK's event processing functions, as described in [Step 3: Process Events](step-3-process-events).
+Note: the call to connect returns immediately. The connection operation is fulfilled on subsequent calls to the Device SDK's event processing functions, as described in [Step 3: Process Events](#step-3-process-events).
 
 #### Connect callback
 
-When the **`iotc_connect()`** function runs, the Device SDK initalizes a callback function. The callback function is invoked when a connection to Cloud IoT Core is established or when the connection is unsuccessful. The callback function is also invoked when an established connection is lost or shut down. See [Step 6: Disconnect and Shut Down](step-6-disconnect-and-shut-down) for details.
+When the **`iotc_connect()`** function runs, the Device SDK initalizes a callback function. The callback function is invoked when a connection to Cloud IoT Core is established or when the connection is unsuccessful. The callback function is also invoked when an established connection is lost or shut down. See [Step 6: Disconnect and Shut Down](#step-6-disconnect-and-shut-down) for details.
 
 ### Step 3: Process events
 
@@ -228,7 +228,7 @@ Incoming messages include the topic data as part of the callback parameters. Spe
 
 #### QoS level
 
-The [specified QoS level](#MQTT v3.1.1) is the maximum QoS level of incoming messages. For instance, if you set the QoS level to 0, QoS 1 messages are downgraded to QoS 0. 
+The [specified QoS level](#mqtt-v3.1.1) is the maximum QoS level of incoming messages. For instance, if you set the QoS level to 0, QoS 1 messages are downgraded to QoS 0. 
 
 The QoS level also affects the Device SDK's memory overhead. QoS 2 messages use the most memory overhead, while QoS 0 messages use the least.
 
@@ -250,17 +250,17 @@ This callback function is for environments with severe memory restrictions. For
 
 ### Step 6: Disconnect and shut down
 
-To disconnect from Cloud IoT Core, invoke the **`iotc_shutdown_connection()`** function. This function enqueues an event that cleanly closes the socket connection. After the connection is terminated, the Device SDK invokes the [connect callback](#Step 2: Connect) function.
+To disconnect from Cloud IoT Core, invoke the **`iotc_shutdown_connection()`** function. This function enqueues an event that cleanly closes the socket connection. After the connection is terminated, the Device SDK invokes the [connect callback](#step-2-connect) function.
 
 Note: Do not delete the context until the the connect callback is invoked.
 
 #### Disconnection types and reconnecting
 
 The Device SDK checks the parameters of the connect callback function to determine if a device was intentionally or incidentally disconnected. See the [**Connect callback**](#connect-callback) section for more information.
 
-If an error disconnects a device, the client application can safely and immediately call **`iotc_connect()`** from within the connect callback function itself, using the same context that was just disconnected. This ques a new [connection request](#Step 2: Connect).
+If an error disconnects a device, the client application can safely and immediately call **`iotc_connect()`** from within the connect callback function itself, using the same context that was just disconnected. This ques a new [connection request](#step-2-connect).
 
-If a client application intentionally closes a connection, it retains the [existing context](#Step 1: Create a context) and invokes the connect function again later.
+If a client application intentionally closes a connection, it retains the [existing context](#step-1-create-a-context) and invokes the connect function again later.
 
 #### Freeing memory and shutting down
 

src/bsp/platform/posix/iotc_bsp_io_fs_posix.c

@@ -25,6 +25,9 @@
 #include <memory.h>
 #include <stdio.h>
 #include <sys/stat.h>
+#include <sys/types.h>
+#include <fcntl.h>
+#include <unistd.h>
 
 /* The size of the buffer to be used for reads. */
 const size_t iotc_bsp_io_fs_buffer_size = 1024;
@@ -41,7 +44,7 @@ const size_t iotc_bsp_io_fs_buffer_size = 1024;
  * Database type of file handles and allocated memory chunks.
  */
 typedef struct iotc_bsp_io_fs_posix_file_handle_container_s {
-  FILE* posix_fp;
+  int posix_fd;
   uint8_t* memory_buffer;
   struct iotc_bsp_io_fs_posix_file_handle_container_s* __next;
 } iotc_bsp_io_fs_posix_file_handle_container_t;
@@ -102,11 +105,11 @@ static iotc_bsp_io_fs_state_t iotc_bsp_io_fs_posix_stat_2_iotc_bsp_io_fs_stat(
  * @return 1 if list element is the one with the matching fp 0 otherwise
  */
 static uint8_t iotc_bsp_io_fs_posix_file_list_cnd(
-    iotc_bsp_io_fs_posix_file_handle_container_t* list_element, FILE* fp) {
+    iotc_bsp_io_fs_posix_file_handle_container_t* list_element, int fd) {
   assert(NULL != list_element);
-  assert(NULL != fp);
+  assert(fd >= 0);
 
-  return (list_element->posix_fp == fp) ? 1 : 0;
+  return (list_element->posix_fd == fd) ? 1 : 0;
 }
 
 iotc_bsp_io_fs_state_t iotc_bsp_io_fs_stat(
@@ -153,12 +156,12 @@ iotc_bsp_io_fs_state_t iotc_bsp_io_fs_open(
   iotc_bsp_io_fs_posix_file_handle_container_t* new_entry = NULL;
   iotc_bsp_io_fs_state_t ret = IOTC_BSP_IO_FS_STATE_OK;
 
-  FILE* fp = fopen(resource_name,
-                   (open_flags & IOTC_BSP_IO_FS_OPEN_READ) ? "rb" : "wb");
+  int fd = open(resource_name,
+                (open_flags & IOTC_BSP_IO_FS_OPEN_READ) ? O_RDONLY : O_WRONLY);
 
   /* if error on fopen check the errno value */
   IOTC_BSP_IO_FS_CHECK_CND(
-      NULL == fp, iotc_bsp_io_fs_posix_errno_2_iotc_bsp_io_fs_state(errno),
+      fd < 0, iotc_bsp_io_fs_posix_errno_2_iotc_bsp_io_fs_state(errno),
       ret);
 
   /* allocate memory for the files database element */
@@ -167,23 +170,23 @@ iotc_bsp_io_fs_state_t iotc_bsp_io_fs_open(
   memset(new_entry, 0, sizeof(iotc_bsp_io_fs_posix_file_handle_container_t));
 
   /* store the posix file pointer */
-  new_entry->posix_fp = fp;
+  new_entry->posix_fd = fd;
 
   /* add the entry to the database */
   IOTC_LIST_PUSH_BACK(iotc_bsp_io_fs_posix_file_handle_container_t,
                       iotc_bsp_io_fs_posix_files_container, new_entry);
 
   /* make sure that the size is as expected. */
-  assert(sizeof(fp) == sizeof(iotc_bsp_io_fs_resource_handle_t));
+  assert(sizeof(fd) <= sizeof(iotc_bsp_io_fs_resource_handle_t));
 
   /* return fp as a resource handle */
-  *resource_handle_out = (iotc_bsp_io_fs_resource_handle_t)fp;
+  *resource_handle_out = (iotc_bsp_io_fs_resource_handle_t)fd;
 
   return ret;
 
 err_handling:
-  if (NULL != fp) {
-    fclose(fp);
+  if (fd < 0) {
+    close(fd);
   }
   iotc_bsp_mem_free(new_entry);
   *resource_handle_out = iotc_bsp_io_fs_init_resource_handle();
@@ -199,13 +202,13 @@ iotc_bsp_io_fs_state_t iotc_bsp_io_fs_read(
   }
 
   iotc_bsp_io_fs_state_t ret = IOTC_BSP_IO_FS_STATE_OK;
-  FILE* fp = (FILE*)resource_handle;
+  int fd = (int)resource_handle;
   int fop_ret = 0;
 
   iotc_bsp_io_fs_posix_file_handle_container_t* elem = NULL;
   IOTC_LIST_FIND(iotc_bsp_io_fs_posix_file_handle_container_t,
                  iotc_bsp_io_fs_posix_files_container,
-                 iotc_bsp_io_fs_posix_file_list_cnd, fp, elem);
+                 iotc_bsp_io_fs_posix_file_list_cnd, fd, elem);
 
   IOTC_BSP_IO_FS_CHECK_CND(NULL == elem, IOTC_BSP_IO_FS_RESOURCE_NOT_AVAILABLE,
                            ret);
@@ -218,16 +221,16 @@ iotc_bsp_io_fs_state_t iotc_bsp_io_fs_read(
   }
 
   /* let's set an offset */
-  fop_ret = fseek(fp, offset, SEEK_SET);
+  fop_ret = lseek(fd, offset, SEEK_SET);
 
   /* if error on fseek check errno */
   IOTC_BSP_IO_FS_CHECK_CND(
-      fop_ret != 0, iotc_bsp_io_fs_posix_errno_2_iotc_bsp_io_fs_state(errno),
+      fop_ret < 0 || (unsigned int)fop_ret != offset,
+      iotc_bsp_io_fs_posix_errno_2_iotc_bsp_io_fs_state(errno),
       ret);
 
   /* use the fread to read the file chunk */
-  fop_ret =
-      fread(elem->memory_buffer, (size_t)1, iotc_bsp_io_fs_buffer_size, fp);
+  fop_ret = read(fd, elem->memory_buffer, iotc_bsp_io_fs_buffer_size);
 
   /* if error on fread check errno */
   IOTC_BSP_IO_FS_CHECK_CND(
@@ -260,30 +263,31 @@ iotc_bsp_io_fs_state_t iotc_bsp_io_fs_write(
   }
 
   iotc_bsp_io_fs_state_t ret = IOTC_BSP_IO_FS_STATE_OK;
-  FILE* fp = (FILE*)resource_handle;
+  int fd = (int)resource_handle;
 
   iotc_bsp_io_fs_posix_file_handle_container_t* elem = NULL;
   IOTC_LIST_FIND(iotc_bsp_io_fs_posix_file_handle_container_t,
                  iotc_bsp_io_fs_posix_files_container,
-                 iotc_bsp_io_fs_posix_file_list_cnd, fp, elem);
+                 iotc_bsp_io_fs_posix_file_list_cnd, fd, elem);
 
   IOTC_BSP_IO_FS_CHECK_CND(NULL == elem, IOTC_BSP_IO_FS_RESOURCE_NOT_AVAILABLE,
                            ret);
 
   /* let's set the offset */
-  const int fop_ret = fseek(fp, offset, SEEK_SET);
+  const int fop_ret = lseek(fd, offset, SEEK_SET);
 
   /* if error on fseek check errno */
   IOTC_BSP_IO_FS_CHECK_CND(
-      fop_ret != 0, iotc_bsp_io_fs_posix_errno_2_iotc_bsp_io_fs_state(errno),
+      fop_ret < 0 || (unsigned int)fop_ret != offset,
+      iotc_bsp_io_fs_posix_errno_2_iotc_bsp_io_fs_state(errno),
       ret);
 
-  *bytes_written = fwrite(buffer, (size_t)1, buffer_size, fp);
+  *bytes_written = write(fd, buffer, buffer_size);
 
   /* if error on fwrite check errno */
   IOTC_BSP_IO_FS_CHECK_CND(
       buffer_size != *bytes_written,
-      iotc_bsp_io_fs_posix_errno_2_iotc_bsp_io_fs_state(ferror(fp)), ret);
+      iotc_bsp_io_fs_posix_errno_2_iotc_bsp_io_fs_state(errno), ret);
 
 err_handling:
 
@@ -297,13 +301,13 @@ iotc_bsp_io_fs_state_t iotc_bsp_io_fs_close(
   }
 
   iotc_bsp_io_fs_state_t ret = IOTC_BSP_IO_FS_STATE_OK;
-  FILE* fp = (FILE*)resource_handle;
+  int fd = (int)resource_handle;
   iotc_bsp_io_fs_posix_file_handle_container_t* elem = NULL;
   int fop_ret = 0;
 
   IOTC_LIST_FIND(iotc_bsp_io_fs_posix_file_handle_container_t,
                  iotc_bsp_io_fs_posix_files_container,
-                 iotc_bsp_io_fs_posix_file_list_cnd, fp, elem);
+                 iotc_bsp_io_fs_posix_file_list_cnd, fd, elem);
 
   /* if element not on the list return resource not available error */
   IOTC_BSP_IO_FS_CHECK_CND(NULL == elem, IOTC_BSP_IO_FS_RESOURCE_NOT_AVAILABLE,
@@ -313,7 +317,7 @@ iotc_bsp_io_fs_state_t iotc_bsp_io_fs_close(
   IOTC_LIST_DROP(iotc_bsp_io_fs_posix_file_handle_container_t,
                  iotc_bsp_io_fs_posix_files_container, elem);
 
-  fop_ret = fclose(fp);
+  fop_ret = close(fd);
 
   /* if error on fclose check errno */
   IOTC_BSP_IO_FS_CHECK_CND(

src/libiotc/iotc_version.h

@@ -1,4 +1,4 @@
-/* Copyright 2018-2020 Google LLC
+/* Copyright 2018-2021 Google LLC
  *
  * This is part of the Google Cloud IoT Device SDK for Embedded C.
  * It is licensed under the BSD 3-Clause license; you may not use this file
@@ -19,6 +19,6 @@
 
 #define IOTC_MAJOR 1
 #define IOTC_MINOR 0
-#define IOTC_REVISION 2
+#define IOTC_REVISION 3 
 
 #endif /* __IOTC_VERSION_H__ */