Body part candidates can be saved

doc/demo_overview.md

@@ -162,11 +162,12 @@ Each flag is divided into flag name, default value, and description.
 - DEFINE_int32(scale_number,              1,              "Number of scales to average.");
 - DEFINE_double(scale_gap,                0.3,            "Scale gap between scales. No effect unless scale_number > 1. Initial scale is always 1. If you want to change the initial scale, you actually want to multiply the `net_resolution` by your desired initial scale.");
 
-5. OpenPose Body Pose Heatmaps
-- DEFINE_bool(heatmaps_add_parts,         false,          "If true, it will add the body part heatmaps to the final op::Datum::poseHeatMaps array, and analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps (program speed will decrease). Not required for our library, enable it only if you intend to process this information later. If more than one `add_heatmaps_X` flag is enabled, it will place then in sequential memory order: body parts + bkg + PAFs. It will follow the order on POSE_BODY_PART_MAPPING in `include/openpose/pose/poseParameters.hpp`.");
+5. OpenPose Body Pose Heatmaps and Part Candidates
+- DEFINE_bool(heatmaps_add_parts,         false,          "If true, it will fill op::Datum::poseHeatMaps array with the body part heatmaps, and analogously face & hand heatmaps to op::Datum::faceHeatMaps & op::Datum::handHeatMaps. If more than one `add_heatmaps_X` flag is enabled, it will place then in sequential memory order: body parts + bkg + PAFs. It will follow the order on POSE_BODY_PART_MAPPING in `src/openpose/pose/poseParameters.cpp`. Program speed will considerably decrease. Not required for OpenPose, enable it only if you intend to explicitly use this information later.");
 - DEFINE_bool(heatmaps_add_bkg,           false,          "Same functionality as `add_heatmaps_parts`, but adding the heatmap corresponding to background.");
 - DEFINE_bool(heatmaps_add_PAFs,          false,          "Same functionality as `add_heatmaps_parts`, but adding the PAFs.");
 - DEFINE_int32(heatmaps_scale,            2,              "Set 0 to scale op::Datum::poseHeatMaps in the range [-1,1], 1 for [0,1]; 2 for integer rounded [0,255]; and 3 for no scaling.");
+- DEFINE_bool(part_candidates,            false,          "Also enable `write_json` in order to save this information. If true, it will fill the op::Datum::poseCandidates array with the body part candidates. Candidates refer to all the detected body parts, before being assembled into people. Note that the number of candidates is equal or higher than the number of final body parts (i.e. after being assembled into people). The empty body parts are filled with 0s. Program speed will slightly decrease. Not required for OpenPose, enable it only if you intend to explicitly use this information.");
 
 6. OpenPose Face
 - DEFINE_bool(face,                       false,          "Enables face keypoint detection. It will share some parameters from the body pose, e.g. `model_folder`. Note that this will considerable slow down the performance and increse the required GPU memory. In addition, the greater number of people on the image, the slower OpenPose will be.");
@@ -209,9 +210,10 @@ Each flag is divided into flag name, default value, and description.
 - DEFINE_string(write_images,             "",             "Directory to write rendered frames in `write_images_format` image format.");
 - DEFINE_string(write_images_format,      "png",          "File extension and format for `write_images`, e.g. png, jpg or bmp. Check the OpenCV function cv::imwrite for all compatible extensions.");
 - DEFINE_string(write_video,              "",             "Full file path to write rendered frames in motion JPEG video format. It might fail if the final path does not finish in `.avi`. It internally uses cv::VideoWriter.");
-- DEFINE_string(write_keypoint,           "",             "Directory to write the people body pose keypoint data. Set format with `write_keypoint_format`.");
-- DEFINE_string(write_keypoint_format,    "yml",          "File extension and format for `write_keypoint`: json, xml, yaml & yml. Json not available for OpenCV < 3.0, use `write_keypoint_json` instead.");
-- DEFINE_string(write_keypoint_json,      "",             "Directory to write people pose data in *.json format, compatible with any OpenCV version.");
-- DEFINE_string(write_coco_json,          "",             "Full file path to write people pose data with *.json COCO validation format.");
-- DEFINE_string(write_heatmaps,           "",             "Directory to write body pose heatmaps in *.png format. At least 1 `add_heatmaps_X` flag must be enabled.");
+- DEFINE_string(write_json,               "",             "Directory to write OpenPose output in JSON format. It includes body, hand, and face pose keypoints, as well as pose candidates (if `--part_candidates` enabled).");
+- DEFINE_string(write_coco_json,          "",             "Full file path to write people pose data with JSON COCO validation format.");
+- DEFINE_string(write_heatmaps,           "",             "Directory to write body pose heatmaps in PNG format. At least 1 `add_heatmaps_X` flag must be enabled.");
 - DEFINE_string(write_heatmaps_format,    "png",          "File extension and format for `write_heatmaps`, analogous to `write_images_format`. For lossless compression, recommended `png` for integer `heatmaps_scale` and `float` for floating values.");
+- DEFINE_string(write_keypoint,           "",             "(Deprecated, use `write_json`) Directory to write the people pose keypoint data. Set format with `write_keypoint_format`.");
+- DEFINE_string(write_keypoint_format,    "yml",          "(Deprecated, use `write_json`) File extension and format for `write_keypoint`: json, xml, yaml & yml. Json not available for OpenCV < 3.0, use `write_keypoint_json` instead.");
+- DEFINE_string(write_keypoint_json,      "",             "(Deprecated, use `write_json`) Directory to write people pose data in JSON format, compatible with any OpenCV version.");

doc/release_notes.md

@@ -33,9 +33,9 @@ OpenPose Library - Release Notes
     7. It prints out the error description before throwing the exception (so that it is written on the Windows cmd).
     8. Highly improved documentation.
 2. Functions or parameters renamed:
-    1. Flag `write_pose` renamed as `write_keypoint` and it also applies to face and/or hands.
-    2. Flag `write_pose_json` renamed as `write_keypoint_json` and it also applies to face and/or hands.
-    3. Flag `write_pose_format` renamed as `write_keypoint_format` and it also applies to face and/or hands.
+    1. Flag `--write_pose` renamed as `--write_keypoint` and it also applies to face and/or hands.
+    2. Flag `--write_pose_json` renamed as `--write_keypoint_json` and it also applies to face and/or hands.
+    3. Flag `--write_pose_format` renamed as `--write_keypoint_format` and it also applies to face and/or hands.
     4. PoseSaver and its JSON variant renamed as KeypointSaver.
     5. PoseJsonCocoSaver renamed as CocoJsonSaver.
 3. Main bugs fixed:
@@ -55,22 +55,22 @@ OpenPose Library - Release Notes
     7. Rendering threshold for pose, face and hands becomes user-configurable.
     8. Check() functions give more feedback.
     9. WCocoJsonSaver finished and removed its 3599-image limit.
-    10. Added `camera_fps` so generated video will use that frame rate.
+    10. Added `--camera_fps` so generated video will use that frame rate.
     11. Reduced the number of printed information messages. Default logging priority threshold increased to Priority::Max.
     12. Google flags to OpenPose configuration parameters reader moved from each demo to utilities/flagsToOpenPose.
     13. Nms classes do not use `numberParts` for `Reshape`, they deduce the value.
     14. Improved documentation.
 2. Functions or parameters renamed:
     1. Render flags renamed in the demo in order to incorporate the CPU/GPU rendering.
-    2. Keypoints saved in JSON files (`write_keypoint_json`) are now saved as `pose_keypoints`, `face_keypoints`, `hand_left_keypoints`, and `hand_right_keypoints`. They all were previously saved as `body_parts`.
-    3. Flag `num_scales` renamed as `scale_number`.
-    4. All hand and pose flags renamed such as they start by `hand_` and `face_` respectively.
+    2. Keypoints saved in JSON files (`--write_keypoint_json`) are now saved as `pose_keypoints`, `face_keypoints`, `hand_left_keypoints`, and `hand_right_keypoints`. They all were previously saved as `body_parts`.
+    3. Flag `--num_scales` renamed as `--scale_number`.
+    4. All hand and pose flags renamed such as they start by `--hand_` and `--face_` respectively.
 3. Main bugs fixed:
     1. Fixed bug in Array::getConstCvMat() if mVolume=0, now returning empty cv::Mat.
     2. Fixed bug: `--process_real_time` threw error with webcam.
     3. Fixed bug: Face not working when input and output resolutions are different.
     4. Fixed some bugs that prevented debug version to run.
-    5. Face saved in JSON files were called `body_parts`. Now they are called `face_keypoints`.
+    5. Face saved in JSON files were called `--body_parts`. Now they are called `--face_keypoints`.
 
 
 
@@ -129,14 +129,14 @@ OpenPose Library - Release Notes
     9. OpenPose successfully compiles if the flags `USE_CAFFE` and/or `USE_CUDA` are not enabled, although it will give an error saying they are required.
     10. COCO JSON file outputs 0 as score for non-detected keypoints.
     11. Added example for OpenPose for user asynchronous output and cleaned all `tutorial_wrapper/` examples.
-    12. Added `-1` option for `net_resolution` in order to auto-select the best possible aspect ratio given the user input.
+    12. Added `-1` option for `--net_resolution` in order to auto-select the best possible aspect ratio given the user input.
     13. Net resolution can be dynamically changed (e.g. for images with different size).
     14. Added example to add functionality/modules to OpenPose.
-    15. Added `disable_multi_thread` flag in order to allow debug and/or highly reduce the latency (e.g. when using webcam in real-time).
+    15. Added `--disable_multi_thread` flag in order to allow debug and/or highly reduce the latency (e.g. when using webcam in real-time).
     16. Allowed to output images without any rendering.
 2. Functions or parameters renamed:
     1. OpenPose able to change its size and initial size dynamically:
-        1. Flag `resolution` renamed as `output_resolution`.
+        1. Flag `--resolution` renamed as `--output_resolution`.
         2. FrameDisplayer, GuiInfoAdder and Gui constructors arguments modified (gui module).
         3. OpOutputToCvMat constructor removed (core module).
         4. New Renders classes to split GpuRenderers from CpuRenderers.
@@ -151,14 +151,17 @@ OpenPose Library - Release Notes
 
 
 
-## Current version (future OpenPose 1.3.0)
+## Current version (future OpenPose 1.2.1)
 1. Main improvements:
     1. Heatmaps can be saved in floating format.
-    2. More efficient non-processing version (i.e. if all keypoint extractors are disabled, and only image extraction and display/saving operations are performed).
-    3. Heat maps scaling: Added `heatmaps_scale` to OpenPoseDemo, added option not to scale the heatmaps, and added custom `float` format to save heatmaps in floating format.
+    2. More efficient non-processing version (i.e., if all keypoint extractors are disabled, and only image extraction and display/saving operations are performed).
+    3. Heat maps scaling: Added `--heatmaps_scale` to OpenPoseDemo, added option not to scale the heatmaps, and added custom `float` format to save heatmaps in floating format.
     4. Detector of the number of GPU also considers the initial GPU index given by the user.
+    5. Added `--write_json` as new version of `--write_keypoint_json`. It includes the body part candidates (if enabled), as well as any extra information added in the future (e.g. person ID).
+    6. Body part candidates can be retrieved in op::Datum and saved with `--write_json`.
 2. Functions or parameters renamed:
     1. `PoseParameters` splitted into `PoseParameters` and `PoseParametersRender` and const parameters turned into functions for more clarity.
 3. Main bugs fixed:
     1. Render working on images > 4K (#324).
     2. Cleaned redundant arguments on `getAverageScore` and `getKeypointsArea`.
+    3. Slight speed up when heatmaps must be returned to the user (not doing a double copy anymore).