feat(esp-alloc): Add heap usage stats #2137

AnthonyGrondin · 2024-09-10T21:40:15Z

Thank you for your contribution!

We appreciate the time and effort you've put into this pull request.
To help us review it efficiently, please ensure you've gone through the following checklist:

Submission Checklist 📝

I have updated existing examples or added new ones (if applicable).
I have used cargo xtask fmt-packages command to ensure that all changed code is formatted correctly.
My changes were added to the CHANGELOG.md in the proper section.
I have added necessary changes to user code to the Migration Guide.
My changes are in accordance to the esp-rs API guidelines

Extra:

I have read the CONTRIBUTING.md guide and followed its instructions.

Pull Request Details 📖

Description

This PR provides a way to easily see the current heap usage of esp_alloc. This adds HeapRegion::stats() and EspHeap::stats(), to return RegionStats and HeapStats respectively.

Both also implement core::fmt::Display to easily pretty-print the heap usage as following:

println!("{}", esp_alloc::get_info!());

HEAP INFO
Size: 117760
Current usage: 47280
Max usage: 70060
Total freed: 103012
Total allocated: 150292
Memory Layout: 
Internal | ██████████████░░░░░░░░░░░░░░░░░░░░░ | Used: 47280 / Total: 117760 (Free: 70480)
Unused   | ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ |
Unused   | ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ |

Testing

I have tested with a few examples locally and with esp-mbedtls, and everything works fine.
I haven't tested with PSRAM as I don't have any.

bugadani · 2024-09-11T06:15:01Z

Fancy, but it should also support defmt, not everybody uses esp-println and making the user add Display2Format makes this somewhat less elegant.

bjoernQ · 2024-09-11T07:49:30Z

Looks nice.

Given this runs a small amount of code every time allloc/dealloc is called would it make sense to have that feature gated?

MabezDev

This looks lovely! I agree with @bjoernQ though that this should be an opt-in feature.

okhsunrog · 2024-09-11T19:12:45Z

@bugadani won't this work with defmt?
defmt::!println!("{}", esp_alloc::get_info!());

AnthonyGrondin · 2024-09-12T16:00:27Z

I was thinking about feature gating the whole get_info!() + HeapStats feature, but I realized that the extra computation in alloc/dealloc is only to keep track of internal heap stats. I added a new feature named internal-heap-stats (for the lack of better name), to cover this.

The trade-off is that we have a feature that will change the amount of fields returned by the struct, if a user calls stats(), but I think we do that in a few places already.

I also found a chip that has PSRAM, so I tested with using an internal heap and a PSRAM heap, and the output looks like the following:

HEAP INFO
Size: 2148352
Current usage: 512028
Max usage: 512028
Total freed: 0
Total allocated: 512028
Memory Layout: 
Internal | ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | Used: 28 / Total: 51200 (Free: 51172)
External | ████████░░░░░░░░░░░░░░░░░░░░░░░░░░░ | Used: 512000 / Total: 2097152 (Free: 1585152)
Unused   | ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ |

AnthonyGrondin · 2024-09-12T17:22:17Z

Fancy, but it should also support defmt, not everybody uses esp-println and making the user add Display2Format makes this somewhat less elegant.

I'm not used to defmt, can you tell what changes are required to make it work with defmt? The following was suggested by @okhsunrog in #2137 (comment):
defmt::!println!("{}", esp_alloc::get_info!());

The examples aren't set up to use defmt so I'll have to create a project that uses defmt.

…o!()` macro

… extra computation. - Introduce the `internal-heap-stats` feature for `esp-alloc`. - Add `esp_alloc::get_info!()` to `psram_quad` example to show usage and ensure coverage of the feature in tests.

bugadani · 2024-09-12T17:47:00Z

With the obvious syntax error, I believe the compiler would just complain about "HeapStats does not implement Format".

How you may do it.

This would also need to be feature gated, similar to how the rest of the crates have a demft feature. We can also do it later if you don't feel comfortable experimenting just for a different printing framework.

okhsunrog · 2024-09-12T18:42:12Z

@bugadani yeah, that was a typo, sorry. For some reason I also thought that core Debug can also be used for defmt's println. Seems like it's not the case.. After stydying the docs a bit I think now I get it right

esp-alloc/src/macros.rs

esp-alloc/src/lib.rs

…ing `HEAP.stats()`

jessebraham

Thanks, LGTM!

Comments have been addressed

MabezDev · 2024-09-18T13:00:02Z

esp-alloc/src/lib.rs

+#[cfg(feature = "defmt")]
+impl defmt::Format for HeapStats {
+    fn format(&self, fmt: defmt::Formatter) {
+        defmt::write!(fmt, "{}", defmt::Display2Format(self))


Whilst convenient, this will end up pulling in core::fmt for defmt users, could you just duplicate the fmt impl using the defmt::write! macro instead?

MabezDev

Sorry, not quite, we should manually impl defmt to avoid pulling in core::fmt machinery for end users

okhsunrog · 2024-09-19T10:15:28Z

@MabezDev maybe we can merge it as it is and I'll implement defmt correctly in a follow up PR?

MabezDev · 2024-09-20T12:40:54Z

@okhsunrog Could you prepare the PR based on Anthony's branch? Once both are ready I'll merge them.

okhsunrog · 2024-09-20T13:29:35Z

@MabezDev Sure, I'm create the PR tomorrow based on this branch

jessebraham · 2024-10-07T11:52:46Z

I'm converting this to a draft for now, no rush with this of course. Please let us know if you'd like any help wrapping this up.

okhsunrog · 2024-10-07T11:57:47Z

I got a trauma in the end of September and had a surgery on my elbow. I still intend to work on this as soon as I recover

jessebraham · 2024-10-07T13:09:56Z

Sorry to hear, get well soon :)

MabezDev · 2024-11-01T10:55:01Z

Any update here? It would be nice to get this merged :).

okhsunrog · 2024-11-01T11:36:27Z

@MabezDev I'll create a new PR this weekend, Saturday or Sunday and finish the defmt part

MabezDev requested changes Sep 11, 2024

View reviewed changes

AnthonyGrondin added 3 commits September 12, 2024 13:22

feat(esp-alloc): Add heap usage stats and provide `esp_alloc::get_inf…

8c95125

…o!()` macro

Add CHANGELOG.md entry

8adb6f7

refactor(esp-alloc): Feature gate internal memory usage that requires…

3132b98

… extra computation. - Introduce the `internal-heap-stats` feature for `esp-alloc`. - Add `esp_alloc::get_info!()` to `psram_quad` example to show usage and ensure coverage of the feature in tests.

AnthonyGrondin force-pushed the feat/heap-debug branch from c37f62b to 3132b98 Compare September 12, 2024 17:23

MabezDev previously requested changes Sep 17, 2024

View reviewed changes

esp-alloc/src/macros.rs Outdated Show resolved Hide resolved

esp-alloc/src/lib.rs Show resolved Hide resolved

refactor(esp-alloc): Remove get_info!() macro in favour of document…

e9da803

…ing `HEAP.stats()`

AnthonyGrondin force-pushed the feat/heap-debug branch from e58ee54 to e9da803 Compare September 17, 2024 16:27

Implement defmt::Format for HeapStats and RegionStats

62b1953

AnthonyGrondin requested a review from MabezDev September 17, 2024 18:26

AnthonyGrondin changed the title ~~feat(esp-alloc): Add heap usage stats and provide esp_alloc::get_info!() macro~~ feat(esp-alloc): Add heap usage stats Sep 17, 2024

bjoernQ approved these changes Sep 18, 2024

View reviewed changes

jessebraham approved these changes Sep 18, 2024

View reviewed changes

MabezDev reviewed Sep 18, 2024

View reviewed changes

jessebraham added this pull request to the merge queue Sep 18, 2024

MabezDev removed this pull request from the merge queue due to a manual request Sep 18, 2024

MabezDev requested changes Sep 18, 2024

View reviewed changes

AnthonyGrondin mentioned this pull request Oct 4, 2024

upgrade to esp-wifi 0.9.1 esp-rs/esp-mbedtls#51

Merged

jessebraham marked this pull request as draft October 7, 2024 11:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(esp-alloc): Add heap usage stats #2137

feat(esp-alloc): Add heap usage stats #2137

AnthonyGrondin commented Sep 10, 2024

bugadani commented Sep 11, 2024 •

edited

Loading

bjoernQ commented Sep 11, 2024

MabezDev left a comment

okhsunrog commented Sep 11, 2024

AnthonyGrondin commented Sep 12, 2024

AnthonyGrondin commented Sep 12, 2024

bugadani commented Sep 12, 2024

okhsunrog commented Sep 12, 2024

jessebraham left a comment

MabezDev Sep 18, 2024

MabezDev left a comment

okhsunrog commented Sep 19, 2024

MabezDev commented Sep 20, 2024 •

edited

Loading

okhsunrog commented Sep 20, 2024

jessebraham commented Oct 7, 2024

okhsunrog commented Oct 7, 2024

jessebraham commented Oct 7, 2024

MabezDev commented Nov 1, 2024

okhsunrog commented Nov 1, 2024

feat(esp-alloc): Add heap usage stats #2137

Are you sure you want to change the base?

feat(esp-alloc): Add heap usage stats #2137

Conversation

AnthonyGrondin commented Sep 10, 2024

Thank you for your contribution!

Submission Checklist 📝

Extra:

Pull Request Details 📖

Description

Testing

bugadani commented Sep 11, 2024 • edited Loading

bjoernQ commented Sep 11, 2024

MabezDev left a comment

Choose a reason for hiding this comment

okhsunrog commented Sep 11, 2024

AnthonyGrondin commented Sep 12, 2024

AnthonyGrondin commented Sep 12, 2024

bugadani commented Sep 12, 2024

okhsunrog commented Sep 12, 2024

jessebraham left a comment

Choose a reason for hiding this comment

MabezDev Sep 18, 2024

Choose a reason for hiding this comment

MabezDev left a comment

Choose a reason for hiding this comment

okhsunrog commented Sep 19, 2024

MabezDev commented Sep 20, 2024 • edited Loading

okhsunrog commented Sep 20, 2024

jessebraham commented Oct 7, 2024

okhsunrog commented Oct 7, 2024

jessebraham commented Oct 7, 2024

MabezDev commented Nov 1, 2024

okhsunrog commented Nov 1, 2024

bugadani commented Sep 11, 2024 •

edited

Loading

MabezDev commented Sep 20, 2024 •

edited

Loading