Create dedicated Neptune hook for metadata logging #10080

AleksanderWWW · 2023-04-04T08:33:08Z

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

We want to introduce NeptuneHook for users who want to log their metadata and models to Neptune.ai

Modification

Added NeptuneHook class and an example config that utilizes it

BC-breaking (Optional)

Does the modification introduce changes that break the backward-compatibility of the downstream repos?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

Automatic metadata logging to Neptune.ai
Adding model checkpoints to Neptune.ai

Checklist

Pre-commit or other linting tools are used to fix the potential lint issues.
The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness. (in progress)
If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMCls.
The documentation has been modified accordingly, like docstring or example tutorials. (in progress)

Co-authored-by: Wenwei Zhang <[email protected]>

…8176) modified: mmdet/utils/util_distribution.py Co-authored-by: Wenwei Zhang <[email protected]>

* Bug fix: open-mmlab#8131 * The better solution from chhluo * open-mmlab#8136 since we changed the filtering type from mask to indexes, we can no longer use ```keep.any()``` to check, because it does not work at index zero. But we can use ``` if keep.size == 0: if self.keep_empty: return None ``` * add a unit test for filtering PolygonMasks * W293 blank line contains whitespace Co-authored-by: Wenwei Zhang <[email protected]> Co-authored-by: m.stepanov <[email protected]>

* Update * Update

* fix typo * fix metafile

* update reppoints readme and metafile * update openimages metafile * update faster rcnn readme and metafile * update convnext readme and metafile * update guided_anchoring metafile * update groie metafile and readme * update reppoints readme and metafile * update metafile * update metafile * release ld and mask_rcnn models * update metafile * update regnet metafile * fix markdown format * Update README.md * Update README.md * Update README.md * Update README.md * update md format * release lad

* Update maskformer.py * Update maskformer_head.py * Update maskformer.py * fix lint * fix lint Co-authored-by: Cedric Luo <[email protected]>

* [Fix]: Fix WandbLogger Hook Error (open-mmlab#8273) * Delete qq_group_qrcode.jpg * Update README_zh-CN.md Co-authored-by: Ayush Thakur <[email protected]>

* tiny add * fix

…en-mmlab#8439) * fix runtime error of "+=" inplace operation in PyTorch 1.10 * minor fix

* Add changelog of v2.25.1 * update changelog

Release v2.25.1

* introduce 3.x in 2.x readme * introduce 3.x in 2.x readme * update

…b#8733)

* Upgrade onnxsim to 0.4.0 * Update pytorch2onnx.py * Update pytorch2onnx.py

* Add .github/workflow/stale.yml * modify prompt message in stale.yml * modify check strategy now, issues and prs with any of 'invalid', 'awaiting response' will be checked

Fix HTC link

the parameter of offset is not set as continuous will trigger the runtime error: offset must be continuous

…ab#9791)

- Modify descriptions - Add example - Add docs link

CLAassistant · 2023-04-04T08:36:17Z

All committers have signed the CLA.

AleksanderWWW · 2023-04-04T08:43:18Z

Inb4 the neptune_config.py file name will be changed to adhere to the convention. I just named it this way for now to make it easier for me to find it in the repo

AleksanderWWW · 2023-04-05T16:34:35Z

Also, I would like to ask whether anyone has any tips for adding predicted masks to the ground truth? Is there any ready utility present somewhere in the repo?

OpenMMLab-Assistant005 · 2023-04-17T03:10:25Z

Hi @AleksanderWWW ！We are grateful for your efforts in helping improve this open-source project during your personal time.
Welcome to join OpenMMLab Special Interest Group (SIG) private channel on Discord, where you can share your experiences, ideas, and build connections with like-minded peers. To join the SIG channel, simply message moderator— OpenMMLab on Discord or briefly share your open-source contributions in the #introductions channel and we will assist you. Look forward to seeing you there! Join us ：https://discord.gg/UjgXkPWNqA
If you have a WeChat account，welcome to join our community on WeChat. You can add our assistant ：openmmlabwx. Please add "mmsig + Github ID" as a remark when adding friends：）
Thank you again for your contribution❤

Mxbonn and others added 30 commits June 5, 2022 17:16

Add anchor pruning to projects documentation. (open-mmlab#8120)

8f3a593

Co-authored-by: Wenwei Zhang <[email protected]>

changed example hook name it was specified wrongly (open-mmlab#8118)

c1ab941

Co-authored-by: Wenwei Zhang <[email protected]>

Fix 1 GPU distributed training of cuda device specifying (open-mmlab#…

ee79605

…8176) modified: mmdet/utils/util_distribution.py Co-authored-by: Wenwei Zhang <[email protected]>

Add mim in extras_require (open-mmlab#8194)

3940957

[Fix] Fix mdformat version to support python3.6 (open-mmlab#8195)

c80f99c

* Update * Update

Clean pillow version check (open-mmlab#8229)

eeb0e80

[Enhance]: Add test commands of mim in CI (open-mmlab#8230)

8ee3caf

Fix testing commands of mim in CI (open-mmlab#8240)

2796771

* fix typo * fix metafile

Merge branch 'master' into dev

7471229

[Fix]: fix GPG key error (open-mmlab#8215)

e3d4519

[Docs] Update and fix reference (open-mmlab#8264)

2d9f81a

[Fix]: Fix WandbLogger Hook Error (open-mmlab#8273)

56e42e7

Update maskformer.py and Update maskformer_head.py (open-mmlab#8263)

a5f9ed3

* Update maskformer.py * Update maskformer_head.py * Update maskformer.py * fix lint * fix lint Co-authored-by: Cedric Luo <[email protected]>

Update qq group qrcode in README_zh-CN.md (open-mmlab#8370)

0f57742

* [Fix]: Fix WandbLogger Hook Error (open-mmlab#8273) * Delete qq_group_qrcode.jpg * Update README_zh-CN.md Co-authored-by: Ayush Thakur <[email protected]>

[Fix]: set new height and width on Mac OS (open-mmlab#8434)

29e1650

Add checkpoint file download link in doc (open-mmlab#8391)

b9f7c28

* tiny add * fix

[Fix] fix runtime error of "+=" inplace operation in PyTorch 1.10 (op…

0c9d05b

…en-mmlab#8439) * fix runtime error of "+=" inplace operation in PyTorch 1.10 * minor fix

[Fix] update mmcv max version in init to fix CI (open-mmlab#8440)

4c526f3

Add changelog of v2.25.1 (open-mmlab#8428)

065ef4d

* Add changelog of v2.25.1 * update changelog

Release v2.25.1

3b72b12

Release v2.25.1

[Doc]: introduce 3.x in 2.x readme (open-mmlab#8677)

df28da9

* introduce 3.x in 2.x readme * introduce 3.x in 2.x readme * update

[Doc] Fix the wrong shape comments of the mask2former head (open-mmla…

f8bbba2

…b#8733)

Upgrade onnxsim to at least 0.4.0 (open-mmlab#8383)

e62c01f

* Upgrade onnxsim to 0.4.0 * Update pytorch2onnx.py * Update pytorch2onnx.py

Add .github/workflow/stale.yml (open-mmlab#8445)

e9c621f

* Add .github/workflow/stale.yml * modify prompt message in stale.yml * modify check strategy now, issues and prs with any of 'invalid', 'awaiting response' will be checked

[Doc]: fix markdown version (open-mmlab#8408)

13c9683

Fix typo in HTC link (open-mmlab#8487)

d929c63

Fix HTC link

Fix DyDCNv2 RuntimeError (open-mmlab#8485)

e9ca9d3

the parameter of offset is not set as continuous will trigger the runtime error: offset must be continuous

Fix method doc (open-mmlab#8512)

80a1c7e

ZwwWayne and others added 21 commits January 28, 2023 19:57

Bump version to v2.28.0 (open-mmlab#9668)

b955832

[CI]: fix isort version in lint (open-mmlab#9685)

eb773be

[Fix] Enable to set float mlp_ratio in SwinTransformer (open-mmlab#8670)

4b02faf

[Fix]: add yolof init_cfg (open-mmlab#8243)

ea29a1e

[Fix] Fix import error that causes training failure. (open-mmlab#9694)

1b7d778

[CI]: Use official isort (open-mmlab#9700)

a47398f

Bump version to v2.28.1 (open-mmlab#9708)

c14dd6c

improve type annotations

afa81ee

[Doc]: fix readthedoc (open-mmlab#9729)

e7cedcd

[Fix] Fix WIDERFace SSD loss NAN. (open-mmlab#9734)

831ba64

[Doc]: Add twitter discord medium youtube link (open-mmlab#9774)

0379d4d

[Doc] Fix the configuration file and log path of centernet. (open-mml…

56a3485

…ab#9791)

[Doc]: update customize_runtime.md (open-mmlab#9797)

d8d03bf

[Maintain]: Update .owners.yml (open-mmlab#9818)

6a904c5

Bump version to v2.28.2 (open-mmlab#9820)

e9cae2d

apply code review suggestions

e1851a9

tweak NeptuneHook docstring

4f8b7d8

- Modify descriptions - Add example - Add docs link

apply code review suggestions

6fdae0d

fix docs and annotations

359cace

apply code review suggestions

b8d1b83

Merge branch 'open-mmlab:master' into aw/add_neptune_logger_hook

bdf1cf5

mm-assistant bot assigned hhaAndroid Apr 4, 2023

rename neptune config

5170271

AleksanderWWW marked this pull request as ready for review May 12, 2023 09:47

AleksanderWWW changed the base branch from 2.x to 3.x May 31, 2023 18:43

AleksanderWWW changed the base branch from 3.x to dev-3.x May 31, 2023 19:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create dedicated Neptune hook for metadata logging #10080

Create dedicated Neptune hook for metadata logging #10080

AleksanderWWW commented Apr 4, 2023 •

edited

Loading

CLAassistant commented Apr 4, 2023 •

edited

Loading

AleksanderWWW commented Apr 4, 2023

AleksanderWWW commented Apr 5, 2023

OpenMMLab-Assistant005 commented Apr 17, 2023

Create dedicated Neptune hook for metadata logging #10080

Are you sure you want to change the base?

Create dedicated Neptune hook for metadata logging #10080

Conversation

AleksanderWWW commented Apr 4, 2023 • edited Loading

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

CLAassistant commented Apr 4, 2023 • edited Loading

AleksanderWWW commented Apr 4, 2023

AleksanderWWW commented Apr 5, 2023

OpenMMLab-Assistant005 commented Apr 17, 2023

AleksanderWWW commented Apr 4, 2023 •

edited

Loading

CLAassistant commented Apr 4, 2023 •

edited

Loading