Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add per-sample gradient norm computation as a functionality #724

Closed
wants to merge 1 commit into from
Closed

Add per-sample gradient norm computation as a functionality #724

wants to merge 1 commit into from

Conversation

iden-kalemaj
Copy link
Contributor

Summary:
Per-sample gradient norm is computed for Ghost Clipping, but it can be useful generally. Exposed it as a functionality.

...

loss.backward()
per_sample_norms  = model.per_sample_gradient_norms

Differential Revision: D68634969

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 27, 2025
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D68634969

iden-kalemaj pushed a commit to iden-kalemaj/opacus that referenced this pull request Jan 28, 2025
@EnayatUllah @facebook-github-bot
Add per-sample gradient norm computation as a functionality (pytorch#724
ba86c7f 
)

Summary:

Per-sample gradient norm is computed for Ghost Clipping, but it can be useful generally. Exposed it as a functionality.


```
...

loss.backward()
per_sample_norms  = model.per_sample_gradient_norms

```

Differential Revision: D68634969
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D68634969

iden-kalemaj pushed a commit to iden-kalemaj/opacus that referenced this pull request Feb 6, 2025
@EnayatUllah @facebook-github-bot
Add per-sample gradient norm computation as a functionality (pytorch#724
8359416 
)

Summary:

Per-sample gradient norm is computed for Ghost Clipping, but it can be useful generally. Exposed it as a functionality.


```
...

loss.backward()
per_sample_norms  = model.per_sample_gradient_norms

```

Reviewed By: iden-kalemaj

Differential Revision: D68634969
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D68634969

@EnayatUllah @facebook-github-bot
Add per-sample gradient norm computation as a functionality (pytorch#724
37c1756 
)

Summary:

Per-sample gradient norm is computed for Ghost Clipping, but it can be useful generally. Exposed it as a functionality.


```
...

loss.backward()
per_sample_norms  = model.per_sample_gradient_norms

```

Reviewed By: iden-kalemaj

Differential Revision: D68634969
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D68634969

@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 0d186a4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported Merged
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@iden-kalemaj @facebook-github-bot @EnayatUllah