Smooth ReLU activation implementation as Triton kernel #196
Conversation
dtunai
commented
Apr 11, 2024
dtunai
commented
- Description: Introduces Smooth ReLU from Google Research as a Triton computation inside Zeta library
- Issue: None
- Dependencies:None
- Tag maintainer: @kyegomez
- Twitter handle:@simudt
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| @staticmethod | ||
| @triton.jit | ||
| def smooth_relu_activation_kernel( | ||
| x_ptr, output_ptr, n_elements, beta, BLOCK_SIZE: tl.constexpr |
Check warning
Code scanning / Pylintpython3 (reported by Codacy)
Argument name "BLOCK_SIZE" doesn't conform to snake_case naming style Warning
| @@ -50,6 +50,16 @@ | |||
| ) | |||
|
|
|||
|
|
|||
| def smooth_relu_activation(x: torch.Tensor, beta: float = 2.0): | |||
Check warning
Code scanning / Pylintpython3 (reported by Codacy)
Missing function or method docstring Warning
| @staticmethod | ||
| @triton.jit | ||
| def smooth_relu_activation_kernel( | ||
| x_ptr, output_ptr, n_elements, beta, BLOCK_SIZE: tl.constexpr |
Check warning
Code scanning / Pylint (reported by Codacy)
Wrong hanging indentation before block (add 4 spaces). Warning
| @@ -93,6 +93,27 @@ | |||
| output = tl.maximum(x, alpha * x) | |||
| tl.store(output_ptr + offsets, output, mask=mask) | |||
|
|
|||
| @staticmethod | |||
| @triton.jit | |||
| def smooth_relu_activation_kernel( | |||
Check warning
Code scanning / Pylint (reported by Codacy)
Argument name "BLOCK_SIZE" doesn't conform to snake_case naming style Warning
| block_st = idx * BLOCK_SIZE | ||
| offsets = block_st + tl.arange(0, BLOCK_SIZE) | ||
| mask = offsets < n_elements | ||
| x = tl.load(x_ptr + offsets, mask=mask) |
Check warning
Code scanning / Pylint (reported by Codacy)
Variable name "x" doesn't conform to snake_case naming style Warning
| @@ -50,6 +50,16 @@ | |||
| ) | |||
|
|
|||
|
|
|||
| def smooth_relu_activation(x: torch.Tensor, beta: float = 2.0): | |||
Check warning
Code scanning / Pylint (reported by Codacy)
Missing function docstring Warning
| @@ -50,6 +50,16 @@ | |||
| ) | |||
|
|
|||
|
|
|||
| def smooth_relu_activation(x: torch.Tensor, beta: float = 2.0): | |||
Check warning
Code scanning / Pylint (reported by Codacy)
Argument name "x" doesn't conform to snake_case naming style Warning
dtunai
changed the title
