New ACLS systems ! Need feedback #523
Replies: 15 comments 23 replies
-
|
One way will be to have this kind of configurations : group:
- name: my-group-name
roles:
- topic/read
conditions:
- type: cluster
values:
- prd
- type: topic
values:
- "test\\.reader.*"
- type: group
values:
- "group\\.reader.*"
- "group\\.writer.*"
- type: connect-cluster
values:
- "connect\\.reader.*"
- type: connect-task
values:
- "connect\\.task.*I think you will be able to to do anythings you want, and it will be extensible in the future |
Beta Was this translation helpful? Give feedback.
-
|
One more standard ways will be : groups:
- name: admin
cluster: prd
topics:
- "topic.*prd.*"
groups:
- "group\\.reader.*"
- "group\\.writer.*"
connect-cluster:
- cluster-1
connect-task:
- "task\\.jdbc.*"
roles:
- topic/read
- topic/insert
- topic/delete
- topic/config/update |
Beta Was this translation helpful? Give feedback.
-
|
Any suggest ? |
Beta Was this translation helpful? Give feedback.
-
|
Let me expose some use case I might have in the next few months. For a given LDAP group, I want to be able :
I believe that solution 2 is clearer than solution 1. In both scenario my use-case is covered. akhq:
security:
groups:
- name: project_1-cluster1
cluster: cluster1
topics:
- "project_1.*"
- "fixed_topic_001"
roles:
- topic/read
- name: project_1-cluster2
cluster: cluster2
topics:
- "project_1.*"
- "fixed_topic_002"
roles:
- topic/read
ldap:
groups:
- name: LDAP-PROJECT-1
groups:
- project_1-cluster1
- project_1-cluster2But solution 2 seems not to address #456 |
Beta Was this translation helpful? Give feedback.
-
|
Maybe consider moving the This approach will make it much easier to read what is permitted, than tracing through a long list of global rules that have cluster-conditions sprinkled around. Of course this splitting may lead to duplicated rules, particularly for clusters that are mostly identical (ie. the same regexps make sense throughout and any cluster-side ACLs/RBACs are equivalent). To avoid making it a breaking change, the current, global approach could be kept as a default. |
Beta Was this translation helpful? Give feedback.
-
|
Hey, I'd be willing to contribute for this! I kind of like the way that acls are defined currently. The only modification I'd make is to bind any attributes defined to a specific group-role. Example yaml file: akhq:
security:
default-group: no-roles
groups:
dev-reader:
name: dev-reader # an example of a dev user that has read permissions to kafkahq
roles:
- topic/read
- node/read
- group/read
- acls/read
- registry/read
- connect/read
connect-topic-reader: # an example of topic data read permissions only on kconnect-* topics
name: connect-topic-reader
roles:
- topic/read
- topic/data/read
attributes:
- topics-filter-regexp: "^kconnect-.*"
ops-reader: # ops has permissions to read all topics
name: topic-reader
roles:
- topic/read
attributes:
- topics-filter-regexp: ".*"
ldap:
groups:
- name: ops
groups:
- dev-reader
- ops-reader
- name: dev
groups:
- dev-reader
- name: connect
groups:
- dev-reader
- connect-topic-readerMy expectation is for the above to let I like both approaches for cluster-specific ACLs as specified by @sverrehu and @twobeeb, but I do think @sverrehu's method may be more flexible. |
Beta Was this translation helpful? Give feedback.
-
|
My case is slightly different because we run a cluster that support username/password authentication based on Active Directory, so the cluster is able to authenticate directly user. My expectation is that in this case AKHQ runs as a pure web interface that use provided credential to connect to the cluster and discover permission and accessible topic. In this way is easy, for us, audit the access to the cluster by user because authentication is performed on the Kafka cluster. |
Beta Was this translation helpful? Give feedback.
-
|
Let's break everything 💣 💥
{
"username": "user",
"exp": 123456789,
"iat": 123456789,
"acls": [
{
"clusters": [ ".*"],
"resource": "TOPIC",
"patterns": [ "project1-.*" ],
"roles": ["READ", "WRITE"]
},
{
"clusters": [ ".*"],
"resource": "CONNECT",
"connect-clusters": [ "connect-cluster-1" ],
"patterns": [ "project1-.*" ],
"roles": ["READ", "DEPLOY"]
},
{
"clusters": [ ".*"],
"resource": "GROUPS",
"patterns": [ "project1-.*" ],
"roles": ["READ"]
}
]
}
{
"clusters": [ ".*"],
"resource": "TOPIC",
"patterns": [ "project1-.*" ],
"roles": ["READ", "WRITE"]
},
{
"clusters": [ ".*"],
"resource": "CONNECT",
"connect-clusters": [ "connect-cluster-1" ],
"patterns": [ "project1-.*" ],
"roles": ["READ", "DEPLOY"]
},
{
"clusters": [ ".*"],
"resource": "GROUPS",
"patterns": [ "project1-.*" ],
"roles": ["READ"]
}
In terms of implementation (API side only, I didn't consider UI side):
|
Beta Was this translation helpful? Give feedback.
-
|
I like the idea overall @twobeeb !
|
| Resource | Action |
|---|---|
| TOPIC | READ, CREATE, UPDATE (partition, replication factor), DELETE |
| TOPIC_CONFIG | READ, UPDATE |
| TOPIC_DATA | READ, CREATE (no more delete as create can delete with simple null) |
| NODE | READ |
| NODE_CONFIG | READ, UPDATE |
| CONSUMER_GROUP | READ, CREATE, DELETE |
| CONSUMER_OFFSET | READ, UPDATE, DELETE |
| REGISTRY_SCHEMA | READ, CREATE, UPDATE, DELETE |
| CONNECT_CLUSTER | READ |
| CONNECT_CONFIG | READ, CREATE, UPDATE, DELETE |
| CONNECT_TASK | READ, UPDATE |
| ACL | READ, (CREATE, UPDATE, DELETE) when available |
permissions
Permissions will be a set of predefined :
permissions:
topic-admin:
resource: TOPIC
roles: ["READ", "WRITE", "CREATE", "DELETE"]bindings
I will keep groups (rename to bindings) to make configuration easy for noob with default bindings : admin, reader, no-permission. Bindings will be a set of permissions of clusters & pattern
bindings:
admin:
permissions: [ "topic-admin-all"]
clusters: [ ".*"] # List
patterns: [ ".*" ] # List
With this, the default config will provide a full list permissions in order to have a default group working
patterns & clusters behaviour
- if not defined, we allow all
- if defined, we limit only on matching entry by regexp
patternswill be applied for eachresourceon different properties (for CONNECT_CONFIG on definition name, for REGISTRY_SCHEMA on schema name, TOPIC / TOPIC_CONFIG / TOPIC_DATA on topic name, ...)- composition of binding will be used for special connect case, we must have
CONNECT_CLUSTER&CONNECT_CONFIG, roleREADon the cluster to be able any thing on connect
User bindings
- Compose ACLs for users or LDAP/OIDC groups
auth:
ldap:
groups:
- name: LDAP-ADMIN-GROUP:
bindings: ["admin", "admin-only-dev" ]
- name: LDAP-READER-PROJECT1:
bindings: ["reader"]
users:
- name: "[email protected]":
bindings: ["reader"]
header:
groups:
- name: header-admin-group
bindings: ["reader"]
users:
- name: my-username
bindings: ["reader"]
oidc:
groups:
- name: oidc-admin-group
bindings: ["reader"]
users:
- name: my-username
bindings: ["reader"]
basic-auth:
- username: user
password: d74ff0ee8da3b9806b18c877dbf29bbde50b5bd8e4dad7a3a725000feb82e8f1
bindings: ["admin", "admin-only-dev" ]- Unauthenticated users (security disabled)
security.default-bindings: [ "binding1", "binding2", ... ]What do you think @twobeeb ?
Beta Was this translation helpful? Give feedback.
-
|
It gets even worse because for TOPIC, you want to manage 3 resources : TOPIC, TOPIC_DATA, TOPIC_CONFIG |
Beta Was this translation helpful? Give feedback.
-
|
It's possible with TOPIC for example:
Just the is always the trouble that user can map strange config like : |
Beta Was this translation helpful? Give feedback.
-
|
If I take this into account, as well as the new rule "one permission per pattern/cluster": akhq:
security:
permissions:
topic-admin:
resource: TOPIC
roles: ["READ", "CREATE", "UPDATE", "DELETE", "CONSUME", "PRODUCE", "READ_CONFIG", "UPDATE_CONFIG"]
topic-reader:
resource: TOPIC
roles: ["READ", "CONSUME", "READ_CONFIG"]
# Repeat for GROUP, CONNECT, SCHEMAS
bindings:
full-admin:
- permission: "topic-admin" # Implicit all cluster and .* patterns
- permission: "registry-admin"
- permission: "connect-admin"
- permission: "registry-admin"
project1-reader: # Implicit all clusters
- permission: "topic-reader"
patterns: ["project1-.*"]
- permission: "connect-reader"
patterns: ["project1-.*"]
- permission: "group-reader"
patterns: ["project1-.*", "connect-project1-.*"]
auth:
basic-auth:
- username: admin
password: d74ff0ee8da3b9806b18c877dbf29bbde50b5bd8e4dad7a3a725000feb82e8f1
bindings: ["full-admin"]
- username: project1-user
password: d74ff0ee8da3b9806b18c877dbf29bbde50b5bd8e4dad7a3a725000feb82e8f1
bindings: ["project1-reader"]I'm fine with this if you are.
|
Beta Was this translation helpful? Give feedback.
-
|
I'm almost fine with this, just hope people will understand this 🙏 |
Beta Was this translation helpful? Give feedback.
-
|
At some point (not necessary today), we will need a detailled explanation for the roles TopicsEverything is further restricted by
|
Beta Was this translation helpful? Give feedback.
-
|
Tried to use the External roles and attributes mapping functionality but stumbled across the limitation that only one object of "roles", "topic-filters", ... is usable and not a list as response object. So i'm commenting here. What we try to achieve:
We do have an external service which would be able to map username and groups to kafka like permissions. Using a rest-service like the current {
"clusters": [ ".*"],
"resource": "TOPIC",
"patterns": [ "project1-.*" ],
"roles": ["READ", "WRITE"]
},
{
"clusters": [ ".*"],
"resource": "CONNECT",
"connect-clusters": [ "connect-cluster-1" ],
"patterns": [ "project1-.*" ],
"roles": ["READ", "DEPLOY"]
},
{
"clusters": [ ".*"],
"resource": "GROUPS",
"patterns": [ "project1-.*" ],
"roles": ["READ"]
}From the discussion above I'm not sure if you still plan to provide a way to completely delegate the authorization/permissions to an external service (which is actually what we need). we would prefere to have no definition of permissions/groups/bindings in akhq itself. mainly due to the fact that we need finegrained permissions on resource level (user a should be able to read from topics XY but not from topics Z). |
Beta Was this translation helpful? Give feedback.
-
|
first of all: thanks for the work on the new acl system. we would be fine with these changes as it covers our needs. here are some changes we would prefer:
this is only a suggestion and we are not using multiple clusters per akhq (which i understand that it makes the acl design more complex). |
Beta Was this translation helpful? Give feedback.
-
|
I originally pushed for this recommendation as well to not use Consider the permissions to be an abstraction of
Our ambition is to cover 90% of the use cases with permissions that makes sense. Regarding the roles names and meaning:
@tchiotludo what do you think ? |
Beta Was this translation helpful? Give feedback.
-
|
fully understand the design and i agree that its probably easier for a lot of people which are using predefined permissions. my main concern is that you have to define the abstraction in akhq config and then use this one in the external service. therefore you are not completely free in the external service. if you have a new combination which is not covered by an abstraction you have to alter akhq and the external service. permissions that makes sense are not infinite so i think we would be fine with the design. |
Beta Was this translation helpful? Give feedback.
-
|
@twobeeb Another short question about this one. Just to be sure i got it right
According to ClaimResponse.java the external Service returns a - permission: "topic-reader"
patterns: ["project1-.*"]
clusters: ["dev"]I was unsure how to reference the cluster but after reading |
Beta Was this translation helpful? Give feedback.
-
|
I confirm, |
Beta Was this translation helpful? Give feedback.
-
|
We also have what I think is an interesting use case: when a topic is created, the topic owner (the team) get full ownership of that topics will all rights, other teams (users) can only view the topic name and its configurations/parameters. As of now because the user who created the topic is not tracked nor persisted/sent to another backend, we cannot use akhq for that flow (we use it for everything else though), but if we could I would love to get rid of our internal tooling. |
Beta Was this translation helpful? Give feedback.
-
|
@tchiotludo any update on this? |
Beta Was this translation helpful? Give feedback.
-
|
@ebrard : no news as I know
Sorry ... Feel free to take it 🙏 |
Beta Was this translation helpful? Give feedback.
-
|
Ok I see that the claims and in particular the topic/cg... regexes are used in the the authorization response, in the attributes. These are then use here and there in the code to filter the list of things visible and actable on. I would rather introduce a RBAC module that authorises or denies any possible action. This would also mean that an external service could authorize it as well instead of akhq itself. It's a little bit more work but this will make RBAC as precise as needed (to the specific topic rule for example, instead of a match). WDYT? |
Beta Was this translation helpful? Give feedback.
-
|
So you have some design implementation to share? |
Beta Was this translation helpful? Give feedback.
-
|
Actually I read again what @twobeeb described and this is very close the only different is the level, instead of just a pattern I would introduce a pattern and/or exact matches (so just like the kafka ACL really), and instead of calling an external service to get the claims, I would just make the external service send the auth ok or deny directly. Ofc, as mentioned by twobeed for listing controller the answer will be used to discard objects (topics, consumer groups...) from the list. All in all Kafka ACL themselves would be a good basis, we would only need an "extension" for schema management. So the idea, someone identified as user1 tries to create a topic, the controller generate a ACL "claim": The implementation or external service checks against the pre-defined rules (again exact match and pattern) and answer true or false, or eventually return the matching ACL. Yes it can be a bit heavy to have to define all the necessary rules (or get them computed by an external service) but at least we are as fine grained as Kafka itself. As a matter of fact a possible implementation of the "RBAC" interface, could be to check against kafka acl themselves (at least for non schema things) which means we get closer to what @angeloxx mentioned (user management would still need to be in akhq and we would either need a mapping or a pass through, I think). I also like the idea to re-use the kafka acl concept itself because ppl are getting familiar with it now. |
Beta Was this translation helpful? Give feedback.
-
|
We currently do something similar in our own tooling, except that the permission is at the "owner" scope (if you own a topic, you get full permissions, otherwise you can just see that it exists and see some of the topic configuration): var isAuthorized = rbacService.authorize(user, topic.get());
if (!isAuthorized) {
return new ResponseEntity<>(
new DeleteTopicResponse(topicName, true, "Delete operation denied"),
HttpStatus.UNAUTHORIZED);
}We use Spring. |
Beta Was this translation helpful? Give feedback.
-
|
as discussed with @AlexisSouquiere, new design: roles:
custom-role:
- resources: ["TOPIC", "TOPIC_DATA"]
actions: ["READ", "WRITE", "CREATE", "DELETE"]
- resources: ["GROUPS"]
actions: ["READ"]
groups:
full-admin:
- role: "admin" # Implicit all cluster and .* patterns
- role: "registry-admin"
- role: "connect-admin"
- role: "registry-admin"
project1-reader: # Implicit all clusters
- role: "topic-reader"
patterns: ["project1-.*"]
- role: "connect-reader"
cluster: ["dev-.*"]
patterns: ["project1-.*"]
- role: "group-reader"
patterns: ["project1-.*", "connect-project1-.*"]
auth:
basic-auth:
- username: admin
password: d74ff0ee8da3b9806b18c877dbf29bbde50b5bd8e4dad7a3a725000feb82e8f1
groups: ["full-admin"]
- username: project1-user
password: d74ff0ee8da3b9806b18c877dbf29bbde50b5bd8e4dad7a3a725000feb82e8f1
groups: ["project1-reader"] |
Beta Was this translation helpful? Give feedback.
-
|
For those you want to follow, the draft PR is here #1472 |
Beta Was this translation helpful? Give feedback.
-
There is many features request about acls: #451 #456 #291 #254.
You want to have more fine grained acl !
Now how ! I create this issue more to have feedback from you !
There is many ways to handle it, so since I want this to be last rework on AKHQ ACL!
For every comment, can you add a 👍 or others reaction in order to track the most wanted and also you can add some comments about the reason and special idea you may have !
Thanks for your help !
Beta Was this translation helpful? Give feedback.
All reactions