Skip to content

Commit

Permalink
[feat](Authorization-plugin)Authorization framework modularization (a…
Browse files Browse the repository at this point in the history
…pache#40750)

# AccessControllerFactory Interface

## Overview

The `AccessControllerFactory` interface is responsible for creating and
managing `CatalogAccessController` instances. The interface includes the
following methods:

- **`default String factoryIdentifier()`**: Returns the identifier for
the factory, defaulting to the simple name of the implementing class. To
maintain compatibility with user-defined plugins from older versions,
the `factoryIdentifier` method provides a default implementation that
returns the simple name of the current implementation class, ensuring
that each factory has a unique identifier.

- **`CatalogAccessController createAccessController(Map<String, String>
prop)`**: Creates a new instance of `CatalogAccessController` and
initializes it with the provided properties.

## Factory Identifier

Each class implementing `AccessControllerFactory` will automatically use
its class name as the factory identifier. This helps in identifying
different factory instances during plugin loading and selection.

## Instance Creation

The `createAccessController` method allows you to create and initialize
`CatalogAccessController` instances. The `prop` parameter provides the
configuration properties needed for initialization.

## Compatibility

- If you are using the previously built-in `range-dorir` authentication
plugin, no configuration changes are required; it will continue to
function as before.
- For custom plugins, configuration information should be defined in
`conf/access.conf`. Then, in `fe.conf`, specify the
`access_controller_type` as the identifier for the custom plugin.

## How to Extend

- Add the `fe-core` dependency to your Maven `pom.xml` file.

```xml
        <dependency>
            <groupId>org.apache.doris</groupId>
            <artifactId>fe-core</artifactId>
            <version>1.2-SNAPSHOT</version>
            <scope>provided</scope>
        </dependency>
```
Then, implement the AccessControllerFactory interface to create your own
plugin factory class as follows:
```java

public class SimpleAccessControllerFactory implements AccessControllerFactory {
    @OverRide
    public String factoryIdentifier() {
        return "local-simple";
    }

    @OverRide
    public CatalogAccessController createAccessController(Map<String, String> map) {
        return new SimpleAccessController(map);
    }
}

package org.example.access;

import org.apache.doris.analysis.ResourceTypeEnum;
import org.apache.doris.analysis.UserIdentity;
import org.apache.doris.common.AuthorizationException;
import org.apache.doris.mysql.privilege.CatalogAccessController;
import org.apache.doris.mysql.privilege.DataMaskPolicy;
import org.apache.doris.mysql.privilege.PrivPredicate;
import org.apache.doris.mysql.privilege.RowFilterPolicy;

import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;

public class SimpleAccessController implements CatalogAccessController {

    private HashMap<String, Boolean> databasePrivs = new HashMap<>();
    // just for test
    public SimpleAccessController(Map<String, String> prop) {
        prop.forEach((k, v) -> {
            databasePrivs.put(k, Boolean.parseBoolean(v));
        });
    }

    @OverRide
    public boolean checkGlobalPriv(UserIdentity userIdentity, PrivPredicate privPredicate) {
        return false;
    }

    @OverRide
    public boolean checkCtlPriv(UserIdentity userIdentity, String s, PrivPredicate privPredicate) {
        return true;
    }
    ...

```
Add a new folder named **META-INF/services** under the resources
directory, Create a new file named
**org.apache.doris.mysql.privilege.AccessControllerFactory.** with a
file containing
**org.apache.doris.mysql.privilege.AccessControllerFactory.**

## How to Use
- In `fe.conf`, specify the **access_controller_type=local-simple**
- Put the jar file containing the custom plugin in the **fe/custom_lib**
directory.
  • Loading branch information
CalvinKirs authored Sep 24, 2024
1 parent ed2a060 commit dd5605e
Show file tree
Hide file tree
Showing 11 changed files with 257 additions and 46 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -2323,6 +2323,12 @@ public class Config extends ConfigBase {
@ConfField
public static long ranger_cache_size = 10000;

@ConfField(description = {
"鉴权插件配置文件路径,需在 DORIS_HOME 下,默认为 conf/authorization.conf",
"Authorization plugin configuration file path, need to be in DORIS_HOME,"
+ "default is conf/authorization.conf"})
public static String authorization_config_file_path = "conf/authorization.conf";

/**
* This configuration is used to enable the statistics of query information, which will record
* the access status of databases, tables, and columns, and can be used to guide the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,12 @@
import java.util.Map;

public class RangerHiveAccessControllerFactory implements AccessControllerFactory {

@Override
public String factoryIdentifier() {
return "ranger-hive";
}

@Override
public CatalogAccessController createAccessController(Map<String, String> prop) {
return new RangerCacheHiveAccessController(prop);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@
import org.apache.doris.common.Config;
import org.apache.doris.common.DdlException;
import org.apache.doris.datasource.CatalogIf;
import org.apache.doris.datasource.CatalogMgr;
import org.apache.doris.datasource.ExternalCatalog;
import org.apache.doris.policy.Policy;
import org.apache.doris.policy.StoragePolicy;
Expand Down Expand Up @@ -1560,16 +1559,6 @@ public static void checkCatalogProperties(Map<String, String> properties, boolea
// "access_controller.properties.prop2" = "yyy",
// )
// 1. get access controller class
String acClass = properties.getOrDefault(CatalogMgr.ACCESS_CONTROLLER_CLASS_PROP, "");
if (!Strings.isNullOrEmpty(acClass)) {
// 2. check if class exists
try {
Class.forName(acClass);
} catch (ClassNotFoundException e) {
throw new AnalysisException("failed to find class " + acClass, e);
}
}

if (isAlter) {
// The 'use_meta_cache' property can not be modified
if (properties.containsKey(ExternalCatalog.USE_META_CACHE)) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,14 @@
import java.util.Map;

public interface AccessControllerFactory {
/**
* Returns the identifier for the factory, such as "range-doris".
*
* @return the factory identifier
*/
default String factoryIdentifier() {
return this.getClass().getSimpleName();
}

CatalogAccessController createAccessController(Map<String, String> prop);
}
Original file line number Diff line number Diff line change
Expand Up @@ -22,24 +22,27 @@
import org.apache.doris.analysis.UserIdentity;
import org.apache.doris.catalog.AuthorizationInfo;
import org.apache.doris.catalog.Env;
import org.apache.doris.catalog.authorizer.ranger.doris.RangerCacheDorisAccessController;
import org.apache.doris.common.Config;
import org.apache.doris.common.UserException;
import org.apache.doris.datasource.CatalogIf;
import org.apache.doris.datasource.ExternalCatalog;
import org.apache.doris.datasource.InternalCatalog;
import org.apache.doris.plugin.PropertiesUtils;
import org.apache.doris.qe.ConnectContext;

import com.google.common.base.Preconditions;
import com.google.common.collect.Maps;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

import java.io.IOException;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.ServiceLoader;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;

/**
* AccessControllerManager is the entry point of privilege authentication.
Expand All @@ -52,19 +55,51 @@ public class AccessControllerManager {
private static final Logger LOG = LogManager.getLogger(AccessControllerManager.class);

private Auth auth;
// Default access controller instance used for handling cases where no specific controller is specified
private CatalogAccessController defaultAccessController;
// Map that stores the mapping between catalogs and their corresponding access controllers
private Map<String, CatalogAccessController> ctlToCtlAccessController = Maps.newConcurrentMap();
// Cache of loaded access controller factories for quick creation of new access controllers
private ConcurrentHashMap<String, AccessControllerFactory> accessControllerFactoriesCache
= new ConcurrentHashMap<>();
// Mapping between access controller class names and their identifiers for easy lookup of factory identifiers
private ConcurrentHashMap<String, String> accessControllerClassNameMapping = new ConcurrentHashMap<>();

public AccessControllerManager(Auth auth) {
this.auth = auth;
if (Config.access_controller_type.equalsIgnoreCase("ranger-doris")) {
defaultAccessController = new RangerCacheDorisAccessController("doris");
} else {
defaultAccessController = new InternalAccessController(auth);
}
loadAccessControllerPlugins();
String accessControllerName = Config.access_controller_type;
this.defaultAccessController = loadAccessControllerOrThrow(accessControllerName);
ctlToCtlAccessController.put(InternalCatalog.INTERNAL_CATALOG_NAME, defaultAccessController);
}

private CatalogAccessController loadAccessControllerOrThrow(String accessControllerName) {
if (accessControllerName.equalsIgnoreCase("default")) {
return new InternalAccessController(auth);
}
if (accessControllerFactoriesCache.containsKey(accessControllerName)) {
Map<String, String> prop;
try {
prop = PropertiesUtils.loadAccessControllerPropertiesOrNull();
} catch (IOException e) {
throw new RuntimeException("Failed to load authorization properties."
+ "Please check the configuration file, authorization name is " + accessControllerName, e);
}
return accessControllerFactoriesCache.get(accessControllerName).createAccessController(prop);
}
throw new RuntimeException("No authorization plugin factory found for " + accessControllerName
+ ". Please confirm that your plugin is placed in the correct location.");
}

private void loadAccessControllerPlugins() {
ServiceLoader<AccessControllerFactory> loader = ServiceLoader.load(AccessControllerFactory.class);
for (AccessControllerFactory factory : loader) {
LOG.info("Found Access Controller Plugin Factory: {}", factory.factoryIdentifier());
accessControllerFactoriesCache.put(factory.factoryIdentifier(), factory);
accessControllerClassNameMapping.put(factory.getClass().getName(), factory.factoryIdentifier());
}
}

public CatalogAccessController getAccessControllerOrDefault(String ctl) {
CatalogAccessController catalogAccessController = ctlToCtlAccessController.get(ctl);
if (catalogAccessController != null) {
Expand Down Expand Up @@ -94,23 +129,28 @@ public boolean checkIfAccessControllerExist(String ctl) {
}

public void createAccessController(String ctl, String acFactoryClassName, Map<String, String> prop,
boolean isDryRun) {
Class<?> factoryClazz = null;
try {
factoryClazz = Class.forName(acFactoryClassName);
AccessControllerFactory factory = (AccessControllerFactory) factoryClazz.newInstance();
CatalogAccessController accessController = factory.createAccessController(prop);
if (!isDryRun) {
ctlToCtlAccessController.put(ctl, accessController);
LOG.info("create access controller {} for catalog {}", ctl, acFactoryClassName);
}
} catch (ClassNotFoundException e) {
throw new RuntimeException(e);
} catch (InstantiationException e) {
throw new RuntimeException(e);
} catch (IllegalAccessException e) {
throw new RuntimeException(e);
boolean isDryRun) {
String pluginIdentifier = getPluginIdentifierForAccessController(acFactoryClassName);
CatalogAccessController accessController = accessControllerFactoriesCache.get(pluginIdentifier)
.createAccessController(prop);
if (!isDryRun) {
ctlToCtlAccessController.put(ctl, accessController);
LOG.info("create access controller {} for catalog {}", acFactoryClassName, ctl);
}
}

private String getPluginIdentifierForAccessController(String acClassName) {
String pluginIdentifier = null;
if (accessControllerClassNameMapping.containsKey(acClassName)) {
pluginIdentifier = accessControllerClassNameMapping.get(acClassName);
}
if (accessControllerFactoriesCache.containsKey(acClassName)) {
pluginIdentifier = acClassName;
}
if (null == pluginIdentifier || !accessControllerFactoriesCache.containsKey(pluginIdentifier)) {
throw new RuntimeException("Access Controller Plugin Factory not found for " + acClassName);
}
return pluginIdentifier;
}

public void removeAccessController(String ctl) {
Expand Down Expand Up @@ -160,7 +200,7 @@ public boolean checkTblPriv(ConnectContext ctx, TableName tableName, PrivPredica
}

public boolean checkTblPriv(ConnectContext ctx, String qualifiedCtl,
String qualifiedDb, String tbl, PrivPredicate wanted) {
String qualifiedDb, String tbl, PrivPredicate wanted) {
if (ctx.isSkipAuth()) {
return true;
}
Expand All @@ -184,7 +224,7 @@ public void checkColumnsPriv(ConnectContext ctx, String ctl, String qualifiedDb,

public void checkColumnsPriv(UserIdentity currentUser, String
ctl, String qualifiedDb, String tbl, Set<String> cols,
PrivPredicate wanted) throws UserException {
PrivPredicate wanted) throws UserException {
boolean hasGlobal = checkGlobalPriv(currentUser, wanted);
CatalogAccessController accessController = getAccessControllerOrDefault(ctl);
accessController.checkColsPriv(hasGlobal, currentUser, ctl, qualifiedDb,
Expand All @@ -207,7 +247,7 @@ public boolean checkCloudPriv(ConnectContext ctx, String cloudName, PrivPredicat
}

public boolean checkCloudPriv(UserIdentity currentUser, String cloudName,
PrivPredicate wanted, ResourceTypeEnum type) {
PrivPredicate wanted, ResourceTypeEnum type) {
return defaultAccessController.checkCloudPriv(currentUser, cloudName, wanted, type);
}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

package org.apache.doris.mysql.privilege;

import org.apache.doris.catalog.authorizer.ranger.doris.RangerCacheDorisAccessController;

import java.util.Map;

public class RangerDorisAccessControllerFactory implements AccessControllerFactory {
@Override
public String factoryIdentifier() {
return "ranger-doris";
}

@Override
public RangerCacheDorisAccessController createAccessController(Map<String, String> prop) {
return new RangerCacheDorisAccessController("doris");
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

package org.apache.doris.plugin;

import org.apache.doris.common.Config;
import org.apache.doris.common.EnvUtils;

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;

public class PropertiesUtils {
public static final String ACCESS_PROPERTIES_FILE_DIR = Config.authorization_config_file_path;

public static Map<String, String> loadAccessControllerPropertiesOrNull() throws IOException {
String configFilePath = EnvUtils.getDorisHome() + ACCESS_PROPERTIES_FILE_DIR;
if (new File(configFilePath).exists()) {
Properties properties = new Properties();
properties.load(Files.newInputStream(Paths.get(configFilePath)));
return propertiesToMap(properties);
}
return null;
}

public static Map<String, String> propertiesToMap(Properties properties) {
Map<String, String> map = new HashMap<>();
for (Map.Entry<Object, Object> entry : properties.entrySet()) {
String key = String.valueOf(entry.getKey());
String value = String.valueOf(entry.getValue());
map.put(key, value);
}
return map;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#
org.apache.doris.mysql.privilege.RangerDorisAccessControllerFactory
org.apache.doris.catalog.authorizer.ranger.hive.RangerHiveAccessControllerFactory
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

package org.apache.doris.nereids.privileges;

import org.apache.doris.mysql.privilege.AccessControllerFactory;
import org.apache.doris.mysql.privilege.CatalogAccessController;

import java.util.Map;

public class CustomAccessControllerFactory implements AccessControllerFactory {
@Override
public String factoryIdentifier() {
return "CustomAccess";
}

@Override
public CatalogAccessController createAccessController(Map<String, String> prop) {
return new TestCheckPrivileges.SimpleCatalogAccessController();
}
}
Loading

0 comments on commit dd5605e

Please sign in to comment.