From 4b18b55cd20cf53c7fe2d568827bf1b30e77cc64 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Pawe=C5=82=20Marczewski?= Date: Fri, 29 Apr 2022 15:07:58 +0200 Subject: [PATCH] [LibOS,Pal/Linux-SGX] Replace old protected files subsystem MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This change replaces the protected files implemented in Linux-SGX PAL by the new encrypted files subsystem in LibOS, implemented in previous commits. - Remove old implementation in SGX PAL (protected-file handles, reference counting, setting the key etc) - Change the old user-facing interfaces (`sgx.protected_files`, `/dev/attestation/protected_files_key`) to control encrypted files, add deprecation warnings to them - Add documentation for the new encrypted files syntax Signed-off-by: PaweÅ‚ Marczewski --- Documentation/attestation.rst | 22 +- Documentation/devel/performance.rst | 2 +- Documentation/manifest-syntax.rst | 151 ++-- Documentation/pal/host-abi.rst | 2 +- LibOS/shim/include/shim_fs_encrypted.h | 5 + LibOS/shim/src/fs/chroot/fs.c | 70 -- LibOS/shim/src/fs/dev/attestation.c | 81 +- LibOS/shim/src/fs/shim_fs.c | 151 ++++ LibOS/shim/src/fs/shim_fs_encrypted.c | 122 ++- LibOS/shim/test/fs/README.md | 11 +- LibOS/shim/test/fs/manifest.template | 12 +- .../shim/test/fs/{test_pf.py => test_enc.py} | 49 +- LibOS/shim/test/regression/attestation.c | 70 ++ .../regression/attestation.manifest.template | 2 + Pal/include/pal/pal.h | 14 +- Pal/include/pal_internal.h | 1 - Pal/src/db_main.c | 4 +- Pal/src/db_misc.c | 4 - Pal/src/db_rtld.c | 13 +- Pal/src/host/Linux-SGX/db_files.c | 368 +------- Pal/src/host/Linux-SGX/db_main.c | 10 +- Pal/src/host/Linux-SGX/db_misc.c | 5 - Pal/src/host/Linux-SGX/db_process.c | 39 - Pal/src/host/Linux-SGX/db_streams.c | 5 - Pal/src/host/Linux-SGX/enclave_pf.c | 811 ------------------ Pal/src/host/Linux-SGX/enclave_pf.h | 108 --- Pal/src/host/Linux-SGX/meson.build | 3 - .../host/Linux-SGX/tools/common/meson.build | 1 + Pal/src/host/Linux/db_misc.c | 5 - Pal/src/host/Skeleton/db_misc.c | 5 - Pal/src/pal-symbols | 1 - common/include/hex.h | 2 +- common/src/protected_files/README.rst | 63 +- 33 files changed, 555 insertions(+), 1657 deletions(-) rename LibOS/shim/test/fs/{test_pf.py => test_enc.py} (85%) delete mode 100644 Pal/src/host/Linux-SGX/enclave_pf.c delete mode 100644 Pal/src/host/Linux-SGX/enclave_pf.h diff --git a/Documentation/attestation.rst b/Documentation/attestation.rst index 20844272b1..81353467aa 100644 --- a/Documentation/attestation.rst +++ b/Documentation/attestation.rst @@ -204,13 +204,23 @@ use the :program:`quote_dump`, :program:`ias_request` and EPID based quote verification) or to use the Intel DCAP libraries and tools (for DCAP based quote verification). -The ``/dev/attestation`` pseudo-filesystem also exposes a pseudo-file to set the -protected files wrap (master) key (see also :doc:`manifest-syntax`): +The ``/dev/attestation`` pseudo-filesystem also exposes pseudo-files to set the +encryption keys (see also :doc:`manifest-syntax`): -- ``/dev/attestation/protected_files_key`` pseudo-file can be opened for write - access. Typically, it is opened before the actual application runs and filled - with a 128-bit key obtained from a remote secret provisioning service. +- ``/dev/attestation/keys/`` file contains the encryption key with a + given name (the default key name is ``default``). Typically, it is opened + before the actual application runs and filled with a 128-bit key obtained from + a remote secret provisioning service. The format of the file is a 16-byte raw + binary value. +.. note:: + Previously, ``/dev/attestation/protected_files_key`` was used for setting the + default encryption key, and Gramine still supports that file for backward + compatibility. + + Note that the old file (``/dev/attestation/protected_files_key``) uses a + 32-character hex value, and the new files + (``/dev/attestation/keys/``) use a 16-byte raw binary value. Mid-level RA-TLS interface -------------------------- @@ -398,7 +408,7 @@ EPID based ``secret_prov_verify_epid.so`` and DCAP/ECDSA based The examples of using RA-TLS can be found under ``CI-Examples/ra-tls-secret-prov``. The examples include minimalistic provisioning of constant-string secrets as -well as provisioning of an encryption key and its later use for protected files. +well as provisioning of an encryption key and its later use for encrypted files. ``secret_prov_attest.so`` ^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Documentation/devel/performance.rst b/Documentation/devel/performance.rst index 3a7851acdd..bddb9666bc 100644 --- a/Documentation/devel/performance.rst +++ b/Documentation/devel/performance.rst @@ -425,7 +425,7 @@ encrypts many means of communication: #. Inter-Process Communication (IPC) is encrypted via TLS-PSK. Regular pipes, FIFO pipes, UNIX domain sockets are all transparently encrypted. -#. Files marked as ``sgx.protected_files`` are transparently encrypted/decrypted +#. Files mounted as ``type = "encrypted"`` are transparently encrypted/decrypted on each file access via SGX SDK Merkle-tree format. #. ``Fork/vfork/clone`` all require to generate an encrypted checkpoint of the diff --git a/Documentation/manifest-syntax.rst b/Documentation/manifest-syntax.rst index 579c4dff7d..68ffd3900f 100644 --- a/Documentation/manifest-syntax.rst +++ b/Documentation/manifest-syntax.rst @@ -134,14 +134,15 @@ If you want your application to use commandline arguments you need to either set ``loader.argv_src_file`` to a file containing output of :ref:`gramine-argv-serializer`. -``loader.argv_src_file`` is intended to point to either a trusted file or a -protected file. The former allows to securely hardcode arguments (current +``loader.argv_src_file`` is intended to point to either a trusted file or an +encrypted file. The former allows to securely hardcode arguments (current manifest syntax doesn't allow to include them inline), the latter allows the arguments to be provided at runtime from an external (trusted) source. .. note :: - Pointing to a protected file is currently not supported, due to the fact that - PF wrap key provisioning currently happens after setting up arguments. + Pointing to an encrypted file is currently not supported, due to the fact + that encryption key provisioning currently happens after setting up + arguments. Environment variables ^^^^^^^^^^^^^^^^^^^^^ @@ -180,15 +181,15 @@ value or be a passthrough. ``loader.env_src_file`` allows to specify a URI to a file containing serialized environment, which can be generated using :ref:`gramine-argv-serializer`. This option is intended -to point to either a trusted file or a protected file. The former allows to +to point to either a trusted file or an encrypted file. The former allows to securely hardcode environments (in a more flexible way than ``loader.env.[ENVIRON]`` option), the latter allows the environments to be provided at runtime from an external (trusted) source. .. note :: - Pointing to a protected file is currently not supported, due to the fact that - PF wrap key provisioning currently happens after setting up environment - variables. + Pointing to an encrypted file is currently not supported, due to the fact + that encryption key provisioning currently happens after setting up + environment variables. If the same variable is set in both, then ``loader.env.[ENVIRON]`` takes precedence. It is prohibited to specify both ``value`` and ``passthrough`` keys @@ -254,7 +255,7 @@ Gramine internal metadata size (default: "0") This syntax specifies how much additional memory Gramine reserves for its -internal use (e.g., metadata for trusted/protected files, internal handles, +internal use (e.g., metadata for trusted files, internal handles, etc.). By default, Gramine pre-allocates 64MB of internal memory for this metadata, but for huge workloads this limit may be not enough. In this case, Gramine loudly fails with "out of PAL memory" error. To run huge workloads, @@ -367,7 +368,7 @@ The ``type`` parameter specifies the mount point type. If omitted, it defaults to ``"chroot"``. The ``path`` parameter must be an absolute path (i.e. must begin with ``/``). -Gramine currently supports two types of mount points: +Gramine currently supports the following types of mount points: * ``chroot`` (default): Host-backed files. All host files and sub-directories found under ``[URI]`` are forwarded to the Gramine instance and placed under @@ -378,6 +379,9 @@ Gramine currently supports two types of mount points: Docker's named volumes. Files under ``chroot`` mount points support mmap and fork/clone. +* ``encrypted``: Host-backed encrypted files. See :ref:`encrypted-files` for + more information. + * ``tmpfs``: Temporary in-memory-only files. These files are *not* backed by host-level files. The tmpfs files are created under ``[PATH]`` (this path is empty on Gramine instance startup) and are destroyed when a Gramine instance @@ -598,7 +602,7 @@ are not cryptographically hashed and are thus not protected. .. warning:: It is insecure to allow files containing code or critical information; - developers must not allow files blindly! Instead, use trusted or protected + developers must not allow files blindly! Instead, use trusted or encrypted files. Trusted files @@ -631,67 +635,67 @@ Marking files as trusted is especially useful for shared libraries: a |~| trusted library cannot be silently replaced by a malicious host because the hash verification will fail. -Protected files +.. _encrypted-files: + +Encrypted files ^^^^^^^^^^^^^^^ :: - sgx.insecure__protected_files_key = "[16-byte hex value]" - - sgx.protected_files = [ - "[URI]", - "[URI]", - ] - - sgx.protected_mrenclave_files = [ - "[URI]", - "[URI]", + fs.mounts = [ + { type = "encrypted", path = "[PATH]", uri = "[URI]", key_name = "[KEY_NAME]" }, ] - sgx.protected_mrsigner_files = [ - "[URI]", - "[URI]", - ] + fs.insecure__keys.[KEY_NAME] = "[32-character hex value]" -This syntax specifies the files that are encrypted on disk and transparently +This syntax allows mounting files that are encrypted on disk and transparently decrypted when accessed by Gramine or by application running inside Gramine. -Protected files guarantee data confidentiality and integrity (tamper -resistance), as well as file swap protection (a protected file can only be -accessed when in a specific path). +Encrypted files guarantee data confidentiality and integrity (tamper +resistance), as well as file swap protection (an encrypted file can only be +accessed when in a specific host path). + +Encrypted files were previously known as *protected files*, and some Gramine +tools might still use the old name. + +URI can be a file or a directory. If a directory is mounted, all existing +files/directories within it are recursively treated as encrypted (and are +expected to be encrypted in the PF format). New files created in an encrypted +mount are also automatically treated as encrypted. -URI can be a file or a directory. If a directory is specified, all existing -files/directories within it are registered as protected recursively (and are -expected to be encrypted in the PF format). New files created in a protected -directory are automatically treated as protected. +.. warning:: + The current implementation assumes that ``type = "encrypted"`` mounts do not + overlap on host, i.e. there are no host files reachable through more than one + ``type = "encrypted"`` mount. Otherwise, changes made to such files might not + be correctly persisted by Gramine. -Note that path size of a protected file is limited to 512 bytes and filename +Note that path size of an encrypted file is limited to 512 bytes and filename size is limited to 260 bytes. -``sgx.insecure__protected_files_key`` specifies the wrap (master) encryption key -and must be used only for debugging purposes. +The ``key_name`` mount parameter specifies the name of the encryption key. If +omitted, it will default to ``"default"``. This feature can be used to mount +different files or directories with different encryption keys. -.. warning:: - ``sgx.insecure__protected_files_key`` hard-codes the key in the manifest. - This option is thus insecure and must not be used in production environments! - Typically, you want to provision the protected files wrap key using SGX - local/remote attestation, thus you should not specify the - ``sgx.insecure__protected_files_key`` manifest option at all. Instead, use - the Secret Provisioning interface (see :doc:`attestation`). +``fs.insecure__keys.[KEY_NAME]`` can be used to specify the encryption keys +directly in manifest. This option must be used only for debugging purposes. -There are three types of protected files: +.. warning:: + ``sgx.insecure__keys.[KEY_NAME]`` hard-codes the key in the manifest. This + option is thus insecure and must not be used in production environments! + Typically, you want to provision the encryption keys using SGX + local/remote attestation, thus you should not specify any + ``sgx.insecure__keys.[KEY_NAME]`` manifest options at all. Instead, use the + Secret Provisioning interface (see :doc:`attestation`). -* ``sgx.protected_files`` are encrypted using the wrap (master) encryption key; - they are well-suited for input files encrypted by the user and sent to the - deployment platform as well as for output files sent back to the user and - decrypted at the user side. +Key names beginning with underscore (``_``) denote special keys provided by +Gramine: -* ``sgx.protected_mrenclave_files`` are encrypted using the SGX sealing key - based on the MRENCLAVE identity of the enclave; they are useful to allow only - the same enclave (on the same platform) to unseal files. +* ``"_sgx_mrenclave"`` (SGX only) is the SGX sealing key based on the MRENCLAVE + identity of the enclave. This is useful to allow only the same enclave (on the + same platform) to unseal files. -* ``sgx.protected_mrsigner_files`` are encrypted using the SGX sealing key based - on the MRSIGNER identity of the enclave; they are useful to allow all enclaves - signed with the same key (and on the same platform) to unseal files. +* ``"_sgx_mrsigner"`` (SGX only) is the SGX sealing key based on the MRSIGNER + identity of the enclave. This is useful to allow all enclaves signed with the + same key (and on the same platform) to unseal files. File check policy ^^^^^^^^^^^^^^^^^ @@ -876,3 +880,40 @@ Experimental sysfs topology support fs.experimental__enable_sysfs_topology = [true|false] This feature is now enabled by default and the option was removed. + +Protected files (deprecated syntax) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + sgx.protected_files = [ + "[URI]", + "[URI]", + ] + + sgx.protected_mrenclave_files = [ + "[URI]", + "[URI]", + ] + + sgx.protected_mrsigner_files = [ + "[URI]", + "[URI]", + ] + +This syntax specified the previous SGX-only protected files. It has been +replaced with ``type = "encrypted"`` mounts (see :ref:`encrypted-files`). + +.. warning:: + Gramine will attempt to convert this syntax to mounted filesystems, but might + fail to do so correctly in more complicated cases (e.g. when a single host + file belongs to multiple mounts). It is recommended to rewrite all usages of + this syntax to ``type = "encrypted"`` mounts. + +:: + + fs.insecure__protected_files_key = "[32-character hex value]" + +This syntax allowed specifying the default encryption key for protected files. +It has been replaced by ``fs.insecure__keys.[KEY_NAME]]``. Note that both old +and new syntax are suitable for debugging purposes only. diff --git a/Documentation/pal/host-abi.rst b/Documentation/pal/host-abi.rst index 4c77e6e95e..151209188d 100644 --- a/Documentation/pal/host-abi.rst +++ b/Documentation/pal/host-abi.rst @@ -338,5 +338,5 @@ and to obtain an attestation report and quote. .. doxygenfunction:: DkAttestationQuote :project: pal -.. doxygenfunction:: DkSetProtectedFilesKey +.. doxygenfunction:: DkGetSpecialKey :project: pal diff --git a/LibOS/shim/include/shim_fs_encrypted.h b/LibOS/shim/include/shim_fs_encrypted.h index affeceb73b..8c7126e561 100644 --- a/LibOS/shim/include/shim_fs_encrypted.h +++ b/LibOS/shim/include/shim_fs_encrypted.h @@ -171,4 +171,9 @@ int encrypted_file_rename(struct shim_encrypted_file* enc, const char* new_uri); int encrypted_file_get_size(struct shim_encrypted_file* enc, file_off_t* out_size); int encrypted_file_set_size(struct shim_encrypted_file* enc, file_off_t size); +int parse_pf_key(const char* key_str, pf_key_t* pf_key); + +/* TODO: This function is used only by a feature deprecated in v1.2, remove two versions later. */ +int dump_pf_key(const pf_key_t* pf_key, char* buf, size_t buf_size); + #endif /* SHIM_FS_ENCRYPTED_ */ diff --git a/LibOS/shim/src/fs/chroot/fs.c b/LibOS/shim/src/fs/chroot/fs.c index 782b2be3a4..cc72227c32 100644 --- a/LibOS/shim/src/fs/chroot/fs.c +++ b/LibOS/shim/src/fs/chroot/fs.c @@ -495,74 +495,6 @@ static int chroot_chmod(struct shim_dentry* dent, mode_t perm) { return ret; } -static int chroot_reopen(struct shim_handle* hdl, PAL_HANDLE* out_palhdl) { - PAL_HANDLE palhdl; - - mode_t mode = 0; - enum pal_access access = LINUX_OPEN_FLAGS_TO_PAL_ACCESS(hdl->flags); - enum pal_create_mode create = PAL_CREATE_NEVER; - pal_stream_options_t options = LINUX_OPEN_FLAGS_TO_PAL_OPTIONS(hdl->flags); - int ret = DkStreamOpen(hdl->uri, access, mode, create, options, &palhdl); - if (ret < 0) - return pal_to_unix_errno(ret); - *out_palhdl = palhdl; - return 0; -} - -/* - * Prepare the handle to be sent to child process. If the corresponding file still exists on the - * host, we will not checkpoint its PAL handle, but let the child process open another one. - * - * TODO: this is only necessary because PAL handles for protected files cannot be sent to child - * process (`DkSendHandle`). This workaround limits the damage: inheriting a handle by child process - * will now fail to work only if it's a handle for a protected file *and* the file has been deleted - * from host. - */ -static int chroot_checkout(struct shim_handle* hdl) { - assert(hdl->type == TYPE_CHROOT); - assert(hdl->pal_handle); - - /* We should be holding `g_dcache_lock` for the whole checkpointing process. */ - assert(locked(&g_dcache_lock)); - - /* We don't take `hdl->lock` because this is actually the handle *copied* for checkpointing (and - * the lock isn't even properly initialized). */ - - /* First, check if we have not deleted or renamed the file (the dentry contains the same - * inode). */ - bool is_in_dentry = (hdl->dentry->inode == hdl->inode); - - if (is_in_dentry) { - /* Then check if the file still exists on host. If so, we assume it can be opened by the - * child process, so the PAL handle doesn't need sending. */ - PAL_STREAM_ATTR attr; - if (DkStreamAttributesQuery(hdl->uri, &attr) == 0) { - hdl->pal_handle = NULL; - } - } - - return 0; -} - -static int chroot_checkin(struct shim_handle* hdl) { - assert(hdl->type == TYPE_CHROOT); - - /* We don't take `hdl->lock` because this handle is being initialized (during checkpoint - * restore). */ - - if (!hdl->pal_handle) { - PAL_HANDLE palhdl = NULL; - int ret = chroot_reopen(hdl, &palhdl); - if (ret < 0) { - log_warning("%s: failed to open %s: %d", __func__, hdl->uri, ret); - return ret; - } - assert(palhdl); - hdl->pal_handle = palhdl; - } - return 0; -} - struct shim_fs_ops chroot_fs_ops = { .mount = &chroot_mount, .flush = &chroot_flush, @@ -576,8 +508,6 @@ struct shim_fs_ops chroot_fs_ops = { .hstat = &generic_inode_hstat, .truncate = &chroot_truncate, .poll = &generic_inode_poll, - .checkout = &chroot_checkout, - .checkin = &chroot_checkin, }; struct shim_d_ops chroot_d_ops = { diff --git a/LibOS/shim/src/fs/dev/attestation.c b/LibOS/shim/src/fs/dev/attestation.c index c8fd53d771..f85aabcf9d 100644 --- a/LibOS/shim/src/fs/dev/attestation.c +++ b/LibOS/shim/src/fs/dev/attestation.c @@ -33,9 +33,6 @@ static size_t g_target_info_size = 0; static size_t g_report_size = 0; -#define PF_KEY_HEX_SIZE 32 -static char g_pf_key_hex[PF_KEY_HEX_SIZE] = {0}; - static int init_attestation_struct_sizes(void) { if (g_user_report_data_size && g_target_info_size && g_report_size) { /* already initialized, nothing to do here */ @@ -239,46 +236,65 @@ static int quote_load(struct shim_dentry* dent, char** out_data, size_t* out_siz return 0; } -static int pfkey_load(struct shim_dentry* dent, char** out_data, size_t* out_size) { +static int deprecated_pfkey_load(struct shim_dentry* dent, char** out_data, size_t* out_size) { __UNUSED(dent); - size_t size = sizeof(g_pf_key_hex); - char* pf_key_hex = malloc(size); - if (!pf_key_hex) - return -ENOMEM; + int ret; + + struct shim_encrypted_files_key* key; + ret = get_or_create_encrypted_files_key("default", &key); + if (ret < 0) + return ret; + + pf_key_t pf_key; + bool is_set = read_encrypted_files_key(key, &pf_key); + if (is_set) { + size_t buf_size = sizeof(pf_key) * 2 + 1; + char* buf = malloc(buf_size); + if (!buf) + return -ENOMEM; - memcpy(pf_key_hex, &g_pf_key_hex, size); - *out_data = pf_key_hex; - *out_size = size; + ret = dump_pf_key(&pf_key, buf, buf_size); + if (ret < 0) { + free(buf); + return -EACCES; + } + + /* NOTE: we disregard the null terminator here, the caller expects raw data */ + *out_data = buf; + *out_size = sizeof(pf_key) * 2; + } else { + *out_data = NULL; + *out_size = 0; + } return 0; } -/*! - * \brief Set new wrap key (master key) for protected files. - * - * This file must be open for write after successful remote attestation and secret provisioning. - * Typically, the remote user/service provisions the PF key as part of remote attestation before - * the user application starts running. The PF key is applied when this file is closed. - * - * The PF key must be a 32-char null-terminated AES-GCM encryption key in hex format. - */ -static int pfkey_save(struct shim_dentry* dent, const char* data, size_t size) { +static int deprecated_pfkey_save(struct shim_dentry* dent, const char* data, size_t size) { __UNUSED(dent); - if (size != sizeof(g_pf_key_hex)) { + int ret; + + struct shim_encrypted_files_key* key; + ret = get_or_create_encrypted_files_key("default", &key); + if (ret < 0) + return ret; + + pf_key_t pf_key; + if (size != sizeof(pf_key) * 2) { log_debug("/dev/attestation/protected_files_key: invalid length"); return -EACCES; } - /* Build a null-terminated string and pass it to `DkSetProtectedFilesKey`. */ - char buffer[sizeof(g_pf_key_hex) + 1]; - memcpy(buffer, data, sizeof(g_pf_key_hex)); - buffer[sizeof(g_pf_key_hex)] = '\0'; - int ret = DkSetProtectedFilesKey(buffer); + char* key_str = alloc_substr(data, size); + if (!key_str) + return -ENOMEM; + ret = parse_pf_key(key_str, &pf_key); + free(key_str); if (ret < 0) return -EACCES; - memcpy(g_pf_key_hex, data, sizeof(g_pf_key_hex)); + update_encrypted_files_key(key, &pf_key); return 0; } @@ -368,10 +384,11 @@ int init_attestation(struct pseudo_node* dev) { pseudo_add_str(attestation, "report", &report_load); pseudo_add_str(attestation, "quote", "e_load); - struct pseudo_node* pfkey = pseudo_add_str(attestation, "protected_files_key", - &pfkey_load); - pfkey->perm = PSEUDO_PERM_FILE_RW; - pfkey->str.save = &pfkey_save; + /* TODO: This file is deprecated in v1.2, remove 2 versions later. */ + struct pseudo_node* deprecated_pfkey = pseudo_add_str(attestation, "protected_files_key", + &deprecated_pfkey_load); + deprecated_pfkey->perm = PSEUDO_PERM_FILE_RW; + deprecated_pfkey->str.save = &deprecated_pfkey_save; } struct pseudo_node* keys = pseudo_add_dir(attestation, "keys"); diff --git a/LibOS/shim/src/fs/shim_fs.c b/LibOS/shim/src/fs/shim_fs.c index 72f0d028ac..d295c3518c 100644 --- a/LibOS/shim/src/fs/shim_fs.c +++ b/LibOS/shim/src/fs/shim_fs.c @@ -417,6 +417,140 @@ static int mount_nonroot_from_toml_array(void) { return 0; } +/* + * Find where a file with given URI was supposed to be mounted. For instance, if the URI is + * `file:a/b/c/d`, and there is a mount `{ uri = "file:a/b", path = "/x/y" }`, then this function + * will set `*out_file_path` to `/x/y/c/d`. Only "chroot" mounts are considered. + * + * The caller is supposed to free `*out_file_path`. + * + * This function is used for interpreting legacy `sgx.protected_files` syntax as mounts. + */ +static int find_host_file_mount_path(const char* uri, char** out_file_path) { + if (!strstartswith(uri, URI_PREFIX_FILE)) + return -EINVAL; + + size_t uri_len = strlen(uri); + + bool found = false; + char* file_path = NULL; + + /* Traverse the mount list in reverse: we want to find the latest mount that applies. */ + struct shim_mount* mount; + lock(&mount_list_lock); + LISTP_FOR_EACH_ENTRY_REVERSE(mount, &mount_list, list) { + if (strcmp(mount->fs->name, "chroot") != 0 || !strstartswith(mount->uri, URI_PREFIX_FILE)) + continue; + + const char* mount_uri = mount->uri; + size_t mount_uri_len = strlen(mount_uri); + + /* Check if `mount_uri` is equal to `uri`, or is an ancestor of `uri` */ + if (mount_uri_len <= uri_len && !memcmp(mount_uri, uri, mount_uri_len) && + (!uri[mount_uri_len] || uri[mount_uri_len] == '/')) { + /* `rest` is either empty, or begins with '/' */ + const char* rest = uri + mount_uri_len; + found = true; + file_path = alloc_concat(mount->path, -1, rest, -1); + break; + } + + /* Special case: this is the mount of the current directory, and `uri` is not absolute */ + if (!strcmp(mount_uri, "file:.") && uri[static_strlen(URI_PREFIX_FILE)] != '/') { + found = true; + file_path = strdup(uri + static_strlen(URI_PREFIX_FILE)); + break; + } + } + unlock(&mount_list_lock); + + if (!found) + return -ENOENT; + + if (!file_path) + return -ENOMEM; + + *out_file_path = file_path; + return 0; +} + +/* + * Parse an array of SGX protected files (`sgx.{array_name}`) and interpret every entry as a `type = + * "encrypted"` mount. + * + * NOTE: This covers only the simplest cases. It will not work correctly if a given file was mounted + * multiple times, or mounted under a path shadowed by another mount. + */ +static int mount_protected_files(const char* array_name, const char* key_name) { + assert(g_manifest_root); + toml_table_t* manifest_sgx = toml_table_in(g_manifest_root, "sgx"); + if (!manifest_sgx) + return 0; + + toml_array_t* manifest_sgx_protected_files = toml_array_in(manifest_sgx, array_name); + if (!manifest_sgx_protected_files) + return 0; + + ssize_t protected_files_cnt = toml_array_nelem(manifest_sgx_protected_files); + if (protected_files_cnt < 0) + return -EINVAL; + + if (protected_files_cnt == 0) + return 0; + + log_error("Detected deprecated syntax: 'sgx.%s'. Consider converting it to a list " + "of mounts with 'type = \"encrypted\"' and 'key_name = \"%s\"'.", + array_name, key_name); + + int ret; + char* uri = NULL; + char* mount_path = NULL; + for (ssize_t i = 0; i < protected_files_cnt; i++) { + toml_raw_t uri_raw = toml_raw_at(manifest_sgx_protected_files, i); + if (!uri_raw) { + log_error("Cannot parse 'sgx.%s[%zd]'", array_name, i); + ret = -EINVAL; + goto out; + } + + ret = toml_rtos(uri_raw, &uri); + if (ret < 0) { + log_error("Cannot parse 'sgx.%s[%zd]' (not a string)", array_name, i); + ret = -EINVAL; + goto out; + } + + ret = find_host_file_mount_path(uri, &mount_path); + if (ret < 0) { + log_error("Cannot determine mount path for encrypted file '%s'. " + "Consider converting 'sgx.%s' to a list of mounts, as mentioned above.", + uri, array_name); + goto out; + } + + struct shim_mount_params params = { + .type = "encrypted", + .path = mount_path, + .uri = uri, + .key_name = key_name, + }; + ret = mount_fs(¶ms); + if (ret < 0) { + ret = -EINVAL; + goto out; + } + + free(uri); + uri = NULL; + free(mount_path); + mount_path = NULL; + } + ret = 0; +out: + free(uri); + free(mount_path); + return ret; +} int init_mount_root(void) { if (mount_migrated) @@ -449,6 +583,23 @@ int init_mount(void) { if (ret < 0) return ret; + /* + * If we're under SGX PAL, handle old protected files syntax. + * + * TODO: this is deprecated in v1.2, remove two versions later. + */ + if (!strcmp(g_pal_public_state->host_type, "Linux-SGX")) { + ret = mount_protected_files("protected_files", "default"); + if (ret < 0) + return ret; + ret = mount_protected_files("protected_mrenclave_files", "_sgx_mrenclave"); + if (ret < 0) + return ret; + ret = mount_protected_files("protected_mrsigner_files", "_sgx_mrsigner"); + if (ret < 0) + return ret; + } + assert(g_manifest_root); char* fs_start_dir = NULL; diff --git a/LibOS/shim/src/fs/shim_fs_encrypted.c b/LibOS/shim/src/fs/shim_fs_encrypted.c index 3bab63cff4..386329674e 100644 --- a/LibOS/shim/src/fs/shim_fs_encrypted.c +++ b/LibOS/shim/src/fs/shim_fs_encrypted.c @@ -196,12 +196,12 @@ static int encrypted_file_internal_open(struct shim_encrypted_file* enc, PAL_HAN return 0; } -static int parse_pf_key(const char* key_str, pf_key_t* pf_key) { +int parse_pf_key(const char* key_str, pf_key_t* pf_key) { size_t len = strlen(key_str); - if (len != PF_KEY_SIZE * 2) { + if (len != sizeof(*pf_key) * 2) { log_warning("%s: wrong key length (%zu instead of %zu)", __func__, len, - (size_t)(PF_KEY_SIZE * 2)); - return -1; + (size_t)(sizeof(*pf_key) * 2)); + return -EINVAL; } pf_key_t tmp_pf_key; @@ -210,7 +210,7 @@ static int parse_pf_key(const char* key_str, pf_key_t* pf_key) { int8_t lo = hex2dec(key_str[i+1]); if (hi < 0 || lo < 0) { log_warning("%s: unexpected character encountered", __func__); - return -1; + return -EINVAL; } tmp_pf_key[i / 2] = hi * 16 + lo; } @@ -218,6 +218,14 @@ static int parse_pf_key(const char* key_str, pf_key_t* pf_key) { return 0; } +int dump_pf_key(const pf_key_t* pf_key, char* buf, size_t buf_size) { + if (buf_size < sizeof(*pf_key) * 2 + 1) + return -EINVAL; + + __bytes2hexstr(pf_key, sizeof(*pf_key), buf, buf_size); + return 0; +} + static void encrypted_file_internal_close(struct shim_encrypted_file* enc) { assert(enc->pf); @@ -232,6 +240,23 @@ static void encrypted_file_internal_close(struct shim_encrypted_file* enc) { return; } +static int parse_and_update_key(const char* key_name, const char* key_str) { + pf_key_t pf_key; + int ret = parse_pf_key(key_str, &pf_key); + if (ret < 0) { + log_error("Cannot parse hex key: '%s'", key_str); + return ret; + } + + struct shim_encrypted_files_key* key; + ret = get_or_create_encrypted_files_key(key_name, &key); + if (ret < 0) + return ret; + + update_encrypted_files_key(key, &pf_key); + return 0; +} + int init_encrypted_files(void) { pf_debug_f cb_debug_ptr = NULL; #ifdef DEBUG @@ -244,58 +269,63 @@ int init_encrypted_files(void) { &cb_aes_cmac, &cb_aes_gcm_encrypt, &cb_aes_gcm_decrypt, &cb_random, cb_debug_ptr); - /* Parse `fs.insecure__keys.*` */ - - toml_table_t* manifest_fs = toml_table_in(g_manifest_root, "fs"); - if (!manifest_fs) - return 0; - - toml_table_t* manifest_fs_keys = toml_table_in(manifest_fs, "insecure__keys"); - if (!manifest_fs_keys) - return 0; - - ssize_t keys_cnt = toml_table_nkval(manifest_fs_keys); - if (keys_cnt < 0) - return -EINVAL; - int ret; - char* key_str = NULL; - for (ssize_t i = 0; i < keys_cnt; i++) { - const char* name = toml_key_in(manifest_fs_keys, i); - assert(name); + /* Parse `fs.insecure__keys.*` */ - struct shim_encrypted_files_key* key; - ret = get_or_create_encrypted_files_key(name, &key); - if (ret < 0) - goto out; + toml_table_t* manifest_fs = toml_table_in(g_manifest_root, "fs"); + toml_table_t* manifest_fs_keys = + manifest_fs ? toml_table_in(manifest_fs, "insecure__keys") : NULL; + if (manifest_fs && manifest_fs_keys) { + ssize_t keys_cnt = toml_table_nkval(manifest_fs_keys); + if (keys_cnt < 0) + return -EINVAL; + + for (ssize_t i = 0; i < keys_cnt; i++) { + const char* key_name = toml_key_in(manifest_fs_keys, i); + assert(key_name); + + char* key_str; + ret = toml_string_in(manifest_fs_keys, key_name, &key_str); + if (ret < 0) { + log_error("Cannot parse 'fs.insecure__keys.%s'", key_name); + return -EINVAL; + } + assert(key_str); - ret = toml_string_in(manifest_fs_keys, name, &key_str); - if (ret < 0) { - log_error("Cannot parse 'fs.insecure__keys.%s'", name); - ret = -EINVAL; - goto out; + ret = parse_and_update_key(key_name, key_str); + free(key_str); + if (ret < 0) + return ret; } - assert(key_str); + } - pf_key_t pf_key; - ret = parse_pf_key(key_str, &pf_key); + /* + * If we're under SGX PAL, parse `sgx.insecure__protected_files_key` (and interpret it as the + * "default" key). + * + * TODO: this is deprecated in v1.2, remove two versions later. + */ + if (!strcmp(g_pal_public_state->host_type, "Linux-SGX")) { + char* key_str; + ret = toml_string_in(g_manifest_root, "sgx.insecure__protected_files_key", &key_str); if (ret < 0) { - log_error("Cannot parse key 'fs.insecure__keys.%s' as a hex key", name); - ret = -EINVAL; - goto out; + log_error("Cannot parse 'sgx.insecure__protected_files_key'"); + return -EINVAL; } - update_encrypted_files_key(key, &pf_key); - free(key_str); - key_str = NULL; - } + if (key_str) { + log_error("Detected deprecated syntax: 'sgx.insecure__protected_files_key'. " + "Consider converting it to 'fs.insecure__keys.default'."); - ret = 0; + ret = parse_and_update_key("default", key_str); + free(key_str); + if (ret < 0) + return ret; + } + } -out: - free(key_str); - return ret; + return 0; } static struct shim_encrypted_files_key* get_key(const char* name) { diff --git a/LibOS/shim/test/fs/README.md b/LibOS/shim/test/fs/README.md index 7c5f6ebf65..42f38f53a3 100644 --- a/LibOS/shim/test/fs/README.md +++ b/LibOS/shim/test/fs/README.md @@ -1,7 +1,8 @@ Test purpose ------------ -These tests perform common FS operations in various ways to exercise the Gramine FS subsystem: +These tests perform common FS operations in various ways to exercise the Gramine +FS subsystem: - open/close - read/write @@ -15,12 +16,10 @@ These tests perform common FS operations in various ways to exercise the Gramine How to execute -------------- -- `make test` to run all tests -- `make fs-test` to test regular files -- `make tmpfs-test` to test tmpfs (temporary in-memory) files -- `make pf-test` to test protected files (SGX only) +- `gramine-test pytest -v` -(SGX only) Protected file tests assume that the SGX tools were installed in this directory: +Encrypted file tests assume that Gramine was built with SGX enabled (see comment +in `test_enc.py`). ``` cd $gramine/Pal/src/host/Linux-SGX/tools diff --git a/LibOS/shim/test/fs/manifest.template b/LibOS/shim/test/fs/manifest.template index 7b50eaaec2..8eabe102df 100644 --- a/LibOS/shim/test/fs/manifest.template +++ b/LibOS/shim/test/fs/manifest.template @@ -11,9 +11,15 @@ fs.mounts = [ { path = "/usr/{{ arch_libdir }}", uri = "file:/usr/{{ arch_libdir }}" }, { path = "/mounted", uri = "file:tmp" }, + { type = "encrypted", path = "/tmp/enc_input", uri = "file:tmp/enc_input" }, + { type = "encrypted", path = "/tmp/enc_output", uri = "file:tmp/enc_output" }, + { type = "encrypted", path = "/mounted/enc_input", uri = "file:tmp/enc_input" }, + { type = "encrypted", path = "/mounted/enc_output", uri = "file:tmp/enc_output" }, { type = "tmpfs", path = "/mnt-tmpfs" }, ] +fs.insecure__keys.default = "ffeeddccbbaa99887766554433221100" + sgx.nonpie_binary = true sgx.debug = true sgx.thread_num = 16 @@ -28,9 +34,3 @@ sgx.trusted_files = [ "file:{{ gramine.runtimedir() }}/", "file:{{ arch_libdir }}/libgcc_s.so.1", ] - -sgx.insecure__protected_files_key = "ffeeddccbbaa99887766554433221100" -sgx.protected_files = [ - "file:tmp/pf_input", - "file:tmp/pf_output", -] diff --git a/LibOS/shim/test/fs/test_pf.py b/LibOS/shim/test/fs/test_enc.py similarity index 85% rename from LibOS/shim/test/fs/test_pf.py rename to LibOS/shim/test/fs/test_enc.py index 2f9fcd253c..77479c3a02 100644 --- a/LibOS/shim/test/fs/test_pf.py +++ b/LibOS/shim/test/fs/test_enc.py @@ -7,13 +7,14 @@ # Named import, so that Pytest does not pick up TC_00_FileSystem as belonging to this module. import test_fs -from graminelibos.regression import ( - HAS_SGX, - expectedFailureIf, -) +from graminelibos import _CONFIG_SGX_ENABLED -@unittest.skipUnless(HAS_SGX, 'Protected files require SGX support') -class TC_50_ProtectedFiles(test_fs.TC_00_FileSystem): +# TODO: While encrypted files are no longer SGX-only, the related tools (gramine-sgx-pf-crypt, +# gramine-sgx-pf-tamper) are still part of Linux-SGX PAL. As a result, we are able to run the tests +# with other PALs, but only if Gramine was built with SGX enabled. + +@unittest.skipUnless(_CONFIG_SGX_ENABLED, 'Encrypted files tests require SGX to be enabled') +class TC_50_EncryptedFiles(test_fs.TC_00_FileSystem): @classmethod def setUpClass(cls): super().setUpClass() @@ -24,13 +25,13 @@ def setUpClass(cls): # CONST_WRAP_KEY must match the one in manifest cls.CONST_WRAP_KEY = [0xff, 0xee, 0xdd, 0xcc, 0xbb, 0xaa, 0x99, 0x88, 0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11, 0x00] - cls.ENCRYPTED_DIR = os.path.join(cls.TEST_DIR, 'pf_input') + cls.ENCRYPTED_DIR = os.path.join(cls.TEST_DIR, 'enc_input') cls.ENCRYPTED_FILES = [os.path.join(cls.ENCRYPTED_DIR, str(v)) for v in cls.FILE_SIZES] cls.LIB_PATH = os.path.join(os.getcwd(), 'lib') if not os.path.exists(cls.ENCRYPTED_DIR): os.mkdir(cls.ENCRYPTED_DIR) - cls.OUTPUT_DIR = os.path.join(cls.TEST_DIR, 'pf_output') + cls.OUTPUT_DIR = os.path.join(cls.TEST_DIR, 'enc_output') cls.OUTPUT_FILES = [os.path.join(cls.OUTPUT_DIR, str(x)) for x in cls.FILE_SIZES] # create encrypted files cls.__set_default_key(cls) @@ -82,24 +83,18 @@ def test_010_encrypt_decrypt(self): # overrides TC_00_FileSystem to change input dir (from plaintext to encrypted) def test_100_open_close(self): - # the test binary expects a path to read-only (existing) file or a path to file that - # will get created input_path = self.ENCRYPTED_FILES[-1] # existing file output_path = os.path.join(self.OUTPUT_DIR, 'test_100') # new file stdout, stderr = self.run_binary(['open_close', 'R', input_path]) self.verify_open_close(stdout, stderr, input_path, 'input') - # the following test tries to open multiple handles to a single writable PF, should fail - try: - stdout, stderr = self.run_binary(['open_close', 'W', output_path]) - self.assertIn('ERROR: Failed to open output file', stderr) - except subprocess.CalledProcessError as exc: - self.assertNotEqual(exc.returncode, 0) - self.assertTrue(os.path.isfile(output_path)) - else: - print('[!] Fail: open_close returned 0') - self.fail() + stdout, stderr = self.run_binary(['open_close', 'W', output_path]) + self.verify_open_close(stdout, stderr, output_path, 'output') + self.assertTrue(os.path.isfile(output_path)) # overrides TC_00_FileSystem to change input dir (from plaintext to encrypted) + # doesn't work because encrypted files do not support truncation (and the test opens an + # existing, non-empty file with O_TRUNC) + @unittest.expectedFailure def test_101_open_flags(self): # the test binary expects a path to file that will get created file_path = os.path.join(self.OUTPUT_DIR, 'test_101') # new file @@ -138,7 +133,7 @@ def verify_size(self, file, size): self.__decrypt_file(file, dec_path) self.assertEqual(os.stat(dec_path).st_size, size) - @expectedFailureIf(HAS_SGX) + @unittest.expectedFailure # pylint: disable=fixme def test_140_file_truncate(self): self.fail() # TODO: port these to the new file format @@ -170,30 +165,24 @@ def do_copy_test(self, executable, timeout): timeout=timeout) self.verify_copy(stdout, stderr, self.ENCRYPTED_DIR, executable) - # TODO: `mmap` on protected files is broken, because we fail to properly register that memory is - # unmapped. The `copy_mmap*` tests technically work, but trigger AddressSanitizer. - # overrides TC_00_FileSystem to not skip this on SGX - @unittest.skipIf(os.environ.get('ASAN') == '1', 'mapping protected files is broken') def test_204_copy_dir_mmap_whole(self): self.do_copy_test('copy_mmap_whole', 30) # overrides TC_00_FileSystem to not skip this on SGX - @unittest.skipIf(os.environ.get('ASAN') == '1', 'mapping protected files is broken') def test_205_copy_dir_mmap_seq(self): self.do_copy_test('copy_mmap_seq', 60) # overrides TC_00_FileSystem to not skip this on SGX - @unittest.skipIf(os.environ.get('ASAN') == '1', 'mapping protected files is broken') def test_206_copy_dir_mmap_rev(self): self.do_copy_test('copy_mmap_rev', 60) # overrides TC_00_FileSystem to change dirs (from plaintext to encrypted) def test_210_copy_dir_mounted(self): executable = 'copy_whole' - stdout, stderr = self.run_binary([executable, '/mounted/pf_input', '/mounted/pf_output'], + stdout, stderr = self.run_binary([executable, '/mounted/enc_input', '/mounted/enc_output'], timeout=30) - self.verify_copy(stdout, stderr, '/mounted/pf_input', executable) + self.verify_copy(stdout, stderr, '/mounted/enc_input', executable) def __corrupt_file(self, input_path, output_path): cmd = [self.PF_TAMPER, '-w', self.WRAP_KEY, '-i', input_path, '-o', output_path] @@ -201,7 +190,7 @@ def __corrupt_file(self, input_path, output_path): # invalid/corrupted files def test_500_invalid(self): - invalid_dir = os.path.join(self.TEST_DIR, 'pf_invalid') + invalid_dir = os.path.join(self.TEST_DIR, 'enc_invalid') if not os.path.exists(invalid_dir): os.mkdir(invalid_dir) diff --git a/LibOS/shim/test/regression/attestation.c b/LibOS/shim/test/regression/attestation.c index 1edeeec111..938917620f 100644 --- a/LibOS/shim/test/regression/attestation.c +++ b/LibOS/shim/test/regression/attestation.c @@ -21,6 +21,13 @@ // #include "mbedtls/base64.h" // #include "mbedtls/cmac.h" +#define MANIFEST_KEY "ffeeddccbbaa99887766554433221100" +#define NEW_KEY "00112233445566778899aabbccddeeff" + +#define KEY_LEN 32 + +#define KEY_PATH "/dev/attestation/protected_files_key" + typedef enum { MBEDTLS_CIPHER_AES_128_ECB = 2, } mbedtls_cipher_type_t; @@ -95,6 +102,67 @@ static int verify_report_mac(sgx_report_t* report) { return SUCCESS; } +static int expect_key(const char* expected_key) { + assert(strlen(expected_key) == KEY_LEN); + + char buf[KEY_LEN]; + ssize_t bytes = file_read_f(KEY_PATH, buf, sizeof(buf)); + if (bytes < 0) { + fprintf(stderr, "Error reading %s\n", KEY_PATH); + return FAILURE; + } + if ((size_t)bytes != sizeof(buf)) { + fprintf(stderr, "Failed: %s has wrong size\n", KEY_PATH); + return FAILURE; + } + if (memcmp(buf, expected_key, sizeof(buf))) { + fprintf(stderr, "Failed: %s has wrong content (expected \"%s\", got \"%.*s\")\n", KEY_PATH, + expected_key, (int)sizeof(buf), buf); + return FAILURE; + } + return SUCCESS; +} + +static int write_key(const char* key) { + assert(strlen(key) == KEY_LEN); + + ssize_t bytes = file_write_f(KEY_PATH, key, KEY_LEN); + if (bytes < 0) { + fprintf(stderr, "Error writing %s\n", KEY_PATH); + return FAILURE; + } + if ((size_t)bytes != KEY_LEN) { + fprintf(stderr, "Failed: not enough bytes written to %s\n", KEY_PATH); + return FAILURE; + } + return SUCCESS; +} + + +/* + * Test the deprecated `/dev/attestation/protected_files_key` file (and setting the initial key + * using deprecated `sgx.insecure__protected_files_key` manifest syntax). + * + * TODO: remove this part of the test when these deprecated interfaces are removed (two versions + * after v1.2). The new way of setting keys (`/dev/attestation/keys`, `fs.insecure__keys`) is + * already tested in `keys.c`. + */ +static int test_protected_files_key(void) { + int ret = expect_key(MANIFEST_KEY); + if (ret != SUCCESS) + return ret; + + ret = write_key(NEW_KEY); + if (ret != SUCCESS) + return ret; + + ret = expect_key(NEW_KEY); + if (ret != SUCCESS) + return ret; + + return SUCCESS; +} + /*! * \brief Test local attestation interface. * @@ -259,6 +327,8 @@ int main(int argc, char** argv) { file_write_f = stdio_file_write; } + printf("Test protected_files_key... %s\n", + test_protected_files_key() == SUCCESS ? "SUCCESS" : "FAIL"); printf("Test local attestation... %s\n", test_local_attestation() == SUCCESS ? "SUCCESS" : "FAIL"); printf("Test quote interface... %s\n", diff --git a/LibOS/shim/test/regression/attestation.manifest.template b/LibOS/shim/test/regression/attestation.manifest.template index df9a44fc5e..377269b4df 100644 --- a/LibOS/shim/test/regression/attestation.manifest.template +++ b/LibOS/shim/test/regression/attestation.manifest.template @@ -20,6 +20,8 @@ fs.mount.bin.type = "chroot" fs.mount.bin.path = "/bin" fs.mount.bin.uri = "file:/bin" +sgx.insecure__protected_files_key = "ffeeddccbbaa99887766554433221100" + sgx.nonpie_binary = true sgx.debug = true sgx.remote_attestation = true diff --git a/Pal/include/pal/pal.h b/Pal/include/pal/pal.h index 63cc7c8140..c744876461 100644 --- a/Pal/include/pal/pal.h +++ b/Pal/include/pal/pal.h @@ -256,7 +256,7 @@ typedef uint32_t pal_stream_options_t; /* bitfield */ #define PAL_OPTION_EFD_SEMAPHORE 0x2 /*!< specific to `eventfd` syscall */ #define PAL_OPTION_NONBLOCK 0x4 #define PAL_OPTION_DUALSTACK 0x8 /*!< Create dual-stack socket (opposite of IPV6_V6ONLY) */ -#define PAL_OPTION_PASSTHROUGH 0x10 /*!< Disregard `sgx.{allowed,trusted,protected}_files` */ +#define PAL_OPTION_PASSTHROUGH 0x10 /*!< Disregard `sgx.{allowed,trusted}_files` */ #define PAL_OPTION_MASK 0x1F /*! @@ -749,18 +749,6 @@ int DkAttestationReport(const void* user_report_data, PAL_NUM* user_report_data_ */ int DkAttestationQuote(const void* user_report_data, PAL_NUM user_report_data_size, void* quote, PAL_NUM* quote_size); - -/*! - * \brief Set wrap key (master key) for protected files. - * - * \param[in] pf_key_hex Wrap key for protected files. Must be a 32-char null-terminated hex string - * in case of SGX PAL (AES-GCM encryption key). - * - * Currently works only for Linux-SGX PAL. This function is supposed to be called during - * remote attestation and secret provisioning, before the user application starts. - */ -int DkSetProtectedFilesKey(const char* pf_key_hex); - /*! * \brief Get special key (specific to PAL host). * diff --git a/Pal/include/pal_internal.h b/Pal/include/pal_internal.h index 27d2d26cce..77f02b0803 100644 --- a/Pal/include/pal_internal.h +++ b/Pal/include/pal_internal.h @@ -232,7 +232,6 @@ int _DkAttestationReport(const void* user_report_data, PAL_NUM* user_report_data PAL_NUM* report_size); int _DkAttestationQuote(const void* user_report_data, PAL_NUM user_report_data_size, void* quote, PAL_NUM* quote_size); -int _DkSetProtectedFilesKey(const char* pf_key_hex); int _DkGetSpecialKey(const char* name, void* key, size_t* key_size); #define INIT_FAIL(exitcode, reason) \ diff --git a/Pal/src/db_main.c b/Pal/src/db_main.c index efe4f87708..b3292590c9 100644 --- a/Pal/src/db_main.c +++ b/Pal/src/db_main.c @@ -443,7 +443,7 @@ noreturn void pal_main(uint64_t instance_id, /* current instance id */ if (argv_src_file) { /* Load argv from a file and discard cmdline argv. We trust the file contents (this can - * be achieved using protected or trusted files). */ + * be achieved using trusted files). */ if (arguments[0] && arguments[1]) log_error("Discarding cmdline arguments (%s %s [...]) because loader.argv_src_file " "was specified in the manifest.", arguments[0], arguments[1]); @@ -481,7 +481,7 @@ noreturn void pal_main(uint64_t instance_id, /* current instance id */ if (env_src_file) { /* Insert environment variables from a file. We trust the file contents (this can be - * achieved using protected or trusted files). */ + * achieved using trusted files). */ ret = load_cstring_array(env_src_file, &orig_environments); if (ret < 0) INIT_FAIL(-ret, "Cannot load environment variables from 'loader.env_src_file'"); diff --git a/Pal/src/db_misc.c b/Pal/src/db_misc.c index a41116fb28..3aa33e80ff 100644 --- a/Pal/src/db_misc.c +++ b/Pal/src/db_misc.c @@ -50,10 +50,6 @@ int DkAttestationQuote(const void* user_report_data, PAL_NUM user_report_data_si return _DkAttestationQuote(user_report_data, user_report_data_size, quote, quote_size); } -int DkSetProtectedFilesKey(const char* pf_key_hex) { - return _DkSetProtectedFilesKey(pf_key_hex); -} - int DkGetSpecialKey(const char* name, void* key, size_t* key_size) { return _DkGetSpecialKey(name, key, key_size); } diff --git a/Pal/src/db_rtld.c b/Pal/src/db_rtld.c index d7c21071c4..61a92a15fe 100644 --- a/Pal/src/db_rtld.c +++ b/Pal/src/db_rtld.c @@ -605,15 +605,10 @@ void set_pal_binary_name(const char* name) { } /* - * TODO: This function assumes that a "file:" URI describes a path that can be opened on a host - * directly (e.g. by GDB or other tools). This is mostly true, except for protected files in - * Linux-SGX, which are stored encrypted. As a result, if we load a binary that is a protected file, - * we will (incorrectly) report the encrypted file as the actual binary, and code that tries to - * parse this file will trip up. - * - * For now, this doesn't seem worth fixing, as there's no use case for running binaries from - * protected files system, and a workaround would be ugly. Instead, the protected files system needs - * rethinking. + * NOTE: This function assumes that a "file:" URI describes a path that can be opened on a host + * directly (e.g. by GDB or other tools). This is mostly true, except for LibOS encrypted files. As + * a result, if we load a binary that is an encrypted file, it will be reported here, but GDB (and + * other tools) will fail to load it. */ void DkDebugMapAdd(const char* uri, void* start_addr) { #ifndef DEBUG diff --git a/Pal/src/host/Linux-SGX/db_files.c b/Pal/src/host/Linux-SGX/db_files.c index 12b53a5436..7cd0291332 100644 --- a/Pal/src/host/Linux-SGX/db_files.c +++ b/Pal/src/host/Linux-SGX/db_files.c @@ -12,7 +12,6 @@ #include "api.h" #include "asan.h" -#include "enclave_pf.h" #include "enclave_tf.h" #include "pal.h" #include "pal_error.h" @@ -75,16 +74,14 @@ static int file_open(PAL_HANDLE* handle, const char* type, const char* uri, free(normpath); /* was copied into the file PAL handle object, not needed anymore */ normpath = NULL; - struct protected_file* pf = NULL; struct trusted_file* tf = NULL; if (!(pal_options & PAL_OPTION_PASSTHROUGH)) { - pf = get_protected_file(hdl->file.realpath); tf = get_trusted_or_allowed_file(hdl->file.realpath); - if (!pf && !tf) { + if (!tf) { if (get_file_check_policy() != FILE_CHECK_POLICY_ALLOW_ALL_BUT_LOG) { - log_warning("Disallowing access to file '%s'; file is not protected, trusted or " - "allowed.", hdl->file.realpath); + log_warning("Disallowing access to file '%s'; file is not trusted or allowed.", + hdl->file.realpath); ret = -PAL_ERROR_DENIED; goto fail; } @@ -93,7 +90,7 @@ static int file_open(PAL_HANDLE* handle, const char* type, const char* uri, } } - if (!pf && !tf) { + if (!tf) { fd = ocall_open(uri, flags, pal_share); if (fd < 0) { ret = unix_to_pal_error(fd); @@ -114,64 +111,6 @@ static int file_open(PAL_HANDLE* handle, const char* type, const char* uri, return 0; } - if (pf) { - pf_lock(); - - pf_file_mode_t pf_mode = 0; - if (pal_access == PAL_ACCESS_RDONLY) { - pf_mode = PF_FILE_MODE_READ; - } else if (pal_access == PAL_ACCESS_WRONLY) { - pf_mode = PF_FILE_MODE_WRITE; - } else { - assert(pal_access == PAL_ACCESS_RDWR); - pf_mode = PF_FILE_MODE_READ | PF_FILE_MODE_WRITE; - } - - if ((pf_mode & PF_FILE_MODE_WRITE) && pf->writable_fd >= 0) { - log_warning("file_open(%s): disallowing concurrent writable handle", - hdl->file.realpath); - ret = -PAL_ERROR_DENIED; - goto fail_pf_unlock; - } - - /* Protected files sometimes needs to be read and written (e.g. to read or save some - * metadata), regardless of the requested open mode. */ - flags = (flags & ~O_ACCMODE) | O_RDWR; - - fd = ocall_open(uri, flags, pal_share); - if (fd < 0) { - ret = unix_to_pal_error(fd); - goto fail_pf_unlock; - } - - ret = ocall_fstat(fd, &st); - if (ret < 0) { - ret = unix_to_pal_error(ret); - goto fail_pf_unlock; - } - - hdl->file.fd = fd; - hdl->file.seekable = !S_ISFIFO(st.st_mode); - - pf = load_protected_file(hdl->file.realpath, (int*)&hdl->file.fd, st.st_size, pf_mode, - do_create, pf); - if (!pf) { - log_warning("load_protected_file(%s, %d) failed", hdl->file.realpath, hdl->file.fd); - ret = -PAL_ERROR_DENIED; - goto fail_pf_unlock; - } - - if (pf_mode & PF_FILE_MODE_WRITE) { - pf->writable_fd = fd; - } - - pf->refcount++; - pf_unlock(); - - *handle = hdl; - return 0; - } - assert(tf); /* at this point, we want to open a trusted or allowed file */ if (!tf->allowed && (do_create @@ -215,12 +154,7 @@ static int file_open(PAL_HANDLE* handle, const char* type, const char* uri, *handle = hdl; return 0; -fail_pf_unlock: - pf_unlock(); fail: - if (pf && pf->context && pf->refcount == 0) - unload_protected_file(pf); - if (fd >= 0) ocall_close(fd); @@ -228,33 +162,8 @@ static int file_open(PAL_HANDLE* handle, const char* type, const char* uri, return ret; } -static int64_t pf_file_read(struct protected_file* pf, PAL_HANDLE handle, uint64_t offset, - uint64_t count, void* buffer) { - int fd = handle->file.fd; - - if (!pf->context) { - log_warning("pf_file_read(PF fd %d): PF not initialized", fd); - return -PAL_ERROR_BADHANDLE; - } - - size_t bytes_read = 0; - pf_status_t pfs = pf_read(pf->context, offset, count, buffer, &bytes_read); - - if (PF_FAILURE(pfs)) { - log_warning("pf_file_read(PF fd %d): pf_read failed: %s", fd, pf_strerror(pfs)); - return -PAL_ERROR_DENIED; - } - - return bytes_read; -} - /* 'read' operation for file streams. */ static int64_t file_read(PAL_HANDLE handle, uint64_t offset, uint64_t count, void* buffer) { - struct protected_file* pf = find_protected_file_handle(handle); - - if (pf) - return pf_file_read(pf, handle, offset, count, buffer); - int64_t ret; sgx_chunk_hash_t* chunk_hashes = handle->file.chunk_hashes; @@ -289,32 +198,8 @@ static int64_t file_read(PAL_HANDLE handle, uint64_t offset, uint64_t count, voi return end - offset; } -static int64_t pf_file_write(struct protected_file* pf, PAL_HANDLE handle, uint64_t offset, - uint64_t count, const void* buffer) { - int fd = handle->file.fd; - - if (!pf->context) { - log_warning("pf_file_write(PF fd %d): PF not initialized", fd); - return -PAL_ERROR_BADHANDLE; - } - - pf_status_t pf_ret = pf_write(pf->context, offset, count, buffer); - - if (PF_FAILURE(pf_ret)) { - log_warning("pf_file_write(PF fd %d): pf_write failed: %s", fd, pf_strerror(pf_ret)); - return -PAL_ERROR_DENIED; - } - - return count; -} - /* 'write' operation for file streams. */ static int64_t file_write(PAL_HANDLE handle, uint64_t offset, uint64_t count, const void* buffer) { - struct protected_file* pf = find_protected_file_handle(handle); - - if (pf) - return pf_file_write(pf, handle, offset, count, buffer); - int64_t ret; sgx_chunk_hash_t* chunk_hashes = handle->file.chunk_hashes; @@ -336,37 +221,9 @@ static int64_t file_write(PAL_HANDLE handle, uint64_t offset, uint64_t count, co return -PAL_ERROR_DENIED; } -static int pf_file_close(struct protected_file* pf, PAL_HANDLE handle) { - int fd = handle->file.fd; - - if (pf->refcount == 0) { - log_error("pf_file_close(PF fd %d): refcount == 0", fd); - return -PAL_ERROR_INVAL; - } - - pf_lock(); - if (pf->writable_fd == fd) - pf->writable_fd = -1; - pf_unlock(); - - pf->refcount--; - - if (pf->refcount == 0) - return unload_protected_file(pf); - - return 0; -} - /* 'close' operation for file streams */ static int file_close(PAL_HANDLE handle) { int fd = handle->file.fd; - struct protected_file* pf = find_protected_file_handle(handle); - - if (pf) { - int ret = pf_file_close(pf, handle); - if (ret < 0) - return ret; - } if (handle->file.chunk_hashes && handle->file.total) { /* case of trusted file: the whole file was mmapped in untrusted memory */ @@ -391,110 +248,6 @@ static int file_delete(PAL_HANDLE handle, enum pal_delete_mode delete_mode) { return ret < 0 ? unix_to_pal_error(ret) : ret; } -static int pf_file_map(struct protected_file* pf, PAL_HANDLE handle, void** addr, - pal_prot_flags_t prot, uint64_t offset, uint64_t size) { - int ret = 0; - void* allocated_enclave_pages = NULL; - int fd = handle->file.fd; - - if (size == 0) - return -PAL_ERROR_INVAL; - - assert(WITHIN_MASK(prot, PAL_PROT_MASK)); - if ((prot & PAL_PROT_READ) && (prot & PAL_PROT_WRITE)) { - log_warning("pf_file_map(PF fd %d): trying to map with R+W access", fd); - return -PAL_ERROR_NOTSUPPORT; - } - - if (!pf->context) { - log_warning("pf_file_map(PF fd %d): PF not initialized", fd); - return -PAL_ERROR_BADHANDLE; - } - - uint64_t pf_size; - pf_status_t pfs = pf_get_size(pf->context, &pf_size); - __UNUSED(pfs); - assert(PF_SUCCESS(pfs)); - - log_debug("pf_file_map(PF fd %d): pf %p, addr %p, prot %d, offset %lu, size %lu", fd, pf, - *addr, prot, offset, size); - - if (*addr == NULL) { - /* No address at which to map, can happen when called from `db_rtld.c` */ - allocated_enclave_pages = get_enclave_pages(/*addr=*/NULL, size, /*is_pal_internal=*/false); - if (!allocated_enclave_pages) - return -PAL_ERROR_NOMEM; - - *addr = allocated_enclave_pages; - } else { - /* - * TODO: We should call `get_enclave_pages` here, but it's unclear how we should clean up in - * case of errors, because we don't know if anything was mapped over this range before. - * - * An unconditional `free_enclave_pages` during cleanup would poison this memory range, even - * if was being used before. The current code errs on the side of caution, and always leaves - * this memory unpoisoned (since we keep `allocated_enclave_pages` set to NULL). - * - * A proper implementation should first load the protected file, and allocate pages only - * after that succeeds. - */ -#ifdef ASAN - asan_unpoison_region((uintptr_t)*addr, size); -#endif - } - - if (prot & PAL_PROT_WRITE) { - struct pf_map* map = calloc(1, sizeof(*map)); - if (!map) { - ret = -PAL_ERROR_NOMEM; - goto out; - } - - map->pf = pf; - map->size = size; - map->offset = offset; - map->buffer = *addr; - - pf_lock(); - LISTP_ADD_TAIL(map, &g_pf_map_list, list); - pf_unlock(); - } - - if (prot & PAL_PROT_READ) { - /* we don't check this on writes since file size may be extended then */ - if (offset >= pf_size) { - log_warning("pf_file_map(PF fd %d): offset (%lu) >= file size (%lu)", fd, offset, - pf_size); - ret = -PAL_ERROR_INVAL; - goto out; - } - - uint64_t copy_size = MIN(size, pf_size - offset); - - size_t bytes_read = 0; - pf_status_t pf_ret = pf_read(pf->context, offset, copy_size, *addr, &bytes_read); - if (bytes_read != copy_size) { - /* mapped region must be read completely from file, otherwise it's an error */ - pf_ret = PF_STATUS_CORRUPTED; - } - if (PF_FAILURE(pf_ret)) { - log_warning("pf_file_map(PF fd %d): pf_read failed: %s", fd, pf_strerror(pf_ret)); - ret = -PAL_ERROR_DENIED; - goto out; - } - memset(*addr + copy_size, 0, size - copy_size); - } - - /* Writes will be flushed to the PF on close. */ - ret = 0; -out: - if (ret < 0 && allocated_enclave_pages) { - free_enclave_pages(allocated_enclave_pages, size); - *addr = NULL; - } - return ret; -} - /* 'map' operation for file stream. */ static int file_map(PAL_HANDLE handle, void** addr, pal_prot_flags_t prot, uint64_t offset, uint64_t size) { @@ -511,10 +264,6 @@ static int file_map(PAL_HANDLE handle, void** addr, pal_prot_flags_t prot, uint6 return -PAL_ERROR_INVAL; } - struct protected_file* pf = find_protected_file_handle(handle); - if (pf) - return pf_file_map(pf, handle, addr, prot, offset, size); - sgx_chunk_hash_t* chunk_hashes = handle->file.chunk_hashes; void* mem = *addr; @@ -613,24 +362,8 @@ static int file_map(PAL_HANDLE handle, void** addr, pal_prot_flags_t prot, uint6 return ret; } -static int64_t pf_file_setlength(struct protected_file* pf, PAL_HANDLE handle, uint64_t length) { - int fd = handle->file.fd; - - pf_status_t pfs = pf_set_size(pf->context, length); - if (PF_FAILURE(pfs)) { - log_warning("pf_file_setlength(PF fd %d, %lu): pf_set_size returned %s", fd, length, - pf_strerror(pfs)); - return -PAL_ERROR_DENIED; - } - return length; -} - /* 'setlength' operation for file stream. */ static int64_t file_setlength(PAL_HANDLE handle, uint64_t length) { - struct protected_file* pf = find_protected_file_handle(handle); - if (pf) - return pf_file_setlength(pf, handle, length); - int ret = ocall_ftruncate(handle->file.fd, length); if (ret < 0) return unix_to_pal_error(ret); @@ -642,21 +375,7 @@ static int64_t file_setlength(PAL_HANDLE handle, uint64_t length) { /* 'flush' operation for file stream. */ static int file_flush(PAL_HANDLE handle) { int fd = handle->file.fd; - struct protected_file* pf = find_protected_file_handle(handle); - if (pf) { - int ret = flush_pf_maps(pf, /*buffer=*/NULL, /*remove=*/false); - if (ret < 0) { - log_warning("file_flush(PF fd %d): flush_pf_maps returned %s", fd, pal_strerror(ret)); - return ret; - } - pf_status_t pfs = pf_flush(pf->context); - if (PF_FAILURE(pfs)) { - log_warning("file_flush(PF fd %d): pf_flush returned %s", fd, pf_strerror(pfs)); - return -PAL_ERROR_DENIED; - } - } else { - ocall_fsync(fd); - } + ocall_fsync(fd); return 0; } @@ -683,39 +402,6 @@ static inline void file_attrcopy(PAL_STREAM_ATTR* attr, struct stat* stat) { attr->pending_size = stat->st_size; } -static int pf_file_attrquery(struct protected_file* pf, int fd_from_attrquery, const char* path, - uint64_t real_size, PAL_STREAM_ATTR* attr) { - pf = load_protected_file(path, &fd_from_attrquery, real_size, PF_FILE_MODE_READ, - /*create=*/false, pf); - if (!pf) { - log_warning("pf_file_attrquery: load_protected_file(%s, %d) failed", path, - fd_from_attrquery); - /* The call above will fail for PFs that were tampered with or have a wrong path. - * glibc kills the process if this fails during directory enumeration, but that - * should be fine given the scenario. - */ - return -PAL_ERROR_DENIED; - } - - uint64_t size; - pf_status_t pfs = pf_get_size(pf->context, &size); - __UNUSED(pfs); - assert(PF_SUCCESS(pfs)); - attr->pending_size = size; - - pf_handle_t pf_handle; - pfs = pf_get_handle(pf->context, &pf_handle); - assert(PF_SUCCESS(pfs)); - - if (fd_from_attrquery == *(int*)pf_handle) { /* this is a PF opened just for us, close it */ - pfs = pf_close(pf->context); - pf->context = NULL; - assert(PF_SUCCESS(pfs)); - } - - return 0; -} - /* 'attrquery' operation for file streams */ static int file_attrquery(const char* type, const char* uri, PAL_STREAM_ATTR* attr) { if (strcmp(type, URI_TYPE_FILE) && strcmp(type, URI_TYPE_DIR)) @@ -751,27 +437,7 @@ static int file_attrquery(const char* type, const char* uri, PAL_STREAM_ATTR* at goto out; } - /* For protected files return the data size, not real FS size */ - struct protected_file* pf = get_protected_file(path); - if (pf && attr->handle_type != PAL_TYPE_DIR) { - /* protected files should be regular files */ - if (S_ISFIFO(stat_buf.st_mode)) { - ret = -PAL_ERROR_DENIED; - goto out; - } - - /* reset O_NONBLOCK because pf_file_attrquery() may issue reads which don't expect - * non-blocking mode */ - ret = ocall_fsetnonblock(fd, 0); - if (ret < 0) { - ret = unix_to_pal_error(ret); - goto out; - } - - ret = pf_file_attrquery(pf, fd, path, stat_buf.st_size, attr); - } else { - ret = 0; - } + ret = 0; out: free(path); @@ -790,21 +456,6 @@ static int file_attrquerybyhdl(PAL_HANDLE handle, PAL_STREAM_ATTR* attr) { file_attrcopy(attr, &stat_buf); - if (attr->handle_type != PAL_TYPE_DIR) { - /* For protected files return the data size, not real FS size */ - struct protected_file* pf = find_protected_file_handle(handle); - if (pf) { - /* protected files should be regular files (seekable) */ - if (!handle->file.seekable) - return -PAL_ERROR_DENIED; - - uint64_t size; - pf_status_t pfs = pf_get_size(pf->context, &size); - __UNUSED(pfs); - assert(PF_SUCCESS(pfs)); - attr->pending_size = size; - } - } return 0; } @@ -821,13 +472,6 @@ static int file_rename(PAL_HANDLE handle, const char* type, const char* uri) { if (strcmp(type, URI_TYPE_FILE)) return -PAL_ERROR_INVAL; - struct protected_file* pf = get_protected_file(handle->file.realpath); - - if (pf) { - log_error("Renaming of protected files is not supported yet."); - return -PAL_ERROR_NOTSUPPORT; - } - char* tmp = strdup(uri); if (!tmp) return -PAL_ERROR_NOMEM; diff --git a/Pal/src/host/Linux-SGX/db_main.c b/Pal/src/host/Linux-SGX/db_main.c index 30be354ce4..27a2679618 100644 --- a/Pal/src/host/Linux-SGX/db_main.c +++ b/Pal/src/host/Linux-SGX/db_main.c @@ -14,7 +14,6 @@ #include #include "api.h" -#include "enclave_pf.h" #include "enclave_tf.h" #include "init.h" #include "pal.h" @@ -267,6 +266,9 @@ static int print_warnings_on_insecure_configs(PAL_HANDLE parent_process) { return 0; } + /* TODO: `sgx.insecure__protected_files_key` is deprecated in v1.2, remove two versions + * later. */ + bool verbose_log_level = false; bool sgx_debug = false; bool use_cmdline_argv = false; @@ -595,7 +597,6 @@ noreturn void pal_linux_main(char* uptr_libpal_uri, size_t libpal_uri_len, char* } g_pal_internal_mem_size = extra_mem_size + PAL_INITIAL_MEM_SIZE; - /* seal-key material initialization must come before protected-files initialization */ if ((ret = init_seal_key_material()) < 0) { log_error("Failed to initialize SGX sealing key material: %d", ret); ocall_exit(1, /*is_exitgroup=*/true); @@ -616,11 +617,6 @@ noreturn void pal_linux_main(char* uptr_libpal_uri, size_t libpal_uri_len, char* ocall_exit(1, /*is_exitgroup=*/true); } - if ((ret = init_protected_files()) < 0) { - log_error("Failed to initialize protected files: %d", ret); - ocall_exit(1, /*is_exitgroup=*/true); - } - /* this should be placed *after all* initialize-from-manifest routines */ if ((ret = print_warnings_on_insecure_configs(parent)) < 0) { log_error("Cannot parse the manifest (while checking for insecure configurations)"); diff --git a/Pal/src/host/Linux-SGX/db_misc.c b/Pal/src/host/Linux-SGX/db_misc.c index 294d3c321f..c542d789dd 100644 --- a/Pal/src/host/Linux-SGX/db_misc.c +++ b/Pal/src/host/Linux-SGX/db_misc.c @@ -10,7 +10,6 @@ #include "api.h" #include "cpu.h" -#include "enclave_pf.h" #include "hex.h" #include "pal.h" #include "pal_error.h" @@ -622,10 +621,6 @@ int _DkAttestationQuote(const void* user_report_data, PAL_NUM user_report_data_s return 0; } -int _DkSetProtectedFilesKey(const char* pf_key_hex) { - return set_protected_files_key(pf_key_hex); -} - int _DkGetSpecialKey(const char* name, void* key, size_t* key_size) { sgx_key_128bit_t sgx_key; diff --git a/Pal/src/host/Linux-SGX/db_process.c b/Pal/src/host/Linux-SGX/db_process.c index d5ade733a7..0793e9dc53 100644 --- a/Pal/src/host/Linux-SGX/db_process.c +++ b/Pal/src/host/Linux-SGX/db_process.c @@ -16,7 +16,6 @@ #include "pal_internal.h" #include "pal_linux.h" #include "pal_linux_error.h" -#include "protected_files.h" /* * For SGX, the creation of a child process requires a clean enclave and a secure channel between @@ -172,24 +171,6 @@ int _DkProcessCreate(PAL_HANDLE* handle, const char** args) { if (ret != sizeof(g_master_key)) goto failed; - /* securely send the wrap key for protected files to child (only if there is one) */ - char pf_wrap_key_set_char[1]; - pf_wrap_key_set_char[0] = g_pf_wrap_key_set ? '1' : '0'; - - ret = _DkStreamSecureWrite(child->process.ssl_ctx, (uint8_t*)&pf_wrap_key_set_char, - sizeof(pf_wrap_key_set_char), - /*is_blocking=*/!child->process.nonblocking); - if (ret != sizeof(pf_wrap_key_set_char)) - goto failed; - - if (g_pf_wrap_key_set) { - ret = _DkStreamSecureWrite(child->process.ssl_ctx, (uint8_t*)&g_pf_wrap_key, - sizeof(g_pf_wrap_key), - /*is_blocking=*/!child->process.nonblocking); - if (ret != sizeof(g_pf_wrap_key)) - goto failed; - } - /* Send this Gramine instance ID. */ uint64_t instance_id = g_pal_common_state.instance_id; ret = _DkStreamSecureWrite(child->process.ssl_ctx, (uint8_t*)&instance_id, sizeof(instance_id), @@ -245,26 +226,6 @@ int init_child_process(int parent_stream_fd, PAL_HANDLE* out_parent_handle, if (ret != sizeof(g_master_key)) goto out_error; - /* securely receive the wrap key for protected files from parent (only if there is one) */ - char pf_wrap_key_set_char[1] = {0}; - ret = _DkStreamSecureRead(parent->process.ssl_ctx, (uint8_t*)&pf_wrap_key_set_char, - sizeof(pf_wrap_key_set_char), - /*is_blocking=*/!parent->process.nonblocking); - if (ret != sizeof(pf_wrap_key_set_char)) - goto out_error; - - if (pf_wrap_key_set_char[0] == '1') { - ret = _DkStreamSecureRead(parent->process.ssl_ctx, (uint8_t*)&g_pf_wrap_key, - sizeof(g_pf_wrap_key), - /*is_blocking=*/!parent->process.nonblocking); - if (ret != sizeof(g_pf_wrap_key)) { - g_pf_wrap_key_set = false; - goto out_error; - } - - g_pf_wrap_key_set = true; - } - uint64_t instance_id; ret = _DkStreamSecureRead(parent->process.ssl_ctx, (uint8_t*)&instance_id, sizeof(instance_id), /*is_blocking=*/!parent->process.nonblocking); diff --git a/Pal/src/host/Linux-SGX/db_streams.c b/Pal/src/host/Linux-SGX/db_streams.c index 411d59d522..27c306f6dc 100644 --- a/Pal/src/host/Linux-SGX/db_streams.c +++ b/Pal/src/host/Linux-SGX/db_streams.c @@ -17,7 +17,6 @@ #include "api.h" #include "crypto.h" #include "enclave_pages.h" -#include "enclave_pf.h" #include "pal.h" #include "pal_error.h" #include "pal_internal.h" @@ -49,10 +48,6 @@ static size_t addr_size(const struct sockaddr* addr) { /* _DkStreamUnmap for internal use. Unmap stream at certain memory address. The memory is unmapped as a whole.*/ int _DkStreamUnmap(void* addr, uint64_t size) { - int ret = flush_pf_maps(/*pf=*/NULL, addr, /*remove=*/true); - if (ret < 0) - return ret; - return free_enclave_pages(addr, size); } diff --git a/Pal/src/host/Linux-SGX/enclave_pf.c b/Pal/src/host/Linux-SGX/enclave_pf.c deleted file mode 100644 index f71037ca15..0000000000 --- a/Pal/src/host/Linux-SGX/enclave_pf.c +++ /dev/null @@ -1,811 +0,0 @@ -/* SPDX-License-Identifier: LGPL-3.0-or-later */ -/* Copyright (C) 2020 Invisible Things Lab - * Rafal Wojdyla - */ - -#include "crypto.h" -#include "enclave_pf.h" -#include "hex.h" -#include "pal_internal.h" -#include "pal_linux.h" -#include "pal_linux_error.h" -#include "spinlock.h" -#include "toml.h" -#include "toml_utils.h" - -/* SGX-specific keys for protected files, used for SGX sealing. The former key is bound to the - * MRENCLAVE measurement of the SGX enclave (only the same enclave can unseal secrets). The latter - * key is bound to the MRSIGNER measurement (all enclaves from the same signer can unseal secrets). - * We don't use synchronization on them since they are only set during initialization where Gramine - * runs single-threaded. */ -pf_key_t g_pf_mrenclave_key = {0}; -pf_key_t g_pf_mrsigner_key = {0}; - -/* Wrap key for protected files, either hard-coded in manifest, provisioned during attestation, or - * inherited from the parent process. We don't use synchronization on them since they are only set - * during initialization where Gramine runs single-threaded. */ -pf_key_t g_pf_wrap_key = {0}; -bool g_pf_wrap_key_set = false; - -/* - * At startup, protected file paths are read from the manifest and the specified files - * or directories registered. For supported I/O operations, handlers (in db_files.c) - * check if the file is a PF to perform the required operation transparently. - * - * Since PF's "logical" size is different than the real FS size (and to avoid potential - * infinite recursion in FS handlers) we don't use PAL file APIs here, but raw OCALLs. - */ - -/* List of map buffers */ -LISTP_TYPE(pf_map) g_pf_map_list = LISTP_INIT; - -/* Callbacks for protected files handling */ -static pf_status_t cb_read(pf_handle_t handle, void* buffer, uint64_t offset, size_t size) { - int fd = *(int*)handle; - size_t buffer_offset = 0; - size_t to_read = size; - while (to_read > 0) { - ssize_t read = ocall_pread(fd, buffer + buffer_offset, to_read, offset + buffer_offset); - if (read == -EINTR) - continue; - - if (read < 0) { - log_warning("cb_read(%d, %p, %lu, %lu): read failed: %ld", fd, buffer, offset, - size, read); - return PF_STATUS_CALLBACK_FAILED; - } - - /* EOF is an error condition, we want to read exactly `size` bytes */ - if (read == 0) { - log_warning("cb_read(%d, %p, %lu, %lu): EOF", fd, buffer, offset, size); - return PF_STATUS_CALLBACK_FAILED; - } - - to_read -= read; - buffer_offset += read; - } - return PF_STATUS_SUCCESS; -} - -static pf_status_t cb_write(pf_handle_t handle, const void* buffer, uint64_t offset, size_t size) { - int fd = *(int*)handle; - size_t buffer_offset = 0; - size_t to_write = size; - while (to_write > 0) { - ssize_t written = ocall_pwrite(fd, buffer + buffer_offset, to_write, - offset + buffer_offset); - if (written == -EINTR) - continue; - - if (written < 0) { - log_warning("cb_write(%d, %p, %lu, %lu): write failed: %ld", fd, buffer, offset, - size, written); - return PF_STATUS_CALLBACK_FAILED; - } - - /* EOF is an error condition, we want to write exactly `size` bytes */ - if (written == 0) { - log_warning("cb_write(%d, %p, %lu, %lu): EOF", fd, buffer, offset, size); - return PF_STATUS_CALLBACK_FAILED; - } - - to_write -= written; - buffer_offset += written; - } - return PF_STATUS_SUCCESS; -} - -static pf_status_t cb_truncate(pf_handle_t handle, uint64_t size) { - int fd = *(int*)handle; - int ret = ocall_ftruncate(fd, size); - if (ret < 0) { - log_warning("cb_truncate(%d, %lu): ocall failed: %d", fd, size, ret); - return PF_STATUS_CALLBACK_FAILED; - } - return PF_STATUS_SUCCESS; -} - -#ifdef DEBUG -static void cb_debug(const char* msg) { - log_debug("%s", msg); -} -#endif - -static pf_status_t cb_aes_cmac(const pf_key_t* key, const void* input, size_t input_size, - pf_mac_t* mac) { - int ret = lib_AESCMAC((const uint8_t*)key, sizeof(*key), input, input_size, (uint8_t*)mac, - sizeof(*mac)); - if (ret != 0) { - log_warning("lib_AESCMAC failed: %d", ret); - return PF_STATUS_CALLBACK_FAILED; - } - return PF_STATUS_SUCCESS; -} - -static pf_status_t cb_aes_gcm_encrypt(const pf_key_t* key, const pf_iv_t* iv, const void* aad, - size_t aad_size, const void* input, size_t input_size, - void* output, pf_mac_t* mac) { - int ret = lib_AESGCMEncrypt((const uint8_t*)key, sizeof(*key), (const uint8_t*)iv, input, - input_size, aad, aad_size, output, (uint8_t*)mac, sizeof(*mac)); - if (ret != 0) { - log_warning("lib_AESGCMEncrypt failed: %d", ret); - return PF_STATUS_CALLBACK_FAILED; - } - return PF_STATUS_SUCCESS; -} - -static pf_status_t cb_aes_gcm_decrypt(const pf_key_t* key, const pf_iv_t* iv, const void* aad, - size_t aad_size, const void* input, size_t input_size, - void* output, const pf_mac_t* mac) { - int ret = lib_AESGCMDecrypt((const uint8_t*)key, sizeof(*key), (const uint8_t*)iv, input, - input_size, aad, aad_size, output, (const uint8_t*)mac, - sizeof(*mac)); - if (ret != 0) { - log_warning("lib_AESGCMDecrypt failed: %d", ret); - return PF_STATUS_CALLBACK_FAILED; - } - return PF_STATUS_SUCCESS; -} - -static pf_status_t cb_random(uint8_t* buffer, size_t size) { - int ret = _DkRandomBitsRead(buffer, size); - if (ret < 0) { - log_warning("_DkRandomBitsRead failed: %d", ret); - return PF_STATUS_CALLBACK_FAILED; - } - return PF_STATUS_SUCCESS; -} - -/* Collection of registered protected files */ -static struct protected_file* g_protected_files = NULL; - -/* Collection of registered protected directories */ -static struct protected_file* g_protected_dirs = NULL; - -/* Lock for operations on global PF structures */ -static spinlock_t g_protected_file_lock = INIT_SPINLOCK_UNLOCKED; - -/* Take ownership of the global PF lock */ -void pf_lock(void) { - spinlock_lock(&g_protected_file_lock); -} - -/* Release ownership of the global PF lock */ -void pf_unlock(void) { - spinlock_unlock(&g_protected_file_lock); -} - -/* Exact match of path in g_protected_files */ -struct protected_file* find_protected_file(const char* path) { - struct protected_file* pf = NULL; - - pf_lock(); - HASH_FIND_STR(g_protected_files, path, pf); - pf_unlock(); - return pf; -} - -/* Find registered pf directory starting with the given path */ -static struct protected_file* find_protected_dir(const char* path) { - struct protected_file* pf = NULL; - struct protected_file* tmp = NULL; - size_t len = strlen(path); - - pf_lock(); - // TODO: avoid linear lookup - for (tmp = g_protected_dirs; tmp != NULL; tmp = tmp->hh.next) { - if (tmp->path_len < len && !memcmp(tmp->path, path, tmp->path_len) && - (!path[tmp->path_len] || path[tmp->path_len] == '/')) { - pf = tmp; - break; - } - } - - pf_unlock(); - return pf; -} - -/* Find PF by handle */ -struct protected_file* find_protected_file_handle(PAL_HANDLE handle) { - char* uri = malloc(URI_MAX); - if (!uri) { - return NULL; - } - - struct protected_file* ret = NULL; - - /* TODO: this logic is inefficient, add a PF reference to PAL_HANDLE instead */ - int uri_len = _DkStreamGetName(handle, uri, URI_MAX); - if (uri_len < 0) { - goto out; - } - - /* uri is prefixed by "file:", we need path */ - assert(strstartswith(uri, URI_PREFIX_FILE)); - ret = find_protected_file(uri + URI_PREFIX_FILE_LEN); - -out: - free(uri); - return ret; -} - -static int register_protected_path(const char* path, enum pf_key_type key_type, - struct protected_file** new_pf); - -/* Return a registered PF that matches specified path - (or the path that is contained in a registered PF directory) */ -struct protected_file* get_protected_file(const char* path) { - struct protected_file* pf = find_protected_file(path); - if (pf) - goto out; - - pf = find_protected_dir(path); - if (pf) { - /* path not registered but matches registered dir */ - log_debug("get_pf: registering new PF '%s' in dir '%s'", path, pf->path); - int ret = register_protected_path(path, pf->key_type, &pf); - __UNUSED(ret); - assert(ret == 0); - /* return newly registered PF */ - } - -out: - return pf; -} - -#define S_ISDIR(m) ((m & 0170000) == 0040000) - -static int is_directory(const char* path, bool* is_dir) { - int fd = -1; - struct stat st; - - *is_dir = false; - int ret = ocall_open(path, O_RDONLY, 0); - if (ret < 0) { - /* this can be called on a path without the file existing, assume non-dir for now */ - ret = 0; - goto out; - } - - fd = ret; - ret = ocall_fstat(fd, &st); - if (ret < 0) { - log_warning("is_directory(%s): fstat failed: %d", path, ret); - goto out; - } - - if (S_ISDIR(st.st_mode)) - *is_dir = true; - -out: - if (fd >= 0) { - int rv = ocall_close(fd); - if (rv < 0) { - log_warning("is_directory(%s): close failed: %d", path, rv); - } - } - - return ret < 0 ? unix_to_pal_error(ret) : ret; -} - -/* Register all files from the given directory recursively */ -static int register_protected_dir(const char* path, enum pf_key_type key_type) { - int fd = -1; - int ret = -PAL_ERROR_NOMEM; - size_t bufsize = 1024; - void* buf = malloc(bufsize); - - if (!buf) - return -PAL_ERROR_NOMEM; - - ret = ocall_open(path, O_RDONLY | O_DIRECTORY, 0); - if (ret < 0) { - log_warning("register_protected_dir: opening %s failed: %d", path, ret); - ret = unix_to_pal_error(ret); - goto out; - } - fd = ret; - - size_t path_size = strlen(path) + 1; - int returned; - do { - returned = ocall_getdents(fd, buf, bufsize); - if (returned < 0) { - ret = unix_to_pal_error(returned); - log_warning("register_protected_dir: reading %s failed: %d", path, ret); - goto out; - } - - int pos = 0; - struct linux_dirent64* dir; - - while (pos < returned) { - dir = (struct linux_dirent64*)((char*)buf + pos); - - if (!strcmp(dir->d_name, ".") || !strcmp(dir->d_name, "..")) - goto next; - - /* register file */ - size_t sub_path_size = URI_PREFIX_FILE_LEN + path_size + 1 + strlen(dir->d_name); - char* sub_path = malloc(sub_path_size); - ret = -PAL_ERROR_NOMEM; - if (!sub_path) - goto out; - - snprintf(sub_path, sub_path_size, URI_PREFIX_FILE "%s/%s", path, dir->d_name); - ret = register_protected_path(sub_path, key_type, NULL); - if (ret != 0) { - free(sub_path); - goto out; - } - free(sub_path); - next: - pos += dir->d_reclen; - } - } while (returned != 0); - - ret = 0; -out: - if (fd >= 0) - ocall_close(fd); - free(buf); - return ret; -} - -/* Register a single PF (if it's a directory, recursively) */ -static int register_protected_path(const char* path, enum pf_key_type key_type, - struct protected_file** new_pf) { - int ret = -PAL_ERROR_NOMEM; - struct protected_file* new = NULL; - - size_t normpath_size = strlen(path) + 1; - char* normpath = malloc(normpath_size); - if (!normpath) - goto out; - - ret = get_norm_path(path, normpath, &normpath_size); - if (ret < 0) { - log_warning("Couldn't normalize path (%s): %s", path, pal_strerror(ret)); - goto out; - } - - /* discard the "file:" prefix */ - if (strstartswith(normpath, URI_PREFIX_FILE)) - path = normpath + URI_PREFIX_FILE_LEN; - else - path = normpath; - - if (find_protected_file(path)) { - ret = 0; - log_debug("register_protected_path: file %s already registered", path); - goto out; - } - - new = calloc(1, sizeof(*new)); - if (!new) { - ret = -PAL_ERROR_NOMEM; - goto out; - } - - new->key_type = key_type; - - new->path_len = strlen(path); - /* This is never freed but so isn't the whole struct, PFs persist for the whole lifetime - of the process. */ - new->path = malloc(new->path_len + 1); - if (!new->path) { - ret = -PAL_ERROR_NOMEM; - goto out; - } - - memcpy(new->path, path, new->path_len + 1); - new->refcount = 0; - new->writable_fd = -1; - - bool is_dir; - ret = is_directory(path, &is_dir); - if (ret < 0) - goto out; - - log_debug("register_protected_path: [%s] %s = %p", is_dir ? "dir" : "file", path, new); - - if (is_dir) - register_protected_dir(path, key_type); - - pf_lock(); - - if (is_dir) { - HASH_ADD_STR(g_protected_dirs, path, new); - } else { - HASH_ADD_STR(g_protected_files, path, new); - } - - pf_unlock(); - - if (new_pf) - *new_pf = new; - - ret = 0; -out: - free(normpath); - if (ret < 0) { - if (new) - free(new->path); - free(new); - } - return ret; -} - -static const char* toml_table_name_from_key_type(enum pf_key_type key_type) { - assert(key_type == PROTECTED_FILE_KEY_WRAP || key_type == PROTECTED_FILE_KEY_MRENCLAVE || - key_type == PROTECTED_FILE_KEY_MRSIGNER); - switch (key_type) { - case PROTECTED_FILE_KEY_WRAP: - return "protected_files"; - case PROTECTED_FILE_KEY_MRENCLAVE: - return "protected_mrenclave_files"; - case PROTECTED_FILE_KEY_MRSIGNER: - return "protected_mrsigner_files"; - } - return NULL; /* unreachable */ -} - -static int register_protected_files_from_toml_table(enum pf_key_type key_type) { - int ret; - toml_table_t* manifest_sgx = toml_table_in(g_pal_public_state.manifest_root, "sgx"); - if (!manifest_sgx) - return 0; - - const char* table_name = toml_table_name_from_key_type(key_type); - - toml_table_t* toml_pfs = toml_table_in(manifest_sgx, table_name); - if (!toml_pfs) - return 0; - - ssize_t toml_pfs_cnt = toml_table_nkval(toml_pfs); - if (toml_pfs_cnt < 0) - return -PAL_ERROR_DENIED; - if (toml_pfs_cnt == 0) - return 0; - - char* toml_pf_str = NULL; - - for (ssize_t i = 0; i < toml_pfs_cnt; i++) { - const char* toml_pf_key = toml_key_in(toml_pfs, i); - assert(toml_pf_key); - - toml_raw_t toml_pf_str_raw = toml_raw_in(toml_pfs, toml_pf_key); - if (!toml_pf_str_raw) { - log_error("Invalid protected file in manifest: '%s'", toml_pf_key); - ret = -PAL_ERROR_INVAL; - goto out; - } - - ret = toml_rtos(toml_pf_str_raw, &toml_pf_str); - if (ret < 0) { - log_error("Invalid protected file in manifest: '%s' (not a string)", toml_pf_key); - ret = -PAL_ERROR_INVAL; - goto out; - } - - if (!strstartswith(toml_pf_str, URI_PREFIX_FILE)) { - log_error("Invalid URI [%s]: Protected files must start with 'file:'", toml_pf_str); - ret = -PAL_ERROR_INVAL; - goto out; - } - - ret = register_protected_path(toml_pf_str, key_type, NULL); - if (ret < 0) - goto out; - - free(toml_pf_str); - toml_pf_str = NULL; - } - - log_error("Detected deprecated syntax. Consider switching to new syntax: 'sgx.protected_files " - "= [\"file1\", ..]'."); - - ret = 0; -out: - free(toml_pf_str); - return ret; -} - -static int register_protected_files_from_toml_array(enum pf_key_type key_type) { - int ret; - toml_table_t* manifest_sgx = toml_table_in(g_pal_public_state.manifest_root, "sgx"); - if (!manifest_sgx) - return 0; - - const char* table_name = toml_table_name_from_key_type(key_type); - - toml_array_t* toml_pfs = toml_array_in(manifest_sgx, table_name); - if (!toml_pfs) - return 0; - - ssize_t toml_pfs_cnt = toml_array_nelem(toml_pfs); - if (toml_pfs_cnt < 0) - return -PAL_ERROR_DENIED; - if (toml_pfs_cnt == 0) - return 0; - - char* toml_pf_str = NULL; - - for (ssize_t i = 0; i < toml_pfs_cnt; i++) { - toml_raw_t toml_pf_str_raw = toml_raw_at(toml_pfs, i); - if (!toml_pf_str_raw) { - log_error("Invalid protected file in manifest at index %ld", i); - ret = -PAL_ERROR_INVAL; - goto out; - } - - ret = toml_rtos(toml_pf_str_raw, &toml_pf_str); - if (ret < 0) { - log_error("Invalid protected file in manifest at index %ld (not a string)", i); - ret = -PAL_ERROR_INVAL; - goto out; - } - - if (!strstartswith(toml_pf_str, URI_PREFIX_FILE)) { - log_error("Invalid URI [%s]: Protected files must start with 'file:'", toml_pf_str); - ret = -PAL_ERROR_INVAL; - goto out; - } - - ret = register_protected_path(toml_pf_str, key_type, NULL); - if (ret < 0) - goto out; - - free(toml_pf_str); - toml_pf_str = NULL; - } - - ret = 0; -out: - free(toml_pf_str); - return ret; -} - -static int register_protected_files(enum pf_key_type key_type) { - int ret; - - /* first try legacy manifest syntax with TOML tables, i.e. `sgx.protected_files.key = "file"` */ - ret = register_protected_files_from_toml_table(key_type); - if (ret < 0) { - log_error("Reading protected files (in TOML-table syntax) from the manifest failed: %s", - pal_strerror(ret)); - return ret; - } - - /* use new manifest syntax with TOML arrays, i.e. `sgx.protected_files = ["file1", ..]` */ - ret = register_protected_files_from_toml_array(key_type); - if (ret < 0) { - log_error("Reading protected files (in TOML-array syntax) from the manifest failed: %s", - pal_strerror(ret)); - return ret; - } - - return 0; -} - - -/* Initialize the PF library, register PFs from the manifest */ -int init_protected_files(void) { - int ret; - pf_debug_f debug_callback = NULL; - -#ifdef DEBUG - debug_callback = cb_debug; -#endif - - pf_set_callbacks(cb_read, cb_write, cb_truncate, cb_aes_cmac, cb_aes_gcm_encrypt, - cb_aes_gcm_decrypt, cb_random, debug_callback); - - ret = sgx_get_seal_key(SGX_KEYPOLICY_MRENCLAVE, &g_pf_mrenclave_key); - if (ret < 0) { - log_error("Cannot obtain MRENCLAVE-bound protected files key"); - return ret; - } - - ret = sgx_get_seal_key(SGX_KEYPOLICY_MRSIGNER, &g_pf_mrsigner_key); - if (ret < 0) { - log_error("Cannot obtain MRSIGNER-bound protected files key"); - return ret; - } - - /* if wrap key is not hard-coded in the manifest, assume that it was received from parent or - * it will be provisioned after local/remote attestation; otherwise read it from manifest */ - char* protected_files_key_str = NULL; - ret = toml_string_in(g_pal_public_state.manifest_root, "sgx.insecure__protected_files_key", - &protected_files_key_str); - if (ret < 0) { - log_error("Cannot parse 'sgx.insecure__protected_files_key'"); - return -PAL_ERROR_INVAL; - } - - if (protected_files_key_str) { - if (strlen(protected_files_key_str) != PF_KEY_SIZE * 2) { - log_error("Malformed 'sgx.insecure__protected_files_key' value in the manifest"); - free(protected_files_key_str); - return -PAL_ERROR_INVAL; - } - - memset(g_pf_wrap_key, 0, sizeof(g_pf_wrap_key)); - for (size_t i = 0; i < strlen(protected_files_key_str); i++) { - int8_t val = hex2dec(protected_files_key_str[i]); - if (val < 0) { - log_error("Malformed 'sgx.insecure__protected_files_key' value in the manifest"); - free(protected_files_key_str); - return -PAL_ERROR_INVAL; - } - g_pf_wrap_key[i / 2] = g_pf_wrap_key[i / 2] * 16 + (uint8_t)val; - } - - free(protected_files_key_str); - g_pf_wrap_key_set = true; - } - - ret = register_protected_files(PROTECTED_FILE_KEY_WRAP); - if (ret < 0) { - log_error("Malformed protected files found in manifest"); - return ret; - } - - ret = register_protected_files(PROTECTED_FILE_KEY_MRENCLAVE); - if (ret < 0) { - log_error("Malformed MRENCLAVE-bound protected files found in manifest"); - return ret; - } - - ret = register_protected_files(PROTECTED_FILE_KEY_MRSIGNER); - if (ret < 0) { - log_error("Malformed MRSIGNER-bound protected files found in manifest"); - return ret; - } - - return 0; -} - -/* Open/create a PF */ -static int open_protected_file(const char* path, struct protected_file* pf, pf_handle_t handle, - uint64_t size, pf_file_mode_t mode, bool create) { - pf_key_t* pf_key = NULL; - switch (pf->key_type) { - case PROTECTED_FILE_KEY_WRAP: - if (!g_pf_wrap_key_set) { - log_error("pf_open failed: wrap key was not provided"); - return -PAL_ERROR_DENIED; - } - pf_key = &g_pf_wrap_key; - break; - case PROTECTED_FILE_KEY_MRENCLAVE: - pf_key = &g_pf_mrenclave_key; - break; - case PROTECTED_FILE_KEY_MRSIGNER: - pf_key = &g_pf_mrsigner_key; - break; - default: - log_error("Invalid key type when opening a protected file!"); - return -PAL_ERROR_DENIED; - } - - pf_status_t pfs; - pfs = pf_open(handle, path, size, mode, create, pf_key, &pf->context); - if (PF_FAILURE(pfs)) { - log_warning("pf_open(%d, %s) failed: %s", *(int*)handle, path, pf_strerror(pfs)); - return -PAL_ERROR_DENIED; - } - return 0; -} - -/* Prepare a PF for I/O - This function registers the PF if path is in a registered PF directory, then - calls the appropriate PF function to open/create it (if allowed) */ -struct protected_file* load_protected_file(const char* path, int* fd, uint64_t size, - pf_file_mode_t mode, bool create, - struct protected_file* pf) { - log_debug("load_protected_file: %s, fd %d, size %lu, mode %d, create %d, pf %p", path, - *fd, size, mode, create, pf); - - if (!pf) - pf = get_protected_file(path); - - if (pf) { - if (!pf->context) { - log_debug("load_protected_file: %s, fd %d: opening new PF %p", path, *fd, pf); - int ret = open_protected_file(path, pf, (pf_handle_t)fd, size, mode, create); - if (ret < 0) - return NULL; - } else { - log_debug("load_protected_file: %s, fd %d: returning old PF %p", path, *fd, pf); - } - } - - return pf; -} - -/* Flush PF map buffers and optionally remove and free them. - * If pf is NULL, process all maps containing given buffer. - * If buffer is NULL, process all maps for given pf. - * If both pf and buffer are NULL, process all maps for all PFs. - */ -int flush_pf_maps(struct protected_file* pf, void* buffer, bool remove) { - struct pf_map* map; - struct pf_map* tmp; - uint64_t pf_size; - pf_status_t pfs; - - pf_lock(); - LISTP_FOR_EACH_ENTRY_SAFE(map, tmp, &g_pf_map_list, list) { - if (pf && map->pf != pf) - continue; - - if (buffer && map->buffer != buffer) - continue; - - size_t map_size = map->size; - struct protected_file* map_pf = pf ? pf : map->pf; - - pfs = pf_get_size(map_pf->context, &pf_size); - assert(PF_SUCCESS(pfs)); - - assert(pf_size >= map->offset); - if (map->offset + map_size > pf_size) - map_size = pf_size - map->offset; - - if (map_size > 0) { - pfs = pf_write(map_pf->context, map->offset, map_size, map->buffer); - if (PF_FAILURE(pfs)) { - log_error("flush_pf_maps: pf_write failed: %s", pf_strerror(pfs)); - pf_unlock(); - return -PAL_ERROR_INVAL; - } - } - - if (remove) { - LISTP_DEL(map, &g_pf_map_list, list); - free(map); - } - } - - pf_unlock(); - return 0; -} - -/* Flush map buffers and unload/close the PF */ -int unload_protected_file(struct protected_file* pf) { - /* flush all pf's maps and delete them */ - int ret = flush_pf_maps(pf, NULL, true); - if (ret < 0) - return ret; - pf_status_t pfs = pf_close(pf->context); - if (PF_FAILURE(pfs)) { - log_warning("unload_protected_file(%p) failed: %s", pf, pf_strerror(pfs)); - } - - pf->context = NULL; - return 0; -} - -int set_protected_files_key(const char* pf_key_hex) { - size_t pf_key_hex_len = strlen(pf_key_hex); - if (pf_key_hex_len != PF_KEY_SIZE * 2) { - return -PAL_ERROR_INVAL; - } - - pf_lock(); - memset(g_pf_wrap_key, 0, sizeof(g_pf_wrap_key)); - for (size_t i = 0; i < pf_key_hex_len; i++) { - int8_t val = hex2dec(pf_key_hex[i]); - if (val < 0) { - memset(g_pf_wrap_key, 0, sizeof(g_pf_wrap_key)); - pf_unlock(); - return -PAL_ERROR_INVAL; - } - g_pf_wrap_key[i / 2] = g_pf_wrap_key[i / 2] * 16 + (uint8_t)val; - } - g_pf_wrap_key_set = true; - pf_unlock(); - - return 0; -} diff --git a/Pal/src/host/Linux-SGX/enclave_pf.h b/Pal/src/host/Linux-SGX/enclave_pf.h deleted file mode 100644 index b5549bca59..0000000000 --- a/Pal/src/host/Linux-SGX/enclave_pf.h +++ /dev/null @@ -1,108 +0,0 @@ -/* SPDX-License-Identifier: LGPL-3.0-or-later */ -/* Copyright (C) 2020 Invisible Things Lab - * Rafal Wojdyla - * Copyright (C) 2021 Intel Corporation - */ - -/* Protected files (PF) are encrypted on disk and transparently decrypted when accessed by Gramine - * or by app running inside Gramine. Internal protected file format was ported from Intel SGX SDK, - * https://github.com/intel/linux-sgx/tree/1eaa4551d4b02677eec505684412dc288e6d6361/sdk/protected_fs - * - * Features: - * - Data is encrypted (confidentiality) and integrity protected (tamper resistance). - * - File swap protection (a PF can only be accessed when in a specific path). - * - Transparency (Gramine app sees PFs as regular files, no need to modify the app). - * - * Limitations: - * - Metadata currently limits PF path size to 512 bytes and filename size to 260 bytes. - * - Truncating protected files is not yet implemented. - * - The recovery file feature is disabled (present in Intel SGX SDK). - */ - -#ifndef ENCLAVE_PF_H_ -#define ENCLAVE_PF_H_ - -#include -#include -#include - -#include "pal.h" -#include "pal_internal.h" -#include "protected_files.h" - -/* Used to track map buffers for protected files */ -DEFINE_LIST(pf_map); -struct pf_map { - LIST_TYPE(pf_map) list; - struct protected_file* pf; - void* buffer; - uint64_t size; - uint64_t offset; /* offset in PF, needed for write buffers when flushing to the PF */ -}; -DEFINE_LISTP(pf_map); - -/* List of PF map buffers; this list is traversed on PF flush (on file close) */ -extern LISTP_TYPE(pf_map) g_pf_map_list; - -enum pf_key_type { - PROTECTED_FILE_KEY_WRAP, - PROTECTED_FILE_KEY_MRENCLAVE, - PROTECTED_FILE_KEY_MRSIGNER, -}; - -/* Data of a protected file */ -struct protected_file { - UT_hash_handle hh; - size_t path_len; - char* path; - pf_context_t* context; /* NULL until PF is opened */ - int64_t refcount; /* used for deciding when to call unload_protected_file() */ - int writable_fd; /* fd of underlying file for writable PF, -1 if no writable handles are open */ - enum pf_key_type key_type; -}; - -/* Take ownership of the global PF lock */ -void pf_lock(void); - -/* Release ownership of the global PF lock */ -void pf_unlock(void); - -/* Set new wrap key for protected files (e.g., provisioned by remote user) */ -int set_protected_files_key(const char* pf_key_hex); - -/* Return a registered PF that matches specified path - (or the path is contained in a registered PF directory) */ -struct protected_file* get_protected_file(const char* path); - -/* Load and initialize a PF (must be called before any I/O operations) - * - * path: normalized host path - * fd: pointer to an opened file descriptor (must point to a valid value for the whole time PF - * is being accessed) - * size: underlying file size (in bytes) - * mode: access mode - * create: if true, the PF is being created/truncated - * pf: (optional) PF pointer if already known - */ -struct protected_file* load_protected_file(const char* path, int* fd, size_t size, - pf_file_mode_t mode, bool create, - struct protected_file* pf); - -/* Flush PF map buffers and optionally remove and free them. - If pf is NULL, process all maps containing given buffer. - If buffer is NULL, process all maps for given pf. */ -int flush_pf_maps(struct protected_file* pf, void* buffer, bool remove); - -/* Flush map buffers and unload/close the PF */ -int unload_protected_file(struct protected_file* pf); - -/* Find registered PF by path (exact match) */ -struct protected_file* find_protected_file(const char* path); - -/* Find protected file by handle (uses handle's path to call find_protected_file) */ -struct protected_file* find_protected_file_handle(PAL_HANDLE handle); - -/* Initialize the PF library, register PFs from the manifest */ -int init_protected_files(void); - -#endif /* ENCLAVE_PF_H_ */ diff --git a/Pal/src/host/Linux-SGX/meson.build b/Pal/src/host/Linux-SGX/meson.build index 6be9e49f81..09f685a635 100644 --- a/Pal/src/host/Linux-SGX/meson.build +++ b/Pal/src/host/Linux-SGX/meson.build @@ -6,7 +6,6 @@ gsgx_h = configure_file( sgx_inc = [ includes_pal_common, - protected_files_inc, include_directories( '.', '../../../include/arch/@0@/Linux'.format(host_machine.cpu_family()), @@ -74,7 +73,6 @@ libpal_sgx = shared_library('pal', 'enclave_framework.c', 'enclave_ocalls.c', 'enclave_pages.c', - 'enclave_pf.c', 'enclave_platform.c', 'enclave_untrusted.c', 'enclave_xstate.c', @@ -119,7 +117,6 @@ libpal_sgx = shared_library('pal', dependencies: [ common_dep, cryptoadapter_dep, - protected_files_dep, ], install: true, diff --git a/Pal/src/host/Linux-SGX/tools/common/meson.build b/Pal/src/host/Linux-SGX/tools/common/meson.build index 7f373f8c6b..c69fcc6b1c 100644 --- a/Pal/src/host/Linux-SGX/tools/common/meson.build +++ b/Pal/src/host/Linux-SGX/tools/common/meson.build @@ -32,6 +32,7 @@ sgx_util_dep = declare_dependency( include_directories: [ include_directories('.'), sgx_inc, # this is mostly for sgx_arch.h + protected_files_inc, ], ) diff --git a/Pal/src/host/Linux/db_misc.c b/Pal/src/host/Linux/db_misc.c index 2191b72567..04e4dfc7fa 100644 --- a/Pal/src/host/Linux/db_misc.c +++ b/Pal/src/host/Linux/db_misc.c @@ -73,11 +73,6 @@ int _DkAttestationQuote(const void* user_report_data, PAL_NUM user_report_data_s return -PAL_ERROR_NOTIMPLEMENTED; } -int _DkSetProtectedFilesKey(const char* pf_key_hex) { - __UNUSED(pf_key_hex); - return -PAL_ERROR_NOTIMPLEMENTED; -} - int _DkGetSpecialKey(const char* name, void* key, size_t* key_size) { __UNUSED(name); __UNUSED(key); diff --git a/Pal/src/host/Skeleton/db_misc.c b/Pal/src/host/Skeleton/db_misc.c index 507fd08f1a..0df87a6c76 100644 --- a/Pal/src/host/Skeleton/db_misc.c +++ b/Pal/src/host/Skeleton/db_misc.c @@ -52,11 +52,6 @@ int _DkAttestationQuote(const void* user_report_data, PAL_NUM user_report_data_s return -PAL_ERROR_NOTIMPLEMENTED; } -int _DkSetProtectedFilesKey(const char* pf_key_hex) { - __UNUSED(pf_key_hex); - return -PAL_ERROR_NOTIMPLEMENTED; -} - int _DkGetSpecialKey(const char* name, void* key, size_t* key_size) { __UNUSED(name); __UNUSED(key); diff --git a/Pal/src/pal-symbols b/Pal/src/pal-symbols index c312c8aad3..f3f64be4ce 100644 --- a/Pal/src/pal-symbols +++ b/Pal/src/pal-symbols @@ -43,7 +43,6 @@ DkDebugMapRemove DkDebugDescribeLocation DkAttestationReport DkAttestationQuote -DkSetProtectedFilesKey DkGetSpecialKey DkDebugLog DkGetPalPublicState diff --git a/common/include/hex.h b/common/include/hex.h index 0ca97e3a37..7ec3d52d45 100644 --- a/common/include/hex.h +++ b/common/include/hex.h @@ -19,7 +19,7 @@ * str is the caller-provided buffer, len is the length of the buffer. * The len must be at least (size * 2)+1. */ -static inline char* __bytes2hexstr(void* hex, size_t size, char* str, size_t len) { +static inline char* __bytes2hexstr(const void* hex, size_t size, char* str, size_t len) { static const char* ch = "0123456789abcdef"; __UNUSED(len); assert(len >= size * 2 + 1); diff --git a/common/src/protected_files/README.rst b/common/src/protected_files/README.rst index 0e19722624..ea0ff49254 100644 --- a/common/src/protected_files/README.rst +++ b/common/src/protected_files/README.rst @@ -1,44 +1,58 @@ -=============== -Protected Files -=============== +=========================== +Protected (Encrypted) Files +=========================== -Protected files (PF) are a type of files that can be specified in the manifest (SGX only). They are -encrypted on disk and transparently decrypted when accessed by Gramine or by application running -inside Gramine. +This directory contains the implementation of Protected Files (PF), a library +used for implementing *encrypted files* in Gramine. These files are encrypted on +disk and transparently decrypted when accessed by Gramine or by application +running inside Gramine. + +Originally, the whole feature was called *protected files*, and was implemented +for SGX only. After moving it to LibOS, we updated the name: + +* *encrypted files* is the name of the feature in Gramine, +* ``protected_files`` is the name of the platform-independent library used by + that feature (i.e. this directory). Features ======== -- Data is encrypted (confidentiality) and integrity protected (tamper resistance). -- File swap protection (a PF can only be accessed when in a specific path). -- Transparency (Gramine application sees PFs as regular files, no need to modify the application). +- Data is encrypted (confidentiality) and integrity protected (tamper + resistance). +- File swap protection (an encrypted file can only be accessed when in a + specific path). +- Transparency (the application sees encrypted files as regular files, no need + to modify the application). Example ------- :: - sgx.protected_files = [ - "file:tmp/some_file", - "file:tmp/some_dir", - "file:tmp/another_dir/some_file", + fs.mounts = [ + ... + { type = "encrypted", path = "/some_file", uri = "file:tmp/some_file" }, + { type = "encrypted", path = "/some_dir", uri = "file:tmp/some_dir" }, + { type = "encrypted", path = "/another_file", uri = "file:another_dir/some_file" }, ] -Paths specifying PF entries can be files or directories. If a directory is specified, -all existing files/directories within are registered as protected recursively (and are expected -to be encrypted in the PF format). New files created in a protected directory are automatically -treated as protected. +Gramine allows mounting files and directories as encrypted. If a directory is +mounted as encrypted, all existing files/directories within it are recursively +treated as encrypted. + +See ``Documentation/manifest-syntax.rst`` for details. Limitations ----------- -Metadata currently limits PF path size to 512 bytes and filename size to 260 bytes. +Metadata currently limits PF path size to 512 bytes and filename size to 260 +bytes. NOTE ---- -The ``tools`` directory contains the ``pf_crypt`` utility that converts files to/from the protected -format. +The ``tools`` directory in Linux-SGX PAL contains the ``pf_crypt`` utility that +converts files to/from the protected format. Internal protected file format in this version was ported from the `SGX SDK `_. @@ -46,11 +60,14 @@ Internal protected file format in this version was ported from the `SGX SDK Tests ===== -Tests in ``LibOS/shim/test/fs`` contain PF tests (target is ``pf-test``). +Tests in ``LibOS/shim/test/fs`` contain encrypted file tests (``test_enc.py``). +Some tests in ``LibOS/shim/test/regression`` also work with encrypted files. TODO ==== - Truncating protected files is not yet implemented. -- The recovery file feature is disabled, this needs to be discussed if it's needed in Gramine. -- Tests for invalid/malformed/corrupted files need to be ported to the new format. +- The recovery file feature is disabled, this needs to be discussed if it's + needed in Gramine. +- Tests for invalid/malformed/corrupted files need to be ported to the new + format.