Skip to content

Commit

Permalink
[LibOS,Pal/Linux-SGX] Replace old protected files subsystem
Browse files Browse the repository at this point in the history 
This change replaces the protected files implemented in Linux-SGX PAL by
the new encrypted files subsystem in LibOS, implemented in previous
commits.

- Remove old implementation in SGX PAL (protected-file handles,
  reference counting, setting the key etc)
- Change the old user-facing interfaces (`sgx.protected_files`,
  `/dev/attestation/protected_files_key`) to control encrypted files, add
  deprecation warnings to them
- Add documentation for the new encrypted files syntax

Signed-off-by: Paweł Marczewski <[email protected]>
  • Loading branch information
@pwmarcz
pwmarcz committed May 10, 2022
1 parent 2598446 commit 4b18b55
Show file tree
Hide file tree
Showing 33 changed files with 555 additions and 1,657 deletions.
22 changes: 16 additions & 6 deletions Documentation/attestation.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -204,13 +204,23 @@ use the :program:`quote_dump`, :program:`ias_request` and
EPID based quote verification) or to use the Intel DCAP libraries and tools (for
DCAP based quote verification).

The ``/dev/attestation`` pseudo-filesystem also exposes a pseudo-file to set the
protected files wrap (master) key (see also :doc:`manifest-syntax`):
The ``/dev/attestation`` pseudo-filesystem also exposes pseudo-files to set the
encryption keys (see also :doc:`manifest-syntax`):

- ``/dev/attestation/protected_files_key`` pseudo-file can be opened for write
access. Typically, it is opened before the actual application runs and filled
with a 128-bit key obtained from a remote secret provisioning service.
- ``/dev/attestation/keys/<key_name>`` file contains the encryption key with a
given name (the default key name is ``default``). Typically, it is opened
before the actual application runs and filled with a 128-bit key obtained from
a remote secret provisioning service. The format of the file is a 16-byte raw
binary value.

.. note::
Previously, ``/dev/attestation/protected_files_key`` was used for setting the
default encryption key, and Gramine still supports that file for backward
compatibility.

Note that the old file (``/dev/attestation/protected_files_key``) uses a
32-character hex value, and the new files
(``/dev/attestation/keys/<key_name>``) use a 16-byte raw binary value.

Mid-level RA-TLS interface
--------------------------
Expand Down Expand Up @@ -398,7 +408,7 @@ EPID based ``secret_prov_verify_epid.so`` and DCAP/ECDSA based

The examples of using RA-TLS can be found under ``CI-Examples/ra-tls-secret-prov``.
The examples include minimalistic provisioning of constant-string secrets as
well as provisioning of an encryption key and its later use for protected files.
well as provisioning of an encryption key and its later use for encrypted files.

``secret_prov_attest.so``
^^^^^^^^^^^^^^^^^^^^^^^^^
Expand Down
2 changes: 1 addition & 1 deletion Documentation/devel/performance.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -425,7 +425,7 @@ encrypts many means of communication:
#. Inter-Process Communication (IPC) is encrypted via TLS-PSK. Regular pipes,
FIFO pipes, UNIX domain sockets are all transparently encrypted.

#. Files marked as ``sgx.protected_files`` are transparently encrypted/decrypted
#. Files mounted as ``type = "encrypted"`` are transparently encrypted/decrypted
on each file access via SGX SDK Merkle-tree format.

#. ``Fork/vfork/clone`` all require to generate an encrypted checkpoint of the
Expand Down
151 changes: 96 additions & 55 deletions Documentation/manifest-syntax.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -134,14 +134,15 @@ If you want your application to use commandline arguments you need to either set
``loader.argv_src_file`` to a file containing output of
:ref:`gramine-argv-serializer<gramine-argv-serializer>`.

``loader.argv_src_file`` is intended to point to either a trusted file or a
protected file. The former allows to securely hardcode arguments (current
``loader.argv_src_file`` is intended to point to either a trusted file or an
encrypted file. The former allows to securely hardcode arguments (current
manifest syntax doesn't allow to include them inline), the latter allows the
arguments to be provided at runtime from an external (trusted) source.

.. note ::
Pointing to a protected file is currently not supported, due to the fact that
PF wrap key provisioning currently happens after setting up arguments.
Pointing to an encrypted file is currently not supported, due to the fact
that encryption key provisioning currently happens after setting up
arguments.
Environment variables
^^^^^^^^^^^^^^^^^^^^^
Expand Down Expand Up @@ -180,15 +181,15 @@ value or be a passthrough.
``loader.env_src_file`` allows to specify a URI to a file containing serialized
environment, which can be generated using
:ref:`gramine-argv-serializer<gramine-argv-serializer>`. This option is intended
to point to either a trusted file or a protected file. The former allows to
to point to either a trusted file or an encrypted file. The former allows to
securely hardcode environments (in a more flexible way than
``loader.env.[ENVIRON]`` option), the latter allows the environments to be
provided at runtime from an external (trusted) source.

.. note ::
Pointing to a protected file is currently not supported, due to the fact that
PF wrap key provisioning currently happens after setting up environment
variables.
Pointing to an encrypted file is currently not supported, due to the fact
that encryption key provisioning currently happens after setting up
environment variables.
If the same variable is set in both, then ``loader.env.[ENVIRON]`` takes
precedence. It is prohibited to specify both ``value`` and ``passthrough`` keys
Expand Down Expand Up @@ -254,7 +255,7 @@ Gramine internal metadata size
(default: "0")

This syntax specifies how much additional memory Gramine reserves for its
internal use (e.g., metadata for trusted/protected files, internal handles,
internal use (e.g., metadata for trusted files, internal handles,
etc.). By default, Gramine pre-allocates 64MB of internal memory for this
metadata, but for huge workloads this limit may be not enough. In this case,
Gramine loudly fails with "out of PAL memory" error. To run huge workloads,
Expand Down Expand Up @@ -367,7 +368,7 @@ The ``type`` parameter specifies the mount point type. If omitted, it defaults
to ``"chroot"``. The ``path`` parameter must be an absolute path (i.e. must
begin with ``/``).

Gramine currently supports two types of mount points:
Gramine currently supports the following types of mount points:

* ``chroot`` (default): Host-backed files. All host files and sub-directories
found under ``[URI]`` are forwarded to the Gramine instance and placed under
Expand All @@ -378,6 +379,9 @@ Gramine currently supports two types of mount points:
Docker's named volumes. Files under ``chroot`` mount points support mmap and
fork/clone.

* ``encrypted``: Host-backed encrypted files. See :ref:`encrypted-files` for
more information.

* ``tmpfs``: Temporary in-memory-only files. These files are *not* backed by
host-level files. The tmpfs files are created under ``[PATH]`` (this path is
empty on Gramine instance startup) and are destroyed when a Gramine instance
Expand Down Expand Up @@ -598,7 +602,7 @@ are not cryptographically hashed and are thus not protected.

.. warning::
It is insecure to allow files containing code or critical information;
developers must not allow files blindly! Instead, use trusted or protected
developers must not allow files blindly! Instead, use trusted or encrypted
files.

Trusted files
Expand Down Expand Up @@ -631,67 +635,67 @@ Marking files as trusted is especially useful for shared libraries: a |~|
trusted library cannot be silently replaced by a malicious host because the hash
verification will fail.

Protected files
.. _encrypted-files:

Encrypted files
^^^^^^^^^^^^^^^

::

sgx.insecure__protected_files_key = "[16-byte hex value]"

sgx.protected_files = [
"[URI]",
"[URI]",
]

sgx.protected_mrenclave_files = [
"[URI]",
"[URI]",
fs.mounts = [
{ type = "encrypted", path = "[PATH]", uri = "[URI]", key_name = "[KEY_NAME]" },
]

sgx.protected_mrsigner_files = [
"[URI]",
"[URI]",
]
fs.insecure__keys.[KEY_NAME] = "[32-character hex value]"

This syntax specifies the files that are encrypted on disk and transparently
This syntax allows mounting files that are encrypted on disk and transparently
decrypted when accessed by Gramine or by application running inside Gramine.
Protected files guarantee data confidentiality and integrity (tamper
resistance), as well as file swap protection (a protected file can only be
accessed when in a specific path).
Encrypted files guarantee data confidentiality and integrity (tamper
resistance), as well as file swap protection (an encrypted file can only be
accessed when in a specific host path).

Encrypted files were previously known as *protected files*, and some Gramine
tools might still use the old name.

URI can be a file or a directory. If a directory is mounted, all existing
files/directories within it are recursively treated as encrypted (and are
expected to be encrypted in the PF format). New files created in an encrypted
mount are also automatically treated as encrypted.

URI can be a file or a directory. If a directory is specified, all existing
files/directories within it are registered as protected recursively (and are
expected to be encrypted in the PF format). New files created in a protected
directory are automatically treated as protected.
.. warning::
The current implementation assumes that ``type = "encrypted"`` mounts do not
overlap on host, i.e. there are no host files reachable through more than one
``type = "encrypted"`` mount. Otherwise, changes made to such files might not
be correctly persisted by Gramine.

Note that path size of a protected file is limited to 512 bytes and filename
Note that path size of an encrypted file is limited to 512 bytes and filename
size is limited to 260 bytes.

``sgx.insecure__protected_files_key`` specifies the wrap (master) encryption key
and must be used only for debugging purposes.
The ``key_name`` mount parameter specifies the name of the encryption key. If
omitted, it will default to ``"default"``. This feature can be used to mount
different files or directories with different encryption keys.

.. warning::
``sgx.insecure__protected_files_key`` hard-codes the key in the manifest.
This option is thus insecure and must not be used in production environments!
Typically, you want to provision the protected files wrap key using SGX
local/remote attestation, thus you should not specify the
``sgx.insecure__protected_files_key`` manifest option at all. Instead, use
the Secret Provisioning interface (see :doc:`attestation`).
``fs.insecure__keys.[KEY_NAME]`` can be used to specify the encryption keys
directly in manifest. This option must be used only for debugging purposes.

There are three types of protected files:
.. warning::
``sgx.insecure__keys.[KEY_NAME]`` hard-codes the key in the manifest. This
option is thus insecure and must not be used in production environments!
Typically, you want to provision the encryption keys using SGX
local/remote attestation, thus you should not specify any
``sgx.insecure__keys.[KEY_NAME]`` manifest options at all. Instead, use the
Secret Provisioning interface (see :doc:`attestation`).

* ``sgx.protected_files`` are encrypted using the wrap (master) encryption key;
they are well-suited for input files encrypted by the user and sent to the
deployment platform as well as for output files sent back to the user and
decrypted at the user side.
Key names beginning with underscore (``_``) denote special keys provided by
Gramine:

* ``sgx.protected_mrenclave_files`` are encrypted using the SGX sealing key
based on the MRENCLAVE identity of the enclave; they are useful to allow only
the same enclave (on the same platform) to unseal files.
* ``"_sgx_mrenclave"`` (SGX only) is the SGX sealing key based on the MRENCLAVE
identity of the enclave. This is useful to allow only the same enclave (on the
same platform) to unseal files.

* ``sgx.protected_mrsigner_files`` are encrypted using the SGX sealing key based
on the MRSIGNER identity of the enclave; they are useful to allow all enclaves
signed with the same key (and on the same platform) to unseal files.
* ``"_sgx_mrsigner"`` (SGX only) is the SGX sealing key based on the MRSIGNER
identity of the enclave. This is useful to allow all enclaves signed with the
same key (and on the same platform) to unseal files.

File check policy
^^^^^^^^^^^^^^^^^
Expand Down Expand Up @@ -876,3 +880,40 @@ Experimental sysfs topology support
fs.experimental__enable_sysfs_topology = [true|false]

This feature is now enabled by default and the option was removed.

Protected files (deprecated syntax)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

::

sgx.protected_files = [
"[URI]",
"[URI]",
]

sgx.protected_mrenclave_files = [
"[URI]",
"[URI]",
]

sgx.protected_mrsigner_files = [
"[URI]",
"[URI]",
]

This syntax specified the previous SGX-only protected files. It has been
replaced with ``type = "encrypted"`` mounts (see :ref:`encrypted-files`).

.. warning::
Gramine will attempt to convert this syntax to mounted filesystems, but might
fail to do so correctly in more complicated cases (e.g. when a single host
file belongs to multiple mounts). It is recommended to rewrite all usages of
this syntax to ``type = "encrypted"`` mounts.

::

fs.insecure__protected_files_key = "[32-character hex value]"

This syntax allowed specifying the default encryption key for protected files.
It has been replaced by ``fs.insecure__keys.[KEY_NAME]]``. Note that both old
and new syntax are suitable for debugging purposes only.
2 changes: 1 addition & 1 deletion Documentation/pal/host-abi.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -338,5 +338,5 @@ and to obtain an attestation report and quote.
.. doxygenfunction:: DkAttestationQuote
:project: pal

.. doxygenfunction:: DkSetProtectedFilesKey
.. doxygenfunction:: DkGetSpecialKey
:project: pal
5 changes: 5 additions & 0 deletions LibOS/shim/include/shim_fs_encrypted.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -171,4 +171,9 @@ int encrypted_file_rename(struct shim_encrypted_file* enc, const char* new_uri);
int encrypted_file_get_size(struct shim_encrypted_file* enc, file_off_t* out_size);
int encrypted_file_set_size(struct shim_encrypted_file* enc, file_off_t size);

int parse_pf_key(const char* key_str, pf_key_t* pf_key);

/* TODO: This function is used only by a feature deprecated in v1.2, remove two versions later. */
int dump_pf_key(const pf_key_t* pf_key, char* buf, size_t buf_size);

#endif /* SHIM_FS_ENCRYPTED_ */
70 changes: 0 additions & 70 deletions LibOS/shim/src/fs/chroot/fs.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -495,74 +495,6 @@ static int chroot_chmod(struct shim_dentry* dent, mode_t perm) {
return ret;
}

static int chroot_reopen(struct shim_handle* hdl, PAL_HANDLE* out_palhdl) {
PAL_HANDLE palhdl;

mode_t mode = 0;
enum pal_access access = LINUX_OPEN_FLAGS_TO_PAL_ACCESS(hdl->flags);
enum pal_create_mode create = PAL_CREATE_NEVER;
pal_stream_options_t options = LINUX_OPEN_FLAGS_TO_PAL_OPTIONS(hdl->flags);
int ret = DkStreamOpen(hdl->uri, access, mode, create, options, &palhdl);
if (ret < 0)
return pal_to_unix_errno(ret);
*out_palhdl = palhdl;
return 0;
}

/*
* Prepare the handle to be sent to child process. If the corresponding file still exists on the
* host, we will not checkpoint its PAL handle, but let the child process open another one.
*
* TODO: this is only necessary because PAL handles for protected files cannot be sent to child
* process (`DkSendHandle`). This workaround limits the damage: inheriting a handle by child process
* will now fail to work only if it's a handle for a protected file *and* the file has been deleted
* from host.
*/
static int chroot_checkout(struct shim_handle* hdl) {
assert(hdl->type == TYPE_CHROOT);
assert(hdl->pal_handle);

/* We should be holding `g_dcache_lock` for the whole checkpointing process. */
assert(locked(&g_dcache_lock));

/* We don't take `hdl->lock` because this is actually the handle *copied* for checkpointing (and
* the lock isn't even properly initialized). */

/* First, check if we have not deleted or renamed the file (the dentry contains the same
* inode). */
bool is_in_dentry = (hdl->dentry->inode == hdl->inode);

if (is_in_dentry) {
/* Then check if the file still exists on host. If so, we assume it can be opened by the
* child process, so the PAL handle doesn't need sending. */
PAL_STREAM_ATTR attr;
if (DkStreamAttributesQuery(hdl->uri, &attr) == 0) {
hdl->pal_handle = NULL;
}
}

return 0;
}

static int chroot_checkin(struct shim_handle* hdl) {
assert(hdl->type == TYPE_CHROOT);

/* We don't take `hdl->lock` because this handle is being initialized (during checkpoint
* restore). */

if (!hdl->pal_handle) {
PAL_HANDLE palhdl = NULL;
int ret = chroot_reopen(hdl, &palhdl);
if (ret < 0) {
log_warning("%s: failed to open %s: %d", __func__, hdl->uri, ret);
return ret;
}
assert(palhdl);
hdl->pal_handle = palhdl;
}
return 0;
}

struct shim_fs_ops chroot_fs_ops = {
.mount = &chroot_mount,
.flush = &chroot_flush,
Expand All @@ -576,8 +508,6 @@ struct shim_fs_ops chroot_fs_ops = {
.hstat = &generic_inode_hstat,
.truncate = &chroot_truncate,
.poll = &generic_inode_poll,
.checkout = &chroot_checkout,
.checkin = &chroot_checkin,
};

struct shim_d_ops chroot_d_ops = {
Expand Down
Loading

0 comments on commit 4b18b55

Please sign in to comment.