Skip to content

Commit

Permalink
Adds a section for HNS to semantics doc (#2510)
Browse files Browse the repository at this point in the history
* adds hns to semantics doc

* refactors as per review comments

* minor fix
  • Loading branch information
ankitaluthra1 authored Sep 23, 2024
1 parent b33f36e commit 847d810
Showing 1 changed file with 13 additions and 2 deletions.
15 changes: 13 additions & 2 deletions docs/semantics.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,17 @@ However, implicit directories does have drawbacks:

Alternatively, users can create a script which lists the buckets and creates the appropriate objects for the directories so that the ```--implicit-dirs``` flag is not used.

**Using Cloud Storage Buckets with [Hierarchical Namespaces Enabled](https://cloud.google.com/storage/docs/hns-overview) :**

Cloud Storage Fuse also offers seamless support for buckets with hierarchical namespaces enabled. Mounting an HNS-enabled bucket using gcsfuse works exactly like mounting a standard bucket, with no additional configuration required.

HNS-enabled buckets offer several advantages over standard buckets when used with cloud storage fuse:

- HNS-enabled buckets eliminates the need for --implicit-dirs flag. HNS buckets inherently understand directories, so gcsfuse does not need to simulate directories using placeholder objects ( 0-byte objects ending with '/' ). Users will see consistent directory listings with or without the flag.
- In HNS buckets, renaming a folder and its child folders is an atomic operation, meaning all associated resources—including objects and managed folders—are renamed in a single step. This ensures data consistency and significantly improves operation performance.
- HNS buckets treat folders as first-class entities, closely aligning with traditional file system semantics. Commands like mkdir now directly create folder resources within the bucket, unlike with traditional buckets where directories were simulated using prefixes and 0-byte objects.
- List object calls ([BucketHandle.Objects](https://cloud.google.com/storage/docs/json_api/v1/objects/list)), are replaced with [get folder](https://cloud.google.com/storage/docs/json_api/v1/folders/getfoldermetadata) calls, resulting in quicker response times and fewer overall list calls for every lookup operation.

# Generations

With each record in Cloud Storage is stored object and metadata [generation numbers](https://cloud.google.com/storage/docs/generations-preconditions). These provide a total order on requests to modify an object's contents and metadata, compatible with causality. So if insert operation A happens before insert operation B, then the generation number resulting from A will be less than that resulting from B.
Expand Down Expand Up @@ -402,8 +413,8 @@ Transient errors can occur in distributed systems like Cloud Storage, such as ne
## Missing features

Not all of the usual file system features are supported. Most prominently:
- Renaming directories is by default not supported. A directory rename cannot be performed atomically in Cloud Storage and would therefore be arbitrarily expensive in terms of Cloud Storage operations, and for large directories would have high probability of failure, leaving the two directories in an inconsistent state.
- However, if your application can tolerate the risks, you may enable renaming directories in a non-atomic way, by setting ```--rename-dir-limit```. If a directory contains fewer files than this limit and no subdirectory, it can be renamed.
- Renaming directories is only supported in Hierarchical Namespace Buckets, where they are fast and atomic. Renaming directories in flat namespace buckets is by default not supported. A directory rename cannot be performed atomically in these flat buckets and would therefore be arbitrarily expensive in terms of Cloud Storage operations, and for large directories would have high probability of failure, leaving the two directories in an inconsistent state.
- However, if your application is using Flat buckets and can tolerate the risks, you may enable renaming directories in a non-atomic way, by setting ```--rename-dir-limit```. If a directory contains fewer files than this limit and no subdirectory, it can be renamed.
- File and directory permissions and ownership cannot be changed. See the permissions section above.
- Modification times are not tracked for any inodes except for files.
- No other times besides modification time are tracked. For example, ctime and atime are not tracked (but will be set to something reasonable). Requests to change them will appear to succeed, but the results are unspecified.
Expand Down

0 comments on commit 847d810

Please sign in to comment.