Skip to content

Commit 0fc0fe4

Browse files
masquaremasquare
committed
initial commit
0 parents  commit 0fc0fe4

File tree

8 files changed

+734
-0
lines changed

8 files changed

+734
-0
lines changed

‎.gitignore

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Python-generated files
2+
__pycache__/
3+
*.py[oc]
4+
build/
5+
dist/
6+
wheels/
7+
*.egg-info
8+
9+
# Virtual environments
10+
.venv
11+
12+
13+
.DS_Store
14+
15+
.vscode
16+
17+
uv.lock

‎.python-version

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
3.12

‎README.md

+150
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# 🤗 Hugging Face Bedrock Importer: Easily deploy HF Models to Amazon Bedrock 🚀
2+
3+
The Hugging Face Bedrock Importer is a command-line tool that simplifies the process of downloading Hugging Face models and deploying them to Amazon Bedrock. This tool automates the workflow of model download, Amazon S3 upload, and Bedrock Custom Model Import, making it easier for developers to leverage Hugging Face language models in their AWS environment.
4+
5+
📚 Hugging Face models overview: https://huggingface.co/models
6+
7+
With this importer, you can quickly bring powerful language models from Hugging Face into your Bedrock ecosystem. It handles the complexities of model download, S3 storage management, and Bedrock custom model creation, allowing you to focus on utilizing these models in your applications.
8+
9+
## Installation
10+
11+
To use the Hugging Face Bedrock Importer, ensure you have Python 3.11+ installed. Then, follow these steps:
12+
13+
#### Install with pip
14+
```bash
15+
pip install git+https://github.com/masquare/huggingface-bedrock-importer.git
16+
```
17+
18+
#### Install with uv
19+
```bash
20+
uv tool install git+https://github.com/masquare/huggingface-bedrock-importer.git
21+
```
22+
23+
## Usage
24+
25+
> [!NOTE]
26+
> Ensure your AWS credentials are properly set up, either through environment variables or the AWS CLI configuration (see [docs](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html)).
27+
> Make sure Amazon Bedrock Custom Model Import is supported in the AWS Region you are planning to use. You can check region support [here](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-import-model.html).
28+
29+
To import a Hugging Face model to Bedrock, use the following command:
30+
```bash
31+
hf-bedrock-import --model-id <model_id> --s3-uri <s3_uri>
32+
```
33+
34+
Replace `<model_id>` with the Hugging Face model ID and `<s3_uri>` with the S3 URI where you want to store the model files (e.g., `s3://amzn-s3-demo-bucket/hf_models/`).
35+
36+
Example:
37+
```bash
38+
hf-bedrock-import --model-id deepseek-ai/DeepSeek-R1-Distill-Llama-8B --s3-uri s3://amzn-s3-demo-bucket/hf_models/
39+
```
40+
41+
### Configuration Options
42+
43+
The importer supports the following command-line options:
44+
45+
- `--model-id`: Hugging Face model ID (default: "deepseek-ai/DeepSeek-R1-Distill-Llama-8B")
46+
- `--s3-uri`: S3 URI for model storage
47+
- `--cleanup-resources`: Cleanup AWS resources (Bedrock custom model, IAM role, S3 model files)
48+
- `--cleanup-model`: Cleanup local model files
49+
- `--test`: Test the model after importing it
50+
51+
### Common Use Cases
52+
53+
1. Import a model and test it:
54+
```bash
55+
hf-bedrock-import --model-id deepseek-ai/DeepSeek-R1-Distill-Llama-8B --s3-uri s3://amzn-s3-demo-bucket/models/
56+
```
57+
58+
The script will print the model ARN that you can use to invoke the model, as well as a link to the Bedrock Playground where you can play around with the model:
59+
```
60+
Model ARN: arn:aws:bedrock:{AWS_REGION}:{ACCOUNT_ID}:imported-model/{MODEL_ID}
61+
62+
Link to the Bedrock playground for the model: https://{AWS_REGION}}.console.aws.amazon.com/bedrock/home#/text-generation-playground?mode=text&modelId=arn%3Aaws%3Abedrock%3A{AWS_REGION}%3A{ACCOUNT_ID}%3Aimported-model%2F{MODEL_ID}
63+
```
64+
65+
Note: Using custom models in Amazon Bedrock incurs costs. See the [Amazon Bedrock pricing page](https://aws.amazon.com/bedrock/pricing/) for more details.
66+
67+
68+
2. Clean up AWS resources for a specific model:
69+
```bash
70+
hf-bedrock-import --model-id deepseek-ai/DeepSeek-R1-Distill-Llama-8B --s3-uri s3://amzn-s3-demo-bucket/models/ --cleanup-resources
71+
```
72+
73+
3. Clean up local model files:
74+
```bash
75+
hf-bedrock-import --model-id bert-base-uncased --cleanup-model
76+
```
77+
78+
## Code Usage
79+
80+
Integrate the Hugging Face Bedrock Importer in your own code. Example:
81+
82+
```python
83+
from huggingface_bedrock_importer import importer
84+
85+
S3_URI = "s3://amzn-s3-demo-bucket/hf-models/"
86+
MODEL_ID = "deepseek-ai/DeepSeek-R1-Distill-Llama-8B"
87+
88+
# import model to Bedrock
89+
model_arn = importer.import_model_to_bedrock(MODEL_ID, S3_URI)
90+
91+
# use the model
92+
bedrock_runtime = boto3.client("bedrock-runtime"))
93+
prompt = "What is the capital of France?"
94+
invoke_response = bedrock_runtime.invoke_model(
95+
modelId=model_arn, body=json.dumps({"prompt": prompt})
96+
)
97+
invoke_response["body"] = json.loads(invoke_response["body"].read().decode("utf-8"))
98+
print(json.dumps(invoke_response, indent=4))
99+
100+
101+
# cleanup
102+
importer.cleanup_aws_resources(S3_URI, MODEL_ID)
103+
importer.cleanup_local_resources(MODEL_ID)
104+
```
105+
106+
### Troubleshooting
107+
108+
1. S3 Access Issues:
109+
- Problem: "Access Denied" errors when uploading to S3
110+
- Solution: Ensure your AWS credentials have the necessary permissions to write to the specified S3 bucket
111+
- Diagnostic steps:
112+
1. Check your AWS credentials configuration
113+
2. Verify IAM user/role permissions for S3 access
114+
3. Try uploading a test file to the S3 bucket using the AWS CLI
115+
116+
2. Model Download Failures:
117+
- Problem: Unable to download the model from Hugging Face
118+
- Solution: Verify internet connection and Hugging Face API status
119+
- Diagnostic steps:
120+
1. Check your internet connection
121+
2. Ensure the model ID is correct and publicly accessible
122+
3. Try downloading the model manually from the Hugging Face website
123+
124+
3. Bedrock Import Errors:
125+
- Problem: Model import to Bedrock fails
126+
- Solution: Check IAM role permissions and S3 bucket accessibility
127+
- Diagnostic steps:
128+
1. Verify the IAM role has the correct permissions for Bedrock and S3
129+
2. Ensure the S3 bucket is in the same region as your Bedrock endpoint
130+
3. Check Bedrock service quotas to ensure you haven't exceeded limits
131+
132+
## Data Flow
133+
134+
The Hugging Face Bedrock Importer follows this data flow when importing a model:
135+
136+
1. Download model from Hugging Face Hub to local storage
137+
2. Upload model files from local storage to specified S3 bucket
138+
3. Create or retrieve IAM role for Bedrock model import
139+
4. Initiate Bedrock model import job using S3 location and IAM role
140+
5. Wait for import job completion and retrieve model ARN
141+
6. (Optional) Test the imported model with a sample prompt
142+
143+
```mermaid
144+
flowchart TD
145+
hf[Hugging Face Hub] --> local[Local Storage]
146+
local --> s3[Amazon S3 Bucket]
147+
s3 --> bedrock[Amazon Bedrock]
148+
```
149+
150+
Note: Ensure sufficient storage capacity in local environment, as language models can be several gigabytes in size.

‎huggingface_bedrock_importer/__init__.py

Whitespace-only changes.

‎huggingface_bedrock_importer/__main__.py

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
if __name__ == "__main__":
2+
from huggingface_bedrock_importer.cli import main
3+
main()

‎huggingface_bedrock_importer/cli.py

+133
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
import argparse
2+
import sys
3+
import urllib
4+
5+
import boto3
6+
7+
from huggingface_bedrock_importer.importer import (
8+
cleanup_aws_resources,
9+
cleanup_local_resources,
10+
import_model_to_bedrock,
11+
test_model,
12+
)
13+
14+
15+
class bcolors:
16+
HEADER = "\033[95m"
17+
OKBLUE = "\033[94m"
18+
OKCYAN = "\033[96m"
19+
OKGREEN = "\033[92m"
20+
WARNING = "\033[93m"
21+
FAIL = "\033[91m"
22+
ENDC = "\033[0m"
23+
BOLD = "\033[1m"
24+
UNDERLINE = "\033[4m"
25+
26+
27+
def validate_aws_credentials():
28+
try:
29+
client = boto3.client("sts")
30+
client.get_caller_identity()
31+
print(
32+
f"{bcolors.HEADER}Using AWS region {client.meta.region_name}. Make sure all functionality is available there.{bcolors.ENDC}"
33+
)
34+
except Exception as e:
35+
print(
36+
f"{bcolors.FAIL}Unable to locate AWS credentials, please configure them: {e}{bcolors.ENDC}"
37+
)
38+
sys.exit(1)
39+
40+
41+
def main():
42+
parser = argparse.ArgumentParser(
43+
description="Download and deploy a Hugging Face model to AWS Bedrock"
44+
)
45+
parser.add_argument(
46+
"--model-id",
47+
help="Hugging Face model ID",
48+
default="deepseek-ai/DeepSeek-R1-Distill-Llama-8B",
49+
required=False,
50+
)
51+
parser.add_argument(
52+
"--s3-uri",
53+
help="S3 URI for model storage. Can be a bucket name only, or a prefix (e.g., s3://amzn-s3-demo-bucket/my_models)",
54+
required=False,
55+
)
56+
parser.add_argument(
57+
"--cleanup-resources",
58+
action="store_true",
59+
help="Cleanup AWS resources (Bedrock custom model, IAM role, S3 model files)",
60+
)
61+
parser.add_argument(
62+
"--cleanup-model",
63+
action="store_true",
64+
help="Cleanup local model files",
65+
)
66+
parser.add_argument(
67+
"--test",
68+
action="store_true",
69+
help="Test the model after importing it",
70+
)
71+
72+
try:
73+
args = parser.parse_args()
74+
except:
75+
parser.print_help()
76+
sys.exit(0)
77+
78+
ran_cleanup = False
79+
if args.cleanup_resources:
80+
print(
81+
f"{bcolors.WARNING}Destroying remote entities (Bedrock custom model, IAM role, S3 model files)...{bcolors.ENDC}"
82+
)
83+
if not args.model_id or not args.s3_uri:
84+
print(
85+
f"{bcolors.FAIL}Please provide both model_id and s3_uri to cleanup resources.{bcolors.ENDC}"
86+
)
87+
sys.exit(1)
88+
validate_aws_credentials()
89+
cleanup_aws_resources(args.s3_uri, args.model_id)
90+
print(f"{bcolors.OKGREEN}Done.{bcolors.ENDC}")
91+
ran_cleanup = True
92+
93+
if args.cleanup_model:
94+
print(f"{bcolors.WARNING}Cleaning up local model resources...{bcolors.ENDC}")
95+
if not args.model_id:
96+
print(
97+
f"{bcolors.FAIL}Please provide model_id to cleanup resources.{bcolors.ENDC}"
98+
)
99+
sys.exit(1)
100+
cleanup_local_resources(args.model_id)
101+
print(f"{bcolors.OKGREEN}Done.{bcolors.ENDC}")
102+
ran_cleanup = True
103+
104+
if not ran_cleanup:
105+
if not args.model_id or not args.s3_uri:
106+
print(
107+
f"{bcolors.FAIL}Please provide both model_id and s3_uri to import a model.{bcolors.ENDC}"
108+
)
109+
sys.exit(1)
110+
print(f"{bcolors.HEADER}Using model ID: {args.model_id}{bcolors.ENDC}")
111+
print(f"{bcolors.HEADER}Using S3 location: {args.s3_uri}{bcolors.ENDC}")
112+
113+
try:
114+
validate_aws_credentials()
115+
model_arn = import_model_to_bedrock(args.model_id, args.s3_uri)
116+
117+
print(f"{bcolors.BOLD}Process completed successfully!{bcolors.ENDC}")
118+
print(
119+
f"{bcolors.OKGREEN}Link to the Bedrock playground for the model: https://{model_arn.split(':')[3]}.console.aws.amazon.com/bedrock/home#/text-generation-playground?mode=text&modelId={urllib.parse.quote_plus(model_arn)}{bcolors.ENDC}"
120+
)
121+
122+
# Step 4: Test the model
123+
if args.test:
124+
print("Testing the model...")
125+
test_model(model_arn)
126+
127+
except Exception as e:
128+
print(f"{bcolors.FAIL}An error occurred: {repr(e)}{bcolors.ENDC}")
129+
raise e
130+
131+
132+
if __name__ == "__main__":
133+
main()

0 commit comments

Comments
 (0)