Skip to content

Commit 043e46a

Browse files
nargokulnargokul
committed
refactor(integ): Replace Nova integ tests with example notebook
Remove us-east-1 Nova integration tests that cannot run in the us-west-2 CodeBuild environment: - Delete tests/integ/test_bedrock_nova_e2e.py - Delete tests/integ/conftest.py (Nova get-or-create fixture) - Remove TestBedrockNovaDeployment class from test_model_customization_deployment.py Add example_notebooks/bedrock_nova_deployment.ipynb covering the full Nova workflow: SFTTrainer fine-tuning, BedrockModelBuilder deploy with model+deployment polling, inference, and cleanup. The BedrockModelBuilder source code and unit tests (48 passing) are unchanged. The us-west-2 integ tests for non-Nova Bedrock deployment (TestModelCustomizationDeployment) remain.
1 parent 3ecf467 commit 043e46a

File tree

4 files changed

+269
-411
lines changed

4 files changed

+269
-411
lines changed

‎sagemaker-serve/example_notebooks/bedrock_nova_deployment.ipynb‎

Lines changed: 269 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,269 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Deploy a Nova Model to Amazon Bedrock\n",
8+
"\n",
9+
"This notebook demonstrates how to fine-tune an Amazon Nova model using the SageMaker SDK\n",
10+
"and deploy it to Amazon Bedrock using `BedrockModelBuilder`.\n",
11+
"\n",
12+
"The workflow:\n",
13+
"1. Fine-tune Nova Micro using `SFTTrainer`\n",
14+
"2. Create a `BedrockModelBuilder` from the completed training job\n",
15+
"3. Deploy to Bedrock — the builder automatically:\n",
16+
" - Detects the model as Nova\n",
17+
" - Reads the checkpoint URI from the training job manifest\n",
18+
" - Calls `CreateCustomModel`\n",
19+
" - Polls until the model is Active\n",
20+
" - Calls `CreateCustomModelDeployment`\n",
21+
" - Polls until the deployment is Active\n",
22+
"4. Clean up resources\n",
23+
"\n",
24+
"**Prerequisites:**\n",
25+
"- AWS credentials with SageMaker and Bedrock access in `us-east-1`\n",
26+
"- `sagemaker-serve` and `sagemaker-train` packages installed\n",
27+
"- An IAM role with Bedrock and SageMaker permissions"
28+
]
29+
},
30+
{
31+
"cell_type": "markdown",
32+
"metadata": {},
33+
"source": [
34+
"## Setup"
35+
]
36+
},
37+
{
38+
"cell_type": "code",
39+
"execution_count": null,
40+
"metadata": {},
41+
"outputs": [],
42+
"source": [
43+
"import os\n",
44+
"import json\n",
45+
"import time\n",
46+
"import random\n",
47+
"import boto3\n",
48+
"\n",
49+
"REGION = \"us-east-1\"\n",
50+
"os.environ[\"AWS_DEFAULT_REGION\"] = REGION\n",
51+
"\n",
52+
"from sagemaker.core.helper.session_helper import get_execution_role\n",
53+
"\n",
54+
"role_arn = get_execution_role()\n",
55+
"account_id = boto3.client(\"sts\").get_caller_identity()[\"Account\"]\n",
56+
"bucket = f\"sagemaker-{REGION}-{account_id}\"\n",
57+
"\n",
58+
"print(f\"Region: {REGION}\")\n",
59+
"print(f\"Account: {account_id}\")\n",
60+
"print(f\"Role: {role_arn}\")\n",
61+
"print(f\"Bucket: {bucket}\")"
62+
]
63+
},
64+
{
65+
"cell_type": "markdown",
66+
"metadata": {},
67+
"source": [
68+
"## Step 1: Prepare training data\n",
69+
"\n",
70+
"Upload a small JSONL dataset in the chat-messages format that Nova expects."
71+
]
72+
},
73+
{
74+
"cell_type": "code",
75+
"execution_count": null,
76+
"metadata": {},
77+
"outputs": [],
78+
"source": [
79+
"s3 = boto3.client(\"s3\", region_name=REGION)\n",
80+
"\n",
81+
"train_key = \"nova-example/train.jsonl\"\n",
82+
"train_uri = f\"s3://{bucket}/{train_key}\"\n",
83+
"\n",
84+
"rows = []\n",
85+
"for i in range(50):\n",
86+
" rows.append(json.dumps({\n",
87+
" \"messages\": [\n",
88+
" {\"role\": \"user\", \"content\": f\"What is {i+1} + {i+1}?\"},\n",
89+
" {\"role\": \"assistant\", \"content\": f\"The answer is {(i+1)*2}.\"}\n",
90+
" ]\n",
91+
" }))\n",
92+
"\n",
93+
"s3.put_object(Bucket=bucket, Key=train_key, Body=\"\\n\".join(rows).encode())\n",
94+
"print(f\"Uploaded {len(rows)} examples to {train_uri}\")"
95+
]
96+
},
97+
{
98+
"cell_type": "markdown",
99+
"metadata": {},
100+
"source": [
101+
"## Step 2: Fine-tune Nova Micro with SFTTrainer\n",
102+
"\n",
103+
"This launches a SageMaker training job. It typically takes 15-30 minutes to complete."
104+
]
105+
},
106+
{
107+
"cell_type": "code",
108+
"execution_count": null,
109+
"metadata": {},
110+
"outputs": [],
111+
"source": [
112+
"from sagemaker.train.sft_trainer import SFTTrainer\n",
113+
"\n",
114+
"trainer = SFTTrainer(\n",
115+
" model=\"nova-textgeneration-micro\",\n",
116+
" training_dataset=train_uri,\n",
117+
" accept_eula=True,\n",
118+
" model_package_group=\"nova-example-models\",\n",
119+
")\n",
120+
"\n",
121+
"# Set wait=True to block until training completes\n",
122+
"trainer.train(wait=True)\n",
123+
"\n",
124+
"training_job = trainer._latest_training_job\n",
125+
"print(f\"Training job: {training_job.training_job_name}\")\n",
126+
"print(f\"Status: {training_job.training_job_status}\")"
127+
]
128+
},
129+
{
130+
"cell_type": "markdown",
131+
"metadata": {},
132+
"source": [
133+
"## Step 3: Deploy to Bedrock with BedrockModelBuilder\n",
134+
"\n",
135+
"The builder handles the full deployment flow:\n",
136+
"- Fetches the model package from the training job\n",
137+
"- Detects it as a Nova model\n",
138+
"- Reads the checkpoint URI from the training output manifest\n",
139+
"- Creates a Bedrock custom model and polls until Active\n",
140+
"- Creates a deployment and polls until Active"
141+
]
142+
},
143+
{
144+
"cell_type": "code",
145+
"execution_count": null,
146+
"metadata": {},
147+
"outputs": [],
148+
"source": [
149+
"from sagemaker.serve.bedrock_model_builder import BedrockModelBuilder\n",
150+
"\n",
151+
"builder = BedrockModelBuilder(model=training_job)\n",
152+
"\n",
153+
"print(f\"Model package: {builder.model_package}\")\n",
154+
"print(f\"S3 artifacts: {builder.s3_model_artifacts}\")"
155+
]
156+
},
157+
{
158+
"cell_type": "code",
159+
"execution_count": null,
160+
"metadata": {},
161+
"outputs": [],
162+
"source": [
163+
"rand = random.randint(1000, 9999)\n",
164+
"custom_model_name = f\"nova-example-{rand}-{int(time.time())}\"\n",
165+
"deployment_name = f\"{custom_model_name}-dep\"\n",
166+
"\n",
167+
"print(f\"Deploying as: {custom_model_name}\")\n",
168+
"print(f\"This will poll for model creation and deployment — may take several minutes...\")\n",
169+
"\n",
170+
"response = builder.deploy(\n",
171+
" custom_model_name=custom_model_name,\n",
172+
" role_arn=role_arn,\n",
173+
" deployment_name=deployment_name,\n",
174+
")\n",
175+
"\n",
176+
"deployment_arn = response.get(\"customModelDeploymentArn\")\n",
177+
"print(f\"\\nDeployment ARN: {deployment_arn}\")\n",
178+
"print(\"Deployment is Active and ready for inference.\")"
179+
]
180+
},
181+
{
182+
"cell_type": "markdown",
183+
"metadata": {},
184+
"source": [
185+
"## Step 4: Test inference (optional)\n",
186+
"\n",
187+
"Once the deployment is Active, you can invoke it via the Bedrock Runtime API."
188+
]
189+
},
190+
{
191+
"cell_type": "code",
192+
"execution_count": null,
193+
"metadata": {},
194+
"outputs": [],
195+
"source": [
196+
"bedrock_runtime = boto3.client(\"bedrock-runtime\", region_name=REGION)\n",
197+
"\n",
198+
"# Get the model ARN from the deployment\n",
199+
"bedrock = boto3.client(\"bedrock\", region_name=REGION)\n",
200+
"dep_info = bedrock.get_custom_model_deployment(\n",
201+
" customModelDeploymentIdentifier=deployment_arn\n",
202+
")\n",
203+
"model_arn = dep_info.get(\"modelArn\")\n",
204+
"print(f\"Model ARN: {model_arn}\")\n",
205+
"\n",
206+
"# Invoke\n",
207+
"invoke_response = bedrock_runtime.invoke_model(\n",
208+
" modelId=deployment_arn,\n",
209+
" contentType=\"application/json\",\n",
210+
" body=json.dumps({\n",
211+
" \"messages\": [{\"role\": \"user\", \"content\": \"What is 7 + 7?\"}]\n",
212+
" }),\n",
213+
")\n",
214+
"\n",
215+
"result = json.loads(invoke_response[\"body\"].read())\n",
216+
"print(f\"Response: {result}\")"
217+
]
218+
},
219+
{
220+
"cell_type": "markdown",
221+
"metadata": {},
222+
"source": [
223+
"## Step 5: Cleanup\n",
224+
"\n",
225+
"Delete the deployment and custom model to avoid ongoing charges."
226+
]
227+
},
228+
{
229+
"cell_type": "code",
230+
"execution_count": null,
231+
"metadata": {},
232+
"outputs": [],
233+
"source": [
234+
"bedrock = boto3.client(\"bedrock\", region_name=REGION)\n",
235+
"\n",
236+
"# Delete deployment first\n",
237+
"if deployment_arn:\n",
238+
" try:\n",
239+
" bedrock.delete_custom_model_deployment(\n",
240+
" customModelDeploymentIdentifier=deployment_arn\n",
241+
" )\n",
242+
" print(f\"Deleted deployment: {deployment_arn}\")\n",
243+
" except Exception as e:\n",
244+
" print(f\"Failed to delete deployment: {e}\")\n",
245+
"\n",
246+
"# Then delete the custom model\n",
247+
"if model_arn:\n",
248+
" try:\n",
249+
" bedrock.delete_custom_model(modelIdentifier=model_arn)\n",
250+
" print(f\"Deleted custom model: {model_arn}\")\n",
251+
" except Exception as e:\n",
252+
" print(f\"Failed to delete custom model: {e}\")"
253+
]
254+
}
255+
],
256+
"metadata": {
257+
"kernelspec": {
258+
"display_name": "Python 3",
259+
"language": "python",
260+
"name": "python3"
261+
},
262+
"language_info": {
263+
"name": "python",
264+
"version": "3.12.0"
265+
}
266+
},
267+
"nbformat": 4,
268+
"nbformat_minor": 4
269+
}

0 commit comments

Comments
 (0)