diff --git a/cloud-service-providers/aws/sagemaker/aws_marketplace_notebooks/data/medical-diarization.wav b/cloud-service-providers/aws/sagemaker/aws_marketplace_notebooks/data/medical-diarization.wav new file mode 100644 index 00000000..e1cfb0d6 Binary files /dev/null and b/cloud-service-providers/aws/sagemaker/aws_marketplace_notebooks/data/medical-diarization.wav differ diff --git a/cloud-service-providers/aws/sagemaker/aws_marketplace_notebooks/data/test.wav b/cloud-service-providers/aws/sagemaker/aws_marketplace_notebooks/data/test.wav new file mode 100644 index 00000000..bd4276be Binary files /dev/null and b/cloud-service-providers/aws/sagemaker/aws_marketplace_notebooks/data/test.wav differ diff --git a/cloud-service-providers/aws/sagemaker/aws_marketplace_notebooks/nim_nvidia-parakeet-tdt-0-6b-v2_aws_marketplace.ipynb b/cloud-service-providers/aws/sagemaker/aws_marketplace_notebooks/nim_nvidia-parakeet-tdt-0-6b-v2_aws_marketplace.ipynb new file mode 100644 index 00000000..7bd0d294 --- /dev/null +++ b/cloud-service-providers/aws/sagemaker/aws_marketplace_notebooks/nim_nvidia-parakeet-tdt-0-6b-v2_aws_marketplace.ipynb @@ -0,0 +1,651 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "fa058068", + "metadata": {}, + "source": [ + "# Deploy NVIDIA Parakeet TDT 0.6B v2 ASR from AWS Marketplace\n" + ] + }, + { + "cell_type": "markdown", + "id": "0f5b8560", + "metadata": {}, + "source": [ + "NVIDIA NIM, a component of NVIDIA AI Enterprise, provides state-of-the-art Automatic Speech Recognition (ASR) capabilities through the Parakeet TDT 0.6B v2 model. This model delivers high-quality speech-to-text conversion with efficient processing and support for real-time inference. Whether you're developing voice assistants, transcription services, or any application that needs to convert speech to text, NVIDIA NIM for ASR has you covered.\n" + ] + }, + { + "cell_type": "markdown", + "id": "47056661", + "metadata": {}, + "source": [ + "In this example we show how to deploy the NVIDIA Parakeet TDT 0.6B v2 ASR model from AWS Marketplace\n", + "\n", + "NVIDIA Parakeet TDT 0.6B v2 is an Automatic Speech Recognition (ASR) model optimized for efficient processing and real-time inference. This model provides high-quality speech-to-text conversion with excellent accuracy and low latency. The model is designed to handle various audio formats and provides robust transcription capabilities for voice applications.\n", + "\n", + "Key Features:\n", + "- **High Accuracy**: State-of-the-art speech recognition performance\n", + "- **Real-time Processing**: Optimized for low-latency inference\n", + "- **Efficient**: 0.6B parameter model for fast processing\n", + "- **GPU Optimized**: Supports multiple GPU architectures (L40S, A100, H100)\n", + "\n", + "The model supports English language processing and is optimized for various audio quality conditions.\n" + ] + }, + { + "cell_type": "markdown", + "id": "89e46469", + "metadata": {}, + "source": [ + "Please check out the [NIM ASR docs](https://docs.nvidia.com/nim/automatic-speech-recognition/latest/introduction.html) for more information.\n" + ] + }, + { + "cell_type": "markdown", + "id": "4e7a5238", + "metadata": {}, + "source": [ + "## ⚠️ Important Notes\n", + "\n", + "This ASR model supports **automatic routing** between HTTP and gRPC protocols based on audio file size and requirements:\n", + "\n", + "- **Small files (< 4MB)**: Automatically routed through HTTP for fast processing\n", + "- **Large files (≥ 4MB)**: Automatically routed through gRPC for efficient streaming\n", + "- **Force HTTP**: Use header `X-Amzn-SageMaker-Custom-Attributes: /invocations/http` or POST to `/invocations/http`\n", + "- **Force gRPC**: Use header `X-Amzn-SageMaker-Custom-Attributes: /invocations/grpc` or POST to `/invocations/grpc`\n", + "\n", + "The gRPC mode enables advanced features like speaker diarization and word-level timestamps.\n" + ] + }, + { + "cell_type": "markdown", + "id": "c9e58135", + "metadata": {}, + "source": [ + "## Pre-requisites:\n", + "1. **Note**: This notebook contains elements which render correctly in Jupyter interface. Open this notebook from an Amazon SageMaker Notebook Instance or Amazon SageMaker Studio.\n", + "1. Ensure that IAM role used has **AmazonSageMakerFullAccess**\n", + "1. To deploy this ML model successfully, ensure that:\n", + " 1. Either your IAM role has these three permissions and you have authority to make AWS Marketplace subscriptions in the AWS account used: \n", + " 1. **aws-marketplace:ViewSubscriptions**\n", + " 1. **aws-marketplace:Unsubscribe**\n", + " 1. **aws-marketplace:Subscribe** \n", + " 2. or your AWS account has a subscription to the Parakeet TDT 0.6B v2 ASR model package.\n", + "1. **Audio Requirements**: The model accepts audio files in common formats (WAV, MP3, FLAC, etc.) and supports various sample rates.\n" + ] + }, + { + "cell_type": "markdown", + "id": "d4a58b90", + "metadata": {}, + "source": [ + "## Subscribe to the model package\n", + "To subscribe to the model package:\n", + "1. Open the model package listing page\n", + "1. On the AWS Marketplace listing, click on the **Continue to subscribe** button.\n", + "1. On the **Subscribe to this software** page, review and click on **\"Accept Offer\"** if you and your organization agrees with EULA, pricing, and support terms. \n", + "1. Once you click on **Continue to configuration button** and then choose a **region**, you will see a **Product Arn** displayed. This is the model package ARN that you need to specify while creating a deployable model. Copy the ARN corresponding to your region and specify the same in the following cell.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "52dbb98e", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:41:48.917043Z", + "iopub.status.busy": "2025-10-22T23:41:48.916772Z", + "iopub.status.idle": "2025-10-22T23:41:49.109037Z", + "shell.execute_reply": "2025-10-22T23:41:49.108029Z", + "shell.execute_reply.started": "2025-10-22T23:41:48.917023Z" + } + }, + "outputs": [], + "source": [ + "!pip install soundfile\n", + "import boto3, json, sagemaker, time, os\n", + "from sagemaker import get_execution_role, ModelPackage\n", + "from botocore.config import Config\n", + "\n", + "config = Config(read_timeout=3600)\n", + "sess = boto3.Session()\n", + "sm = sess.client(\"sagemaker\")\n", + "sagemaker_session = sagemaker.Session(boto_session=sess)\n", + "role = get_execution_role()\n", + "client = boto3.client(\"sagemaker-runtime\", config=config)\n", + "region = sess.region_name\n", + "sm_runtime = boto3.client(\"sagemaker-runtime\")\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "b552f608", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:41:49.336325Z", + "iopub.status.busy": "2025-10-22T23:41:49.336090Z", + "iopub.status.idle": "2025-10-22T23:41:49.348557Z", + "shell.execute_reply": "2025-10-22T23:41:49.348048Z", + "shell.execute_reply.started": "2025-10-22T23:41:49.336308Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "'arn:aws:sagemaker:us-east-1:865070037744:model-package/nvidia-parakeet-tdt-0-6b-v2-02d6240216f539a185542e8e7f41706d'" + ] + }, + "execution_count": 26, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# replace the arn below with the model package arn you want to deploy\n", + "nim_package = \"nvidia-parakeet-tdt-0-6b-v2-02d6240216f539a185542e8e7f41706d\"\n", + "\n", + "# Mapping for Model Packages\n", + "model_package_map = {\n", + " \"us-east-1\": f\"arn:aws:sagemaker:us-east-1:865070037744:model-package/{nim_package}\",\n", + " \"us-east-2\": f\"arn:aws:sagemaker:us-east-2:057799348421:model-package/{nim_package}\",\n", + " \"us-west-1\": f\"arn:aws:sagemaker:us-west-1:382657785993:model-package/{nim_package}\",\n", + " \"us-west-2\": f\"arn:aws:sagemaker:us-west-2:594846645681:model-package/{nim_package}\",\n", + " \"ca-central-1\": f\"arn:aws:sagemaker:ca-central-1:470592106596:model-package/{nim_package}\",\n", + " \"eu-central-1\": f\"arn:aws:sagemaker:eu-central-1:446921602837:model-package/{nim_package}\",\n", + " \"eu-west-1\": f\"arn:aws:sagemaker:eu-west-1:985815980388:model-package/{nim_package}\",\n", + " \"eu-west-2\": f\"arn:aws:sagemaker:eu-west-2:856760150666:model-package/{nim_package}\",\n", + " \"eu-west-3\": f\"arn:aws:sagemaker:eu-west-3:843114510376:model-package/{nim_package}\",\n", + " \"eu-north-1\": f\"arn:aws:sagemaker:eu-north-1:136758871317:model-package/{nim_package}\",\n", + " \"ap-southeast-1\": f\"arn:aws:sagemaker:ap-southeast-1:192199979996:model-package/{nim_package}\",\n", + " \"ap-southeast-2\": f\"arn:aws:sagemaker:ap-southeast-2:666831318237:model-package/{nim_package}\",\n", + " \"ap-northeast-2\": f\"arn:aws:sagemaker:ap-northeast-2:745090734665:model-package/{nim_package}\",\n", + " \"ap-northeast-1\": f\"arn:aws:sagemaker:ap-northeast-1:977537786026:model-package/{nim_package}\",\n", + " \"ap-south-1\": f\"arn:aws:sagemaker:ap-south-1:077584701553:model-package/{nim_package}\",\n", + " \"sa-east-1\": f\"arn:aws:sagemaker:sa-east-1:270155090741:model-package/{nim_package}\",\n", + "}\n", + "\n", + "region = boto3.Session().region_name\n", + "if region not in model_package_map.keys():\n", + " raise Exception(f\"Current boto3 session region {region} is not supported.\")\n", + "\n", + "model_package_arn = model_package_map[region]\n", + "model_package_arn\n" + ] + }, + { + "cell_type": "markdown", + "id": "2c101589", + "metadata": {}, + "source": [ + "## Create the SageMaker Endpoint\n", + "\n", + "We first define SageMaker model using the specified ModelPackageArn.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "3c908204", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:41:51.339994Z", + "iopub.status.busy": "2025-10-22T23:41:51.339783Z", + "iopub.status.idle": "2025-10-22T23:41:52.001478Z", + "shell.execute_reply": "2025-10-22T23:41:52.000764Z", + "shell.execute_reply.started": "2025-10-22T23:41:51.339974Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Model Arn: arn:aws:sagemaker:us-east-1:492681118881:model/nvidia-parakeet-tdt-0-6b-v2-asr\n" + ] + } + ], + "source": [ + "# Define the model details\n", + "sm_model_name = \"nvidia-parakeet-tdt-0-6b-v2-asr\"\n", + "\n", + "# Create the SageMaker model\n", + "create_model_response = sm.create_model(\n", + " ModelName=sm_model_name,\n", + " PrimaryContainer={\n", + " 'ModelPackageName': model_package_arn\n", + " },\n", + " ExecutionRoleArn=role,\n", + " EnableNetworkIsolation=True\n", + ")\n", + "print(\"Model Arn: \" + create_model_response[\"ModelArn\"])\n" + ] + }, + { + "cell_type": "markdown", + "id": "b0c74caf", + "metadata": {}, + "source": [ + "Next we create endpoint configuration specifying instance type\n" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "id": "264d87b3", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:44:31.268465Z", + "iopub.status.busy": "2025-10-22T23:44:31.268178Z", + "iopub.status.idle": "2025-10-22T23:44:31.657563Z", + "shell.execute_reply": "2025-10-22T23:44:31.657059Z", + "shell.execute_reply.started": "2025-10-22T23:44:31.268446Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Endpoint Config Arn: arn:aws:sagemaker:us-east-1:492681118881:endpoint-config/nvidia-parakeet-tdt-0-6b-v2-asr\n" + ] + } + ], + "source": [ + "# Create the endpoint configuration\n", + "endpoint_config_name = sm_model_name\n", + "\n", + "create_endpoint_config_response = sm.create_endpoint_config(\n", + " EndpointConfigName=endpoint_config_name,\n", + " ProductionVariants=[\n", + " {\n", + " 'VariantName': 'AllTraffic',\n", + " 'ModelName': sm_model_name,\n", + " 'InitialInstanceCount': 1,\n", + " 'InstanceType': 'ml.g6e.2xlarge', \n", + " 'InferenceAmiVersion': 'al2-ami-sagemaker-inference-gpu-2',\n", + " 'RoutingConfig': {'RoutingStrategy': 'LEAST_OUTSTANDING_REQUESTS'},\n", + " 'ModelDataDownloadTimeoutInSeconds': 3600, # Specify the model download timeout in seconds.\n", + " 'ContainerStartupHealthCheckTimeoutInSeconds': 3600, # Specify the health checkup timeout in seconds\n", + " }\n", + " ]\n", + ")\n", + "print(\"Endpoint Config Arn: \" + create_endpoint_config_response[\"EndpointConfigArn\"])\n" + ] + }, + { + "cell_type": "markdown", + "id": "bf36262f", + "metadata": {}, + "source": [ + "Using the above endpoint configuration we create a new sagemaker endpoint and wait for the deployment to finish. The status will change to InService once the deployment is successful.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "id": "efeb3703", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:44:33.256155Z", + "iopub.status.busy": "2025-10-22T23:44:33.255964Z", + "iopub.status.idle": "2025-10-22T23:44:33.832707Z", + "shell.execute_reply": "2025-10-22T23:44:33.832223Z", + "shell.execute_reply.started": "2025-10-22T23:44:33.256138Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Endpoint Arn: arn:aws:sagemaker:us-east-1:492681118881:endpoint/nvidia-parakeet-tdt-0-6b-v2-asr\n" + ] + } + ], + "source": [ + "# Create the endpoint\n", + "endpoint_name = endpoint_config_name\n", + "create_endpoint_response = sm.create_endpoint(\n", + " EndpointName=endpoint_name,\n", + " EndpointConfigName=endpoint_config_name\n", + ")\n", + "\n", + "print(\"Endpoint Arn: \" + create_endpoint_response[\"EndpointArn\"])\n" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "id": "01ba1bb0", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:56:13.809475Z", + "iopub.status.busy": "2025-10-22T23:56:13.809234Z", + "iopub.status.idle": "2025-10-22T23:56:13.938681Z", + "shell.execute_reply": "2025-10-22T23:56:13.938233Z", + "shell.execute_reply.started": "2025-10-22T23:56:13.809455Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Status: InService\n", + "Arn: arn:aws:sagemaker:us-east-1:492681118881:endpoint/nvidia-parakeet-tdt-0-6b-v2-asr\n", + "Status: InService\n" + ] + } + ], + "source": [ + "resp = sm.describe_endpoint(EndpointName=endpoint_name)\n", + "status = resp[\"EndpointStatus\"]\n", + "print(\"Status: \" + status)\n", + "\n", + "while status == \"Creating\":\n", + " time.sleep(60)\n", + " resp = sm.describe_endpoint(EndpointName=endpoint_name)\n", + " status = resp[\"EndpointStatus\"]\n", + " print(\"Status: \" + status)\n", + "\n", + "print(\"Arn: \" + resp[\"EndpointArn\"])\n", + "print(\"Status: \" + status)\n" + ] + }, + { + "cell_type": "markdown", + "id": "bc9f3bc4", + "metadata": {}, + "source": [ + "## Run Inference\n", + "\n", + "### Test Endpoint with Different Protocols\n", + "\n", + "Our NIM ASR endpoint supports multiple inference methods:\n", + "\n", + "1. **Auto-routing** (`/invocations`): Automatically chooses HTTP or gRPC based on file size\n", + "2. **Force HTTP** (`X-Amzn-SageMaker-Custom-Attributes: /invocations/http`): Direct HTTP route\n", + "\n", + "#### Test 1: Auto-routing (Recommended)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "id": "a6e4fc84-8aef-4d76-a223-148bed9dbb98", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:56:16.715847Z", + "iopub.status.busy": "2025-10-22T23:56:16.715553Z", + "iopub.status.idle": "2025-10-22T23:56:17.298657Z", + "shell.execute_reply": "2025-10-22T23:56:17.298157Z", + "shell.execute_reply.started": "2025-10-22T23:56:16.715810Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Testing auto-routing with data/test.wav\n", + "Audio file size: 237,964 bytes (0.23 MB)\n", + "result: \n", + "\n", + "{'text': \"Well, I don't wish to see it any more, observed Phebe, turning away her eyes. It is certainly very like the old portrait. \"}\n", + "\n", + "Auto-routing inference successful!\n", + "Response: {\n", + " \"text\": \"Well, I don't wish to see it any more, observed Phebe, turning away her eyes. It is certainly very like the old portrait. \"\n", + "}\n" + ] + }, + { + "data": { + "text/plain": [ + "{'text': \"Well, I don't wish to see it any more, observed Phebe, turning away her eyes. It is certainly very like the old portrait. \"}" + ] + }, + "execution_count": 37, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def test_asr_endpoint_autorouting(audio_file_path, endpoint_name):\n", + " \"\"\"Test endpoint with auto-routing\"\"\"\n", + " print(f\"Testing auto-routing with {audio_file_path}\")\n", + " \n", + " try:\n", + " # Read audio file\n", + " with open(audio_file_path, 'rb') as f:\n", + " audio_data = f.read()\n", + " \n", + " file_size = len(audio_data)\n", + " print(f\"Audio file size: {file_size:,} bytes ({file_size / (1024*1024):.2f} MB)\")\n", + " \n", + " # Create multipart form data\n", + " boundary = f\"----WebKitFormBoundary{uuid.uuid4().hex}\"\n", + " \n", + " # Build multipart payload\n", + " parts = []\n", + " parts.append(f'--{boundary}')\n", + " parts.append('Content-Disposition: form-data; name=\"file\"; filename=\"test.wav\"')\n", + " parts.append('Content-Type: audio/wav')\n", + " parts.append('')\n", + " \n", + " # Join text parts\n", + " text_part = '\\r\\n'.join(parts) + '\\r\\n'\n", + " language_part = f'\\r\\n--{boundary}\\r\\nContent-Disposition: form-data; name=\"language_code\"\\r\\n\\r\\nen-US\\r\\n--{boundary}--'\n", + " \n", + " # Combine all parts\n", + " payload = text_part.encode() + audio_data + language_part.encode()\n", + " \n", + " # Invoke endpoint\n", + " response = sm_runtime.invoke_endpoint(\n", + " EndpointName=endpoint_name,\n", + " ContentType=f'multipart/form-data; boundary={boundary}',\n", + " Body=payload\n", + " )\n", + " \n", + " # Parse response\n", + " result = json.loads(response['Body'].read().decode())\n", + " print(\"result: \\n\")\n", + " print(result)\n", + " print(f\"\\nAuto-routing inference successful!\")\n", + " print(f\"Response: {json.dumps(result, indent=2)}\")\n", + " return result\n", + " \n", + " except Exception as e:\n", + " print(f\"Auto-routing test failed: {e}\")\n", + " return None \n", + "\n", + "# Run auto-routing test\n", + "# Example usage (replace with your audio file path)\n", + "audio_file = \"data/test.wav\"\n", + "test_asr_endpoint_autorouting(audio_file, endpoint_name)\n" + ] + }, + { + "cell_type": "markdown", + "id": "79834abb", + "metadata": {}, + "source": [ + "#### Test 2: Force HTTP Route\n", + "\n", + "Test the HTTP-only route (optimized for files <5MB):\n" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "id": "0d1fb2f6", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:56:18.716164Z", + "iopub.status.busy": "2025-10-22T23:56:18.715930Z", + "iopub.status.idle": "2025-10-22T23:56:18.818596Z", + "shell.execute_reply": "2025-10-22T23:56:18.817962Z", + "shell.execute_reply.started": "2025-10-22T23:56:18.716144Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Testing HTTP route with data/test.wav\n", + "\n", + "HTTP route inference successful!\n", + "Response: {\n", + " \"text\": \"Well, I don't wish to see it any more, observed Phebe, turning away her eyes. It is certainly very like the old portrait. \"\n", + "}\n" + ] + } + ], + "source": [ + "# Test 2: Force HTTP route\n", + "def test_endpoint_http_route(audio_file_path):\n", + " \"\"\"Test endpoint with forced HTTP route\"\"\"\n", + " print(f\"Testing HTTP route with {audio_file_path}\")\n", + " \n", + " try:\n", + " # Read audio file\n", + " with open(audio_file_path, 'rb') as f:\n", + " audio_data = f.read()\n", + " \n", + " # Create multipart form data\n", + " boundary = f\"----WebKitFormBoundary{uuid.uuid4().hex}\"\n", + " \n", + " # Build multipart payload (same as auto-routing)\n", + " parts = []\n", + " parts.append(f'--{boundary}')\n", + " parts.append('Content-Disposition: form-data; name=\"file\"; filename=\"test.wav\"')\n", + " parts.append('Content-Type: audio/wav')\n", + " parts.append('')\n", + " \n", + " text_part = '\\r\\n'.join(parts) + '\\r\\n'\n", + " language_part = f'\\r\\n--{boundary}\\r\\nContent-Disposition: form-data; name=\"language_code\"\\r\\n\\r\\nen-US\\r\\n--{boundary}--'\n", + " payload = text_part.encode() + audio_data + language_part.encode()\n", + " \n", + " # Invoke endpoint with HTTP route forced\n", + " response = sm_runtime.invoke_endpoint(\n", + " EndpointName=endpoint_name,\n", + " ContentType=f'multipart/form-data; boundary={boundary}',\n", + " Body=payload,\n", + " CustomAttributes='/invocations/http' # Force HTTP route\n", + " )\n", + " \n", + " result = json.loads(response['Body'].read().decode())\n", + " print(f\"\\nHTTP route inference successful!\")\n", + " print(f\"Response: {json.dumps(result, indent=2)}\")\n", + " return result\n", + " \n", + " except Exception as e:\n", + " print(f\"HTTP route test failed: {e}\")\n", + " return None\n", + "\n", + "# Run HTTP route test\n", + "audio_file = \"data/test.wav\"\n", + "http_result = test_endpoint_http_route(audio_file)" + ] + }, + { + "cell_type": "markdown", + "id": "3ce63ea9", + "metadata": {}, + "source": [ + "### Clean up resources\n", + "\n", + "Clean up the SageMaker endpoint and related resources when you're done testing.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "id": "ea55b1de", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:56:41.404535Z", + "iopub.status.busy": "2025-10-22T23:56:41.404269Z", + "iopub.status.idle": "2025-10-22T23:56:42.040505Z", + "shell.execute_reply": "2025-10-22T23:56:42.039492Z", + "shell.execute_reply.started": "2025-10-22T23:56:41.404516Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Cleaning up SageMaker resources...\n", + "✅ Deleted endpoint: nvidia-parakeet-tdt-0-6b-v2-asr\n", + "✅ Deleted endpoint config: nvidia-parakeet-tdt-0-6b-v2-asr\n", + "✅ Deleted model: nvidia-parakeet-tdt-0-6b-v2-asr\n", + "Cleanup completed!\n" + ] + } + ], + "source": [ + "# Clean up SageMaker resources\n", + "print(\"Cleaning up SageMaker resources...\")\n", + "\n", + "# Delete the endpoint\n", + "try:\n", + " sm.delete_endpoint(EndpointName=endpoint_name)\n", + " print(f\"✅ Deleted endpoint: {endpoint_name}\")\n", + "except Exception as e:\n", + " print(f\"⚠️ Error deleting endpoint: {e}\")\n", + "\n", + "# Delete the endpoint configuration\n", + "try:\n", + " sm.delete_endpoint_config(EndpointConfigName=endpoint_config_name)\n", + " print(f\"✅ Deleted endpoint config: {endpoint_config_name}\")\n", + "except Exception as e:\n", + " print(f\"⚠️ Error deleting endpoint config: {e}\")\n", + "\n", + "# Delete the model\n", + "try:\n", + " sm.delete_model(ModelName=sm_model_name)\n", + " print(f\"✅ Deleted model: {sm_model_name}\")\n", + "except Exception as e:\n", + " print(f\"⚠️ Error deleting model: {e}\")\n", + "\n", + "print(\"Cleanup completed!\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "42eea0c8-efe7-44c3-bbd2-d96cdf2d92ad", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.9" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/cloud-service-providers/aws/sagemaker/aws_marketplace_notebooks/nim_nvidia_-parakeet-1-1b-ctc-en-us_aws_marketplace.ipynb b/cloud-service-providers/aws/sagemaker/aws_marketplace_notebooks/nim_nvidia_-parakeet-1-1b-ctc-en-us_aws_marketplace.ipynb new file mode 100644 index 00000000..2ed00c19 --- /dev/null +++ b/cloud-service-providers/aws/sagemaker/aws_marketplace_notebooks/nim_nvidia_-parakeet-1-1b-ctc-en-us_aws_marketplace.ipynb @@ -0,0 +1,2571 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f0488cae-f2b2-4d0f-9e42-7cd4faae07d8", + "metadata": {}, + "source": [ + "# NVIDIA NIM ASR Model Deployment on Amazon SageMaker AI\n", + "\n", + "## Introduction\n", + "\n", + "The **parakeet-ctc-1.1b-en-us** is a 1.1-billion-parameter automatic speech recognition (ASR) model developed by NVIDIA, designed for high-accuracy English (US) transcription with robust performance across diverse acoustic conditions. It delivers industry-leading accuracy for offline speech-to-text applications, making it an excellent choice for developers and researchers requiring reliable transcription capabilities.\n", + "\n", + "## About the model:\n", + "\n", + "**Architecture:** The model architecture is based on the FastConformer encoder combined with a CTC (Connectionist Temporal Classification) decoder. This proven architecture enables efficient and accurate speech transcription with excellent generalization across different speaking styles and acoustic environments.\n", + "\n", + "**Model:** Parakeet CTC 1.1B EN-US\n", + "\n", + "**Performance:** The model achieves exceptional word error rate (WER) performance with fast inference speeds on NVIDIA GPUs, providing state-of-the-art accuracy for US English transcription tasks. The larger 1.1B parameter size enables superior handling of challenging audio conditions including accents, background noise, and domain-specific vocabulary.\n", + "\n", + "**Features:**\n", + "- High-accuracy transcription for US English speech\n", + "- Robust performance across diverse acoustic conditions and speaking styles\n", + "- Automatic punctuation and capitalization\n", + "- Excellent handling of spontaneous speech, disfluencies, and conversational audio\n", + "- Supports various audio formats and sampling rates\n", + "- Optimized for NVIDIA GPUs (Ampere, Hopper, Blackwell, Volta architectures)\n", + "\n", + "**Deployment:** The model is deployment-ready for production inference and can be integrated into platforms such as Amazon SageMaker. It is compatible with NVIDIA NeMo toolkit and packaged as a NIM microservice, enabling seamless deployment, scaling, and integration for enterprise speech transcription applications.\n", + "\n", + "## Use Cases:\n", + "\n", + "- **Call Center Analytics** - High-accuracy transcription of customer service calls, sales conversations, and support interactions for quality assurance and analytics.\n", + "- **Meeting & Conference Transcription** - Reliable transcription of business meetings, webinars, and conference calls with support for multiple speakers and spontaneous speech.\n", + "- **Medical Documentation** - Accurate dictation and transcription for clinical notes, patient consultations, and healthcare documentation where precision is paramount.\n", + "- **Legal & Compliance** - Transcription of depositions, court proceedings, and legal interviews where accuracy and reliability are critical.\n", + "- **Content Creation & Media** - Transcription of podcasts, interviews, and video content for subtitles, captions, and content accessibility.\n", + "- **Enterprise Voice Applications** - Integration into business workflows for voice-enabled documentation, automated note-taking, and voice-command systems.\n", + "\n", + "## Helpful Links\n", + "\n", + "**Free API Access & Prototyping** \n", + "[NVIDIA API Catalog (build.nvidia.com)](https://build.nvidia.com)\n", + "- Get free API credits for testing (10K requests)\n", + "- Try multiple NVIDIA speech models through browser UI\n", + "- Generate API keys for integration into your applications\n", + "\n", + "**Technical Overview & Deployment Guide** \n", + "[NVIDIA Speech AI Models Blog Post](https://developer.nvidia.com/blog)\n", + "- Comprehensive overview of all Parakeet models\n", + "- Performance benchmarks and use case examples\n", + "- Links to deployment options (Riva, NIM, NGC)" + ] + }, + { + "cell_type": "markdown", + "id": "d1048b66-14f3-4f47-91fd-e653979c7cd5", + "metadata": {}, + "source": [ + "In this example we show how to deploy the **Parakeet CTC 1.1B EN-US** NIM from AWS Marketplace on Amazon SageMaker. The Parakeet CTC 1.1B EN-US NIM simplifies the deployment of the automatic speech recognition model which is optimized for high-accuracy US English transcription with robust performance across diverse acoustic conditions, punctuation, and capitalization, and outperforms many available open source ASR models on common industry benchmarks. The NIM is built on robust foundations including inference engines like Triton Inference Server,and NVIDIA Riva. NIM provides features like low latency, high throughput, metrics export, standard API, optimized profiles & enterprise support." + ] + }, + { + "cell_type": "markdown", + "id": "740a3631-f207-4803-98ed-45d6895aa80a", + "metadata": {}, + "source": [ + "## Pre-requisites:\n", + "1. **Note**: This notebook contains elements which render correctly in Jupyter interface. Open this notebook from an Amazon SageMaker Notebook Instance or Amazon SageMaker Studio.\n", + "1. Ensure that IAM role used has **AmazonSageMakerFullAccess**\n", + "1. To deploy this ML model successfully, ensure that:\n", + " 1. Either your IAM role has these three permissions and you have authority to make AWS Marketplace subscriptions in the AWS account used: \n", + " 1. **aws-marketplace:ViewSubscriptions**\n", + " 1. **aws-marketplace:Unsubscribe**\n", + " 1. **aws-marketplace:Subscribe** \n", + " 2. or your AWS account has a subscription to one of the models listed above.\n" + ] + }, + { + "cell_type": "markdown", + "id": "5004eb9a-5817-484c-a412-2ce80a2e2f36", + "metadata": {}, + "source": [ + "## Subscribe to the model package\n", + "To subscribe to the model package:\n", + "1. Open the model package listing page\n", + "1. On the AWS Marketplace listing, click on the **Continue to subscribe** button.\n", + "1. On the **Subscribe to this software** page, review and click on **\"Accept Offer\"** if you and your organization agrees with EULA, pricing, and support terms. \n", + "1. Once you click on **Continue to configuration button** and then choose a **region**, you will see a **Product Arn** displayed. This is the model package ARN that you need to specify while creating a deployable model. Copy the ARN corresponding to your region and specify the same in the following cell." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "288fcdc9-3dbd-4bff-8e49-40783a5a2b3e", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:38:38.837681Z", + "iopub.status.busy": "2025-10-22T23:38:38.837431Z", + "iopub.status.idle": "2025-10-22T23:38:39.100139Z", + "shell.execute_reply": "2025-10-22T23:38:39.098977Z", + "shell.execute_reply.started": "2025-10-22T23:38:38.837662Z" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "!pip install soundfile\n", + "import boto3, json, sagemaker, time, os\n", + "from sagemaker import get_execution_role, ModelPackage\n", + "import time\n", + "\n", + "sess = boto3.Session()\n", + "sm = sess.client(\"sagemaker\")\n", + "sagemaker_session = sagemaker.Session(boto_session=sess)\n", + "role = get_execution_role()\n", + "client = boto3.client(\"sagemaker-runtime\")\n", + "region = sess.region_name" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "adc82963-2d0b-4fb6-8ef4-a0e5f6a15c2c", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:38:40.396765Z", + "iopub.status.busy": "2025-10-22T23:38:40.396478Z", + "iopub.status.idle": "2025-10-22T23:38:40.410898Z", + "shell.execute_reply": "2025-10-22T23:38:40.410150Z", + "shell.execute_reply.started": "2025-10-22T23:38:40.396743Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "'arn:aws:sagemaker:us-east-1:865070037744:model-package/nvidia-parakeet-1-1b-ctc-en-us-7c08e725839733449bed125e2a10a2c8'" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# replace the arn below with the model package arn you want to deploy\n", + "nim_package = \"nvidia-parakeet-1-1b-ctc-en-us-7c08e725839733449bed125e2a10a2c8\" # Need to change \n", + "\n", + "# Mapping for Model Packages\n", + "model_package_map = {\n", + " \"us-east-1\": f\"arn:aws:sagemaker:us-east-1:865070037744:model-package/{nim_package}\",\n", + " \"us-east-2\": f\"arn:aws:sagemaker:us-east-2:057799348421:model-package/{nim_package}\",\n", + " \"us-west-1\": f\"arn:aws:sagemaker:us-west-1:382657785993:model-package/{nim_package}\",\n", + " \"us-west-2\": f\"arn:aws:sagemaker:us-west-2:594846645681:model-package/{nim_package}\",\n", + " \"ca-central-1\": f\"arn:aws:sagemaker:ca-central-1:470592106596:model-package/{nim_package}\",\n", + " \"eu-central-1\": f\"arn:aws:sagemaker:eu-central-1:446921602837:model-package/{nim_package}\",\n", + " \"eu-west-1\": f\"arn:aws:sagemaker:eu-west-1:985815980388:model-package/{nim_package}\",\n", + " \"eu-west-2\": f\"arn:aws:sagemaker:eu-west-2:856760150666:model-package/{nim_package}\",\n", + " \"eu-west-3\": f\"arn:aws:sagemaker:eu-west-3:843114510376:model-package/{nim_package}\",\n", + " \"eu-north-1\": f\"arn:aws:sagemaker:eu-north-1:136758871317:model-package/{nim_package}\",\n", + " \"ap-southeast-1\": f\"arn:aws:sagemaker:ap-southeast-1:192199979996:model-package/{nim_package}\",\n", + " \"ap-southeast-2\": f\"arn:aws:sagemaker:ap-southeast-2:666831318237:model-package/{nim_package}\",\n", + " \"ap-northeast-2\": f\"arn:aws:sagemaker:ap-northeast-2:745090734665:model-package/{nim_package}\",\n", + " \"ap-northeast-1\": f\"arn:aws:sagemaker:ap-northeast-1:977537786026:model-package/{nim_package}\",\n", + " \"ap-south-1\": f\"arn:aws:sagemaker:ap-south-1:077584701553:model-package/{nim_package}\",\n", + " \"sa-east-1\": f\"arn:aws:sagemaker:sa-east-1:270155090741:model-package/{nim_package}\",\n", + "}\n", + "\n", + "region = boto3.Session().region_name\n", + "if region not in model_package_map.keys():\n", + " raise Exception(f\"Current boto3 session region {region} is not supported.\")\n", + "\n", + "model_package_arn = model_package_map[region]\n", + "model_package_arn" + ] + }, + { + "cell_type": "markdown", + "id": "f8132cd1-6055-4e45-9a92-1e74726ed61b", + "metadata": {}, + "source": [ + "## Create the SageMaker Endpoint\n", + "\n", + "We first define SageMaker model using the specified ModelPackageArn." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "07ee9475-8611-41d1-8c29-5df33f66a1bc", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:38:55.421329Z", + "iopub.status.busy": "2025-10-22T23:38:55.420981Z", + "iopub.status.idle": "2025-10-22T23:38:55.992265Z", + "shell.execute_reply": "2025-10-22T23:38:55.991688Z", + "shell.execute_reply.started": "2025-10-22T23:38:55.421295Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Model Arn: arn:aws:sagemaker:us-east-1:492681118881:model/parakeet-ctc-1-1b-en-us\n" + ] + } + ], + "source": [ + "# Define the model details\n", + "sm_model_name ='parakeet-ctc-1-1b-en-us'\n", + "timestamp = int(time.time())\n", + "\n", + "# Create the SageMaker model\n", + "create_model_response = sm.create_model(\n", + " ModelName=sm_model_name,\n", + " PrimaryContainer={\n", + " 'ModelPackageName': model_package_arn\n", + " },\n", + " ExecutionRoleArn=role,\n", + " EnableNetworkIsolation=True\n", + ")\n", + "print(\"Model Arn: \" + create_model_response[\"ModelArn\"])\n" + ] + }, + { + "cell_type": "markdown", + "id": "a7e096db-0e25-467b-aa9c-18a6a4f1ca60", + "metadata": { + "tags": [] + }, + "source": [ + "Next we create endpoint configuration specifying instance type, in this case it's g5.4xlarge or g5.12xlarge." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "002bb1b0-d5a6-439d-aa32-01f3fb0953a7", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:39:05.653361Z", + "iopub.status.busy": "2025-10-22T23:39:05.653071Z", + "iopub.status.idle": "2025-10-22T23:39:06.031227Z", + "shell.execute_reply": "2025-10-22T23:39:06.030718Z", + "shell.execute_reply.started": "2025-10-22T23:39:05.653342Z" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Endpoint Config Arn: arn:aws:sagemaker:us-east-1:492681118881:endpoint-config/parakeet-ctc-1-1b-en-us\n" + ] + } + ], + "source": [ + "# Create the endpoint configuration\n", + "endpoint_config_name = sm_model_name\n", + "\n", + "create_endpoint_config_response = sm.create_endpoint_config(\n", + " EndpointConfigName=endpoint_config_name,\n", + " ProductionVariants=[\n", + " {\n", + " 'VariantName': 'primary',\n", + " 'ModelName': sm_model_name,\n", + " 'InitialInstanceCount': 1,\n", + " 'InstanceType': 'ml.g5.4xlarge', # or g5.12xlarge\n", + " 'ModelDataDownloadTimeoutInSeconds': 3600, # Specify the model download timeout in seconds.\n", + " 'ContainerStartupHealthCheckTimeoutInSeconds': 3600, # Specify the health checkup timeout in seconds\n", + " }\n", + " ]\n", + ")\n", + "print(\"Endpoint Config Arn: \" + create_endpoint_config_response[\"EndpointConfigArn\"])" + ] + }, + { + "cell_type": "markdown", + "id": "4065575f-2ad7-438e-a5be-0e80c995414c", + "metadata": {}, + "source": [ + "Using the above endpoint configuration we create a new sagemaker endpoint and wait for the deployment to finish. The status will change to InService once the deployment is successful." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "b28fcac3-4c9d-420e-87fe-7d170b752738", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:39:07.166023Z", + "iopub.status.busy": "2025-10-22T23:39:07.165812Z", + "iopub.status.idle": "2025-10-22T23:39:07.700287Z", + "shell.execute_reply": "2025-10-22T23:39:07.699638Z", + "shell.execute_reply.started": "2025-10-22T23:39:07.166006Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Endpoint Arn: arn:aws:sagemaker:us-east-1:492681118881:endpoint/parakeet-ctc-1-1b-en-us\n" + ] + } + ], + "source": [ + "# Create the endpoint\n", + "endpoint_name = endpoint_config_name\n", + "create_endpoint_response = sm.create_endpoint(\n", + " EndpointName=endpoint_name,\n", + " EndpointConfigName=endpoint_config_name\n", + ")\n", + "\n", + "print(\"Endpoint Arn: \" + create_endpoint_response[\"EndpointArn\"])" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "42dbe400-99f0-4888-a71e-00259a86fc17", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:39:08.897176Z", + "iopub.status.busy": "2025-10-22T23:39:08.896977Z", + "iopub.status.idle": "2025-10-22T23:47:09.817865Z", + "shell.execute_reply": "2025-10-22T23:47:09.817121Z", + "shell.execute_reply.started": "2025-10-22T23:39:08.897160Z" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Status: Creating\n", + "Status: Creating\n", + "Status: Creating\n", + "Status: Creating\n", + "Status: Creating\n", + "Status: Creating\n", + "Status: Creating\n", + "Status: Creating\n", + "Status: InService\n", + "Arn: arn:aws:sagemaker:us-east-1:492681118881:endpoint/parakeet-ctc-1-1b-en-us\n", + "Status: InService\n" + ] + } + ], + "source": [ + "resp = sm.describe_endpoint(EndpointName=endpoint_name)\n", + "status = resp[\"EndpointStatus\"]\n", + "print(\"Status: \" + status)\n", + "\n", + "while status == \"Creating\":\n", + " time.sleep(60)\n", + " resp = sm.describe_endpoint(EndpointName=endpoint_name)\n", + " status = resp[\"EndpointStatus\"]\n", + " print(\"Status: \" + status)\n", + "\n", + "print(\"Arn: \" + resp[\"EndpointArn\"])\n", + "print(\"Status: \" + status)" + ] + }, + { + "cell_type": "markdown", + "id": "4eb28262-0ea5-44b6-a4d2-258793382911", + "metadata": {}, + "source": [ + "## Inference Testing and Validation\n", + "\n", + "### Test Audio File Preparation\n", + "\n", + "Let's create a sample audio file for testing our NIM ASR endpoint:" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "12ed8c97-57e1-4858-8f56-c5e0cec86eb2", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:56:47.534718Z", + "iopub.status.busy": "2025-10-22T23:56:47.534470Z", + "iopub.status.idle": "2025-10-22T23:56:47.537858Z", + "shell.execute_reply": "2025-10-22T23:56:47.537256Z", + "shell.execute_reply.started": "2025-10-22T23:56:47.534698Z" + } + }, + "outputs": [], + "source": [ + "# Create a simple test audio file using text-to-speech or use existing file\n", + "import tempfile\n", + "import soundfile as sf\n", + "\n", + "# Create test audio file\n", + "test_audio_path = \"data/test.wav\"" + ] + }, + { + "cell_type": "markdown", + "id": "c2a97e4c-6dd8-4d9a-841c-e443b7c1583f", + "metadata": {}, + "source": [ + "### Run Inference" + ] + }, + { + "cell_type": "markdown", + "id": "c9146f44-b85c-4125-b842-fceaf5c3cfa8", + "metadata": {}, + "source": [ + "### Test Endpoint with Different Protocols\n", + "\n", + "Our NIM ASR endpoint supports multiple inference methods:\n", + "\n", + "1. **Auto-routing** (`/invocations`): Automatically chooses HTTP or gRPC based on file size\n", + "2. **Force HTTP** (`X-Amzn-SageMaker-Custom-Attributes: /invocations/http`): Direct HTTP route\n", + "3. **Force gRPC** (`X-Amzn-SageMaker-Custom-Attributes: /invocations/grpc`): Direct gRPC route with diarization support\n", + "\n", + "#### Test 1: Auto-routing (Recommended)" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "a57265e9-98bb-4255-ad7d-143e3aeaf9d4", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:56:53.323159Z", + "iopub.status.busy": "2025-10-22T23:56:53.322903Z", + "iopub.status.idle": "2025-10-22T23:56:54.151081Z", + "shell.execute_reply": "2025-10-22T23:56:54.150420Z", + "shell.execute_reply.started": "2025-10-22T23:56:53.323139Z" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Testing auto-routing with data/test.wav\n", + "Audio file size: 237,964 bytes (0.23 MB)\n", + "Result:/n\n", + "{'ResponseMetadata': {'RequestId': '04603604-f707-441d-9402-f80f424d2988', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '04603604-f707-441d-9402-f80f424d2988', 'x-amzn-invoked-production-variant': 'primary', 'date': 'Wed, 22 Oct 2025 23:56:54 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '135', 'connection': 'keep-alive'}, 'RetryAttempts': 0}, 'ContentType': 'application/json; charset=utf-8', 'InvokedProductionVariant': 'primary', 'Body': }\n", + "\n", + "Auto-routing inference successful!\n", + "Response: {\n", + " \"text\": \"Well, I don't wish to see it any more, observed Phoebe, turning away her eyes. It is certainly very like the old portrait. \"\n", + "}\n" + ] + } + ], + "source": [ + "# Test 1: Auto-routing inference\n", + "import uuid\n", + "sm_runtime = boto3.client('sagemaker-runtime')\n", + "def test_endpoint_auto_routing(audio_file_path):\n", + " \"\"\"Test endpoint with auto-routing\"\"\"\n", + " print(f\"Testing auto-routing with {audio_file_path}\")\n", + " \n", + " try:\n", + " # Read audio file\n", + " with open(audio_file_path, 'rb') as f:\n", + " audio_data = f.read()\n", + " \n", + " file_size = len(audio_data)\n", + " print(f\"Audio file size: {file_size:,} bytes ({file_size / (1024*1024):.2f} MB)\")\n", + " \n", + " # Create multipart form data\n", + " boundary = f\"----WebKitFormBoundary{uuid.uuid4().hex}\"\n", + " \n", + " # Build multipart payload\n", + " parts = []\n", + " parts.append(f'--{boundary}')\n", + " parts.append('Content-Disposition: form-data; name=\"file\"; filename=\"test.wav\"')\n", + " parts.append('Content-Type: audio/wav')\n", + " parts.append('')\n", + " \n", + " # Join text parts\n", + " text_part = '\\r\\n'.join(parts) + '\\r\\n'\n", + " language_part = f'\\r\\n--{boundary}\\r\\nContent-Disposition: form-data; name=\"language_code\"\\r\\n\\r\\nen-US\\r\\n--{boundary}--'\n", + " \n", + " # Combine all parts\n", + " payload = text_part.encode() + audio_data + language_part.encode()\n", + "\n", + " # print(\"Input:/n\")\n", + " # print(payload)\n", + " \n", + " # Invoke endpoint\n", + " response = sm_runtime.invoke_endpoint(\n", + " EndpointName=endpoint_name,\n", + " ContentType=f'multipart/form-data; boundary={boundary}',\n", + " Body=payload\n", + " )\n", + "\n", + " print(\"Result:/n\")\n", + " print(response)\n", + " \n", + " # Parse response\n", + " result = json.loads(response['Body'].read().decode())\n", + " print(f\"\\nAuto-routing inference successful!\")\n", + " print(f\"Response: {json.dumps(result, indent=2)}\")\n", + " return result\n", + " \n", + " except Exception as e:\n", + " print(f\"Auto-routing test failed: {e}\")\n", + " return None\n", + "\n", + "# Run auto-routing test\n", + "auto_result = test_endpoint_auto_routing(test_audio_path)" + ] + }, + { + "cell_type": "markdown", + "id": "368d4021-9c10-4fb7-a4a8-8b45aaf701c8", + "metadata": {}, + "source": [ + "#### Test 2: Force HTTP Route\n", + "Test the HTTP-only route (optimized for files <5MB):" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "id": "6257bdc3-58c4-412d-ba85-c903b6a520f1", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:56:58.746908Z", + "iopub.status.busy": "2025-10-22T23:56:58.746691Z", + "iopub.status.idle": "2025-10-22T23:56:59.295695Z", + "shell.execute_reply": "2025-10-22T23:56:59.295006Z", + "shell.execute_reply.started": "2025-10-22T23:56:58.746890Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Testing HTTP route with data/test.wav\n", + "\n", + "HTTP route inference successful!\n", + "Response: {\n", + " \"text\": \"Well, I don't wish to see it any more, observed Phoebe, turning away her eyes. It is certainly very like the old portrait. \"\n", + "}\n" + ] + } + ], + "source": [ + "# Test 2: Force HTTP route\n", + "def test_endpoint_http_route(audio_file_path):\n", + " \"\"\"Test endpoint with forced HTTP route\"\"\"\n", + " print(f\"Testing HTTP route with {audio_file_path}\")\n", + " \n", + " try:\n", + " # Read audio file\n", + " with open(audio_file_path, 'rb') as f:\n", + " audio_data = f.read()\n", + " \n", + " # Create multipart form data\n", + " boundary = f\"----WebKitFormBoundary{uuid.uuid4().hex}\"\n", + " \n", + " # Build multipart payload (same as auto-routing)\n", + " parts = []\n", + " parts.append(f'--{boundary}')\n", + " parts.append('Content-Disposition: form-data; name=\"file\"; filename=\"test.wav\"')\n", + " parts.append('Content-Type: audio/wav')\n", + " parts.append('')\n", + " \n", + " text_part = '\\r\\n'.join(parts) + '\\r\\n'\n", + " language_part = f'\\r\\n--{boundary}\\r\\nContent-Disposition: form-data; name=\"language_code\"\\r\\n\\r\\nen-US\\r\\n--{boundary}--'\n", + " payload = text_part.encode() + audio_data + language_part.encode()\n", + " \n", + " # Invoke endpoint with HTTP route forced\n", + " response = sm_runtime.invoke_endpoint(\n", + " EndpointName=endpoint_name,\n", + " ContentType=f'multipart/form-data; boundary={boundary}',\n", + " Body=payload,\n", + " CustomAttributes='/invocations/http' # Force HTTP route\n", + " )\n", + " \n", + " result = json.loads(response['Body'].read().decode())\n", + " print(f\"\\nHTTP route inference successful!\")\n", + " print(f\"Response: {json.dumps(result, indent=2)}\")\n", + " return result\n", + " \n", + " except Exception as e:\n", + " print(f\"HTTP route test failed: {e}\")\n", + " return None\n", + "\n", + "# Run HTTP route test\n", + "http_result = test_endpoint_http_route(test_audio_path)" + ] + }, + { + "cell_type": "markdown", + "id": "cb1f5dac-1243-4ffc-8dc9-3abd67b610f7", + "metadata": {}, + "source": [ + "#### Test 3: Force gRPC Route with Speaker Diarization\n", + "\n", + "Test the gRPC route with speaker diarization capabilities:" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "8bef5e0e-3aa1-450f-baf9-34b7da73cf03", + "metadata": { + "collapsed": true, + "execution": { + "iopub.execute_input": "2025-10-22T23:57:07.182091Z", + "iopub.status.busy": "2025-10-22T23:57:07.181805Z", + "iopub.status.idle": "2025-10-22T23:57:12.219174Z", + "shell.execute_reply": "2025-10-22T23:57:12.218682Z", + "shell.execute_reply.started": "2025-10-22T23:57:07.182072Z" + }, + "jupyter": { + "outputs_hidden": true + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Testing gRPC route with speaker diarization: True\n", + "\n", + "gRPC route inference successful!\n", + "Speaker diarization enabled: True\n", + "Max speakers: 4\n", + "Response: {\n", + " \"predictions\": [\n", + " {\n", + " \"results\": [\n", + " {\n", + " \"alternatives\": [\n", + " {\n", + " \"transcript\": \"Hey Jane, so what brings you into my office today? \",\n", + " \"confidence\": 0.1833893358707428,\n", + " \"words\": [\n", + " {\n", + " \"word\": \"Hey\",\n", + " \"start_time\": 1.04,\n", + " \"end_time\": 1.28,\n", + " \"confidence\": 0.14124539494514465,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"Jane,\",\n", + " \"start_time\": 1.36,\n", + " \"end_time\": 1.6,\n", + " \"confidence\": 0.0971662625670433,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"so\",\n", + " \"start_time\": 1.76,\n", + " \"end_time\": 1.84,\n", + " \"confidence\": 0.10358044505119324,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"what\",\n", + " \"start_time\": 1.92,\n", + " \"end_time\": 2.0,\n", + " \"confidence\": 0.16306869685649872,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"brings\",\n", + " \"start_time\": 2.08,\n", + " \"end_time\": 2.32,\n", + " \"confidence\": 0.1342037469148636,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"you\",\n", + " \"start_time\": 2.32,\n", + " \"end_time\": 2.4,\n", + " \"confidence\": 0.6987138390541077,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"into\",\n", + " \"start_time\": 2.48,\n", + " \"end_time\": 2.56,\n", + " \"confidence\": 0.2678801417350769,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"my\",\n", + " \"start_time\": 2.64,\n", + " \"end_time\": 2.72,\n", + " \"confidence\": 0.31000736355781555,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"office\",\n", + " \"start_time\": 2.8,\n", + " \"end_time\": 2.88,\n", + " \"confidence\": 0.22248579561710358,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"today?\",\n", + " \"start_time\": 3.04,\n", + " \"end_time\": 3.12,\n", + " \"confidence\": 0.10713304579257965,\n", + " \"speaker_tag\": 0\n", + " }\n", + " ]\n", + " }\n", + " ],\n", + " \"is_final\": true,\n", + " \"channel_tag\": 1\n", + " },\n", + " {\n", + " \"alternatives\": [\n", + " {\n", + " \"transcript\": \"Hey Dr., Michaels, good to see you. I'm just coming in from my routine shake up. All right, let's see. I, I last saw you about what? Like a year ago and at that time I think you were having some minor headaches. I don't recall prescribing anything and we said we'd maintain some observations unless things were getting worse. \",\n", + " \"confidence\": 0.16683875024318695,\n", + " \"words\": [\n", + " {\n", + " \"word\": \"hey\",\n", + " \"start_time\": 3.68,\n", + " \"end_time\": 3.84,\n", + " \"confidence\": 0.14164088666439056,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"michaels\",\n", + " \"start_time\": 4.32,\n", + " \"end_time\": 4.88,\n", + " \"confidence\": 0.020878443494439125,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"good\",\n", + " \"start_time\": 5.12,\n", + " \"end_time\": 5.2,\n", + " \"confidence\": 0.057266540825366974,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"to\",\n", + " \"start_time\": 5.36,\n", + " \"end_time\": 5.44,\n", + " \"confidence\": 0.2002880871295929,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"see\",\n", + " \"start_time\": 5.44,\n", + " \"end_time\": 5.52,\n", + " \"confidence\": 0.41729333996772766,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"you\",\n", + " \"start_time\": 5.6,\n", + " \"end_time\": 5.68,\n", + " \"confidence\": 0.32379451394081116,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"i'm\",\n", + " \"start_time\": 5.84,\n", + " \"end_time\": 6.08,\n", + " \"confidence\": 0.2194986641407013,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"just\",\n", + " \"start_time\": 6.08,\n", + " \"end_time\": 6.16,\n", + " \"confidence\": 0.3098202645778656,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"coming\",\n", + " \"start_time\": 6.24,\n", + " \"end_time\": 6.32,\n", + " \"confidence\": 0.08914101123809814,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"in\",\n", + " \"start_time\": 6.48,\n", + " \"end_time\": 6.56,\n", + " \"confidence\": 0.4488358795642853,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"from\",\n", + " \"start_time\": 6.56,\n", + " \"end_time\": 6.64,\n", + " \"confidence\": 0.3653273582458496,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"my\",\n", + " \"start_time\": 6.72,\n", + " \"end_time\": 6.8,\n", + " \"confidence\": 0.1789325475692749,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"routine\",\n", + " \"start_time\": 6.88,\n", + " \"end_time\": 7.12,\n", + " \"confidence\": 0.06537152081727982,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"shake\",\n", + " \"start_time\": 7.2,\n", + " \"end_time\": 7.44,\n", + " \"confidence\": 0.0376877561211586,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"up\",\n", + " \"start_time\": 7.44,\n", + " \"end_time\": 7.52,\n", + " \"confidence\": 0.1354888677597046,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"all\",\n", + " \"start_time\": 7.92,\n", + " \"end_time\": 8.0,\n", + " \"confidence\": 0.09671889245510101,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"right\",\n", + " \"start_time\": 8.24,\n", + " \"end_time\": 8.32,\n", + " \"confidence\": 0.027603471651673317,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"let's\",\n", + " \"start_time\": 8.48,\n", + " \"end_time\": 8.72,\n", + " \"confidence\": 0.14779438078403473,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"see\",\n", + " \"start_time\": 8.72,\n", + " \"end_time\": 8.8,\n", + " \"confidence\": 0.4694344401359558,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"i\",\n", + " \"start_time\": 8.88,\n", + " \"end_time\": 8.96,\n", + " \"confidence\": 0.3056332767009735,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"i\",\n", + " \"start_time\": 9.12,\n", + " \"end_time\": 9.2,\n", + " \"confidence\": 0.34396037459373474,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"last\",\n", + " \"start_time\": 9.28,\n", + " \"end_time\": 9.36,\n", + " \"confidence\": 0.03096597082912922,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"saw\",\n", + " \"start_time\": 9.44,\n", + " \"end_time\": 9.6,\n", + " \"confidence\": 1.0,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"you\",\n", + " \"start_time\": 9.68,\n", + " \"end_time\": 9.76,\n", + " \"confidence\": 0.36910480260849,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"about\",\n", + " \"start_time\": 9.84,\n", + " \"end_time\": 9.92,\n", + " \"confidence\": 0.08197201788425446,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"what\",\n", + " \"start_time\": 10.08,\n", + " \"end_time\": 10.16,\n", + " \"confidence\": 0.24465684592723846,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"like\",\n", + " \"start_time\": 10.4,\n", + " \"end_time\": 10.48,\n", + " \"confidence\": 0.05114663019776344,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"a\",\n", + " \"start_time\": 10.56,\n", + " \"end_time\": 10.64,\n", + " \"confidence\": 0.4248896837234497,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"year\",\n", + " \"start_time\": 10.72,\n", + " \"end_time\": 10.8,\n", + " \"confidence\": 0.1509919911623001,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"ago\",\n", + " \"start_time\": 10.8,\n", + " \"end_time\": 11.04,\n", + " \"confidence\": 1.0,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"and\",\n", + " \"start_time\": 11.2,\n", + " \"end_time\": 11.28,\n", + " \"confidence\": 0.45389869809150696,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"at\",\n", + " \"start_time\": 11.44,\n", + " \"end_time\": 11.52,\n", + " \"confidence\": 0.13034537434577942,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"that\",\n", + " \"start_time\": 11.6,\n", + " \"end_time\": 11.68,\n", + " \"confidence\": 0.35941192507743835,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"time\",\n", + " \"start_time\": 11.76,\n", + " \"end_time\": 11.84,\n", + " \"confidence\": 0.6481927037239075,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"i\",\n", + " \"start_time\": 12.0,\n", + " \"end_time\": 12.08,\n", + " \"confidence\": 0.3444666862487793,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"think\",\n", + " \"start_time\": 12.08,\n", + " \"end_time\": 12.16,\n", + " \"confidence\": 0.35150519013404846,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"you\",\n", + " \"start_time\": 12.24,\n", + " \"end_time\": 12.32,\n", + " \"confidence\": 0.31789451837539673,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"were\",\n", + " \"start_time\": 12.4,\n", + " \"end_time\": 12.48,\n", + " \"confidence\": 0.23863451182842255,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"having\",\n", + " \"start_time\": 12.56,\n", + " \"end_time\": 12.64,\n", + " \"confidence\": 0.11462902277708054,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"some\",\n", + " \"start_time\": 12.72,\n", + " \"end_time\": 12.8,\n", + " \"confidence\": 0.26872554421424866,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"minor\",\n", + " \"start_time\": 12.96,\n", + " \"end_time\": 13.2,\n", + " \"confidence\": 0.18700699508190155,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"headaches\",\n", + " \"start_time\": 13.28,\n", + " \"end_time\": 13.84,\n", + " \"confidence\": 0.035322219133377075,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"i\",\n", + " \"start_time\": 14.48,\n", + " \"end_time\": 14.56,\n", + " \"confidence\": 0.18832509219646454,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"don't\",\n", + " \"start_time\": 14.56,\n", + " \"end_time\": 14.8,\n", + " \"confidence\": 0.5926258563995361,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"recall\",\n", + " \"start_time\": 14.8,\n", + " \"end_time\": 15.04,\n", + " \"confidence\": 0.2032231241464615,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"prescribing\",\n", + " \"start_time\": 15.2,\n", + " \"end_time\": 15.76,\n", + " \"confidence\": 0.0186734851449728,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"anything\",\n", + " \"start_time\": 15.84,\n", + " \"end_time\": 15.92,\n", + " \"confidence\": 0.05401425063610077,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"and\",\n", + " \"start_time\": 16.24,\n", + " \"end_time\": 16.32,\n", + " \"confidence\": 0.24042317271232605,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"we\",\n", + " \"start_time\": 16.72,\n", + " \"end_time\": 16.8,\n", + " \"confidence\": 0.24344421923160553,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"said\",\n", + " \"start_time\": 16.8,\n", + " \"end_time\": 16.88,\n", + " \"confidence\": 0.14940795302391052,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"we'd\",\n", + " \"start_time\": 17.04,\n", + " \"end_time\": 17.28,\n", + " \"confidence\": 0.37774398922920227,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"maintain\",\n", + " \"start_time\": 17.36,\n", + " \"end_time\": 17.6,\n", + " \"confidence\": 0.032487910240888596,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"some\",\n", + " \"start_time\": 17.68,\n", + " \"end_time\": 17.76,\n", + " \"confidence\": 0.17500576376914978,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"observations\",\n", + " \"start_time\": 17.84,\n", + " \"end_time\": 18.32,\n", + " \"confidence\": 0.09075526148080826,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"unless\",\n", + " \"start_time\": 18.56,\n", + " \"end_time\": 18.8,\n", + " \"confidence\": 0.07863186299800873,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"things\",\n", + " \"start_time\": 18.88,\n", + " \"end_time\": 18.96,\n", + " \"confidence\": 0.07724761217832565,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"were\",\n", + " \"start_time\": 19.12,\n", + " \"end_time\": 19.2,\n", + " \"confidence\": 0.21745437383651733,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"getting\",\n", + " \"start_time\": 19.28,\n", + " \"end_time\": 19.36,\n", + " \"confidence\": 0.17534911632537842,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"worse\",\n", + " \"start_time\": 19.52,\n", + " \"end_time\": 19.76,\n", + " \"confidence\": 0.32335954904556274,\n", + " \"speaker_tag\": 0\n", + " }\n", + " ]\n", + " }\n", + " ],\n", + " \"is_final\": true,\n", + " \"channel_tag\": 1\n", + " },\n", + " {\n", + " \"alternatives\": [\n", + " {\n", + " \"transcript\": \"That's right. Actually the headaches have gone away. I think getting more sleep was super helpful. I've also been more careful about my water intake throughout my work day. \",\n", + " \"confidence\": 0.16150188446044922,\n", + " \"words\": [\n", + " {\n", + " \"word\": \"That's\",\n", + " \"start_time\": 20.72,\n", + " \"end_time\": 20.96,\n", + " \"confidence\": 0.34903985261917114,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"right.\",\n", + " \"start_time\": 21.04,\n", + " \"end_time\": 21.12,\n", + " \"confidence\": 0.41917183995246887,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"Actually\",\n", + " \"start_time\": 21.92,\n", + " \"end_time\": 22.0,\n", + " \"confidence\": 0.04254724085330963,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"the\",\n", + " \"start_time\": 22.32,\n", + " \"end_time\": 22.4,\n", + " \"confidence\": 0.23515938222408295,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"headaches\",\n", + " \"start_time\": 22.56,\n", + " \"end_time\": 23.04,\n", + " \"confidence\": 0.027907712385058403,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"have\",\n", + " \"start_time\": 23.12,\n", + " \"end_time\": 23.2,\n", + " \"confidence\": 0.1091347336769104,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"gone\",\n", + " \"start_time\": 23.28,\n", + " \"end_time\": 23.44,\n", + " \"confidence\": 0.31499072909355164,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"away.\",\n", + " \"start_time\": 23.52,\n", + " \"end_time\": 23.6,\n", + " \"confidence\": 0.23496346175670624,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"I\",\n", + " \"start_time\": 24.0,\n", + " \"end_time\": 24.08,\n", + " \"confidence\": 0.22716930508613586,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"think\",\n", + " \"start_time\": 24.16,\n", + " \"end_time\": 24.24,\n", + " \"confidence\": 0.33145204186439514,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"getting\",\n", + " \"start_time\": 24.4,\n", + " \"end_time\": 24.48,\n", + " \"confidence\": 0.04049116373062134,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"more\",\n", + " \"start_time\": 24.64,\n", + " \"end_time\": 24.72,\n", + " \"confidence\": 0.19106054306030273,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"sleep\",\n", + " \"start_time\": 24.8,\n", + " \"end_time\": 25.12,\n", + " \"confidence\": 0.22222939133644104,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"was\",\n", + " \"start_time\": 25.2,\n", + " \"end_time\": 25.28,\n", + " \"confidence\": 0.14946062862873077,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"super\",\n", + " \"start_time\": 25.36,\n", + " \"end_time\": 25.44,\n", + " \"confidence\": 0.046468108892440796,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"helpful.\",\n", + " \"start_time\": 25.68,\n", + " \"end_time\": 25.92,\n", + " \"confidence\": 0.1827031821012497,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"I've\",\n", + " \"start_time\": 26.32,\n", + " \"end_time\": 26.56,\n", + " \"confidence\": 0.12086930125951767,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"also\",\n", + " \"start_time\": 26.64,\n", + " \"end_time\": 26.72,\n", + " \"confidence\": 0.1355309933423996,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"been\",\n", + " \"start_time\": 26.88,\n", + " \"end_time\": 26.96,\n", + " \"confidence\": 0.45278429985046387,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"more\",\n", + " \"start_time\": 27.04,\n", + " \"end_time\": 27.12,\n", + " \"confidence\": 0.12580135464668274,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"careful\",\n", + " \"start_time\": 27.2,\n", + " \"end_time\": 27.44,\n", + " \"confidence\": 0.3545719385147095,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"about\",\n", + " \"start_time\": 27.52,\n", + " \"end_time\": 27.6,\n", + " \"confidence\": 0.5137729644775391,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"my\",\n", + " \"start_time\": 27.76,\n", + " \"end_time\": 27.84,\n", + " \"confidence\": 0.1691187173128128,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"water\",\n", + " \"start_time\": 27.92,\n", + " \"end_time\": 28.0,\n", + " \"confidence\": 0.07359946519136429,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"intake\",\n", + " \"start_time\": 28.16,\n", + " \"end_time\": 28.48,\n", + " \"confidence\": 0.34989771246910095,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"throughout\",\n", + " \"start_time\": 28.56,\n", + " \"end_time\": 28.8,\n", + " \"confidence\": 0.10542239248752594,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"my\",\n", + " \"start_time\": 28.88,\n", + " \"end_time\": 28.96,\n", + " \"confidence\": 0.200903058052063,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"work\",\n", + " \"start_time\": 28.96,\n", + " \"end_time\": 29.04,\n", + " \"confidence\": 0.19961881637573242,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"day.\",\n", + " \"start_time\": 29.12,\n", + " \"end_time\": 29.2,\n", + " \"confidence\": 0.06420860439538956,\n", + " \"speaker_tag\": 1\n", + " }\n", + " ]\n", + " }\n", + " ],\n", + " \"is_final\": true,\n", + " \"channel_tag\": 1\n", + " },\n", + " {\n", + " \"alternatives\": [\n", + " {\n", + " \"transcript\": \"Yeah, I'm not surprised at all. Sleep deprivation and chronic dehydration are two common contributors to potential headaches. Rest is definitely vital when you become dehydrated. also, your brain tissue loses water, causing your brain to shrink and, you know, kind of pull away from the skull and this can trigger the pain receptors around the brain, giving you the sensation of a headache. So how much water are you roughly taking in each day? \",\n", + " \"confidence\": 0.1568450778722763,\n", + " \"words\": [\n", + " {\n", + " \"word\": \"Yeah,\",\n", + " \"start_time\": 29.92,\n", + " \"end_time\": 30.0,\n", + " \"confidence\": 0.2032550573348999,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"I'm\",\n", + " \"start_time\": 30.4,\n", + " \"end_time\": 30.64,\n", + " \"confidence\": 0.5164117217063904,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"not\",\n", + " \"start_time\": 30.64,\n", + " \"end_time\": 30.72,\n", + " \"confidence\": 0.45941776037216187,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"surprised\",\n", + " \"start_time\": 30.72,\n", + " \"end_time\": 31.2,\n", + " \"confidence\": 0.28761059045791626,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"at\",\n", + " \"start_time\": 31.2,\n", + " \"end_time\": 31.28,\n", + " \"confidence\": 0.4617224335670471,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"all.\",\n", + " \"start_time\": 31.28,\n", + " \"end_time\": 31.36,\n", + " \"confidence\": 0.2859147787094116,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"Sleep\",\n", + " \"start_time\": 31.44,\n", + " \"end_time\": 31.76,\n", + " \"confidence\": 0.06664705276489258,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"deprivation\",\n", + " \"start_time\": 31.76,\n", + " \"end_time\": 32.4,\n", + " \"confidence\": 0.3323269784450531,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"and\",\n", + " \"start_time\": 32.4,\n", + " \"end_time\": 32.48,\n", + " \"confidence\": 0.48908424377441406,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"chronic\",\n", + " \"start_time\": 32.48,\n", + " \"end_time\": 32.8,\n", + " \"confidence\": 0.05144517123699188,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"dehydration\",\n", + " \"start_time\": 32.8,\n", + " \"end_time\": 33.36,\n", + " \"confidence\": 0.053087785840034485,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"are\",\n", + " \"start_time\": 33.44,\n", + " \"end_time\": 33.52,\n", + " \"confidence\": 0.15498751401901245,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"two\",\n", + " \"start_time\": 33.52,\n", + " \"end_time\": 33.6,\n", + " \"confidence\": 0.11268683522939682,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"common\",\n", + " \"start_time\": 33.68,\n", + " \"end_time\": 33.92,\n", + " \"confidence\": 0.2483319491147995,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"contributors\",\n", + " \"start_time\": 34.0,\n", + " \"end_time\": 34.56,\n", + " \"confidence\": 0.03499847277998924,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"to\",\n", + " \"start_time\": 34.56,\n", + " \"end_time\": 34.64,\n", + " \"confidence\": 0.49667829275131226,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"potential\",\n", + " \"start_time\": 34.72,\n", + " \"end_time\": 34.8,\n", + " \"confidence\": 0.02955312840640545,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"headaches.\",\n", + " \"start_time\": 35.12,\n", + " \"end_time\": 35.6,\n", + " \"confidence\": 0.025746066123247147,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"Rest\",\n", + " \"start_time\": 36.08,\n", + " \"end_time\": 36.16,\n", + " \"confidence\": 0.026637624949216843,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"is\",\n", + " \"start_time\": 36.4,\n", + " \"end_time\": 36.48,\n", + " \"confidence\": 0.20806927978992462,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"definitely\",\n", + " \"start_time\": 36.64,\n", + " \"end_time\": 36.72,\n", + " \"confidence\": 0.06988045573234558,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"vital\",\n", + " \"start_time\": 37.04,\n", + " \"end_time\": 37.36,\n", + " \"confidence\": 0.08325270563364029,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"when\",\n", + " \"start_time\": 37.76,\n", + " \"end_time\": 37.84,\n", + " \"confidence\": 0.11506795883178711,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"you\",\n", + " \"start_time\": 37.84,\n", + " \"end_time\": 37.92,\n", + " \"confidence\": 0.4035564959049225,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"become\",\n", + " \"start_time\": 38.0,\n", + " \"end_time\": 38.08,\n", + " \"confidence\": 0.09846752882003784,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"dehydrated.\",\n", + " \"start_time\": 38.24,\n", + " \"end_time\": 38.88,\n", + " \"confidence\": 0.02675021067261696,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"also,\",\n", + " \"start_time\": 39.04,\n", + " \"end_time\": 39.12,\n", + " \"confidence\": 0.0819411426782608,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"your\",\n", + " \"start_time\": 39.2,\n", + " \"end_time\": 39.28,\n", + " \"confidence\": 0.08137592673301697,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"brain\",\n", + " \"start_time\": 39.36,\n", + " \"end_time\": 39.52,\n", + " \"confidence\": 0.3041834235191345,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"tissue\",\n", + " \"start_time\": 39.52,\n", + " \"end_time\": 40.08,\n", + " \"confidence\": 0.30725225806236267,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"loses\",\n", + " \"start_time\": 40.16,\n", + " \"end_time\": 40.4,\n", + " \"confidence\": 0.04215365648269653,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"water,\",\n", + " \"start_time\": 40.48,\n", + " \"end_time\": 40.56,\n", + " \"confidence\": 0.07522591203451157,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"causing\",\n", + " \"start_time\": 40.72,\n", + " \"end_time\": 40.96,\n", + " \"confidence\": 0.07779209315776825,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"your\",\n", + " \"start_time\": 41.04,\n", + " \"end_time\": 41.12,\n", + " \"confidence\": 0.1350143551826477,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"brain\",\n", + " \"start_time\": 41.2,\n", + " \"end_time\": 41.36,\n", + " \"confidence\": 0.23553147912025452,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"to\",\n", + " \"start_time\": 41.44,\n", + " \"end_time\": 41.52,\n", + " \"confidence\": 0.2863079607486725,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"shrink\",\n", + " \"start_time\": 41.52,\n", + " \"end_time\": 41.84,\n", + " \"confidence\": 0.07985348999500275,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"and,\",\n", + " \"start_time\": 41.84,\n", + " \"end_time\": 41.92,\n", + " \"confidence\": 0.4112929105758667,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"you\",\n", + " \"start_time\": 42.08,\n", + " \"end_time\": 42.16,\n", + " \"confidence\": 0.12933027744293213,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"know,\",\n", + " \"start_time\": 42.16,\n", + " \"end_time\": 42.24,\n", + " \"confidence\": 0.43177130818367004,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"kind\",\n", + " \"start_time\": 42.4,\n", + " \"end_time\": 42.48,\n", + " \"confidence\": 0.0974094420671463,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"of\",\n", + " \"start_time\": 42.48,\n", + " \"end_time\": 42.56,\n", + " \"confidence\": 0.7750440835952759,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"pull\",\n", + " \"start_time\": 42.64,\n", + " \"end_time\": 42.8,\n", + " \"confidence\": 0.09938612580299377,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"away\",\n", + " \"start_time\": 42.88,\n", + " \"end_time\": 42.96,\n", + " \"confidence\": 0.20952579379081726,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"from\",\n", + " \"start_time\": 43.04,\n", + " \"end_time\": 43.12,\n", + " \"confidence\": 0.7014749646186829,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"the\",\n", + " \"start_time\": 43.2,\n", + " \"end_time\": 43.28,\n", + " \"confidence\": 0.6510332226753235,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"skull\",\n", + " \"start_time\": 43.28,\n", + " \"end_time\": 43.68,\n", + " \"confidence\": 0.06580144912004471,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"and\",\n", + " \"start_time\": 43.92,\n", + " \"end_time\": 44.0,\n", + " \"confidence\": 0.5090440511703491,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"this\",\n", + " \"start_time\": 44.08,\n", + " \"end_time\": 44.16,\n", + " \"confidence\": 0.13317689299583435,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"can\",\n", + " \"start_time\": 44.24,\n", + " \"end_time\": 44.32,\n", + " \"confidence\": 0.16898459196090698,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"trigger\",\n", + " \"start_time\": 44.4,\n", + " \"end_time\": 44.72,\n", + " \"confidence\": 0.1593226045370102,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"the\",\n", + " \"start_time\": 44.8,\n", + " \"end_time\": 44.88,\n", + " \"confidence\": 0.490689754486084,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"pain\",\n", + " \"start_time\": 44.88,\n", + " \"end_time\": 45.04,\n", + " \"confidence\": 0.10243593156337738,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"receptors\",\n", + " \"start_time\": 45.12,\n", + " \"end_time\": 45.6,\n", + " \"confidence\": 0.1410498321056366,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"around\",\n", + " \"start_time\": 45.68,\n", + " \"end_time\": 45.76,\n", + " \"confidence\": 0.06033305451273918,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"the\",\n", + " \"start_time\": 45.92,\n", + " \"end_time\": 46.0,\n", + " \"confidence\": 0.561355471611023,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"brain,\",\n", + " \"start_time\": 46.0,\n", + " \"end_time\": 46.16,\n", + " \"confidence\": 0.16906583309173584,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"giving\",\n", + " \"start_time\": 46.32,\n", + " \"end_time\": 46.64,\n", + " \"confidence\": 0.08420229703187943,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"you\",\n", + " \"start_time\": 46.64,\n", + " \"end_time\": 46.72,\n", + " \"confidence\": 0.2749597132205963,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"the\",\n", + " \"start_time\": 46.88,\n", + " \"end_time\": 46.96,\n", + " \"confidence\": 0.49576857686042786,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"sensation\",\n", + " \"start_time\": 46.96,\n", + " \"end_time\": 47.36,\n", + " \"confidence\": 0.10295350849628448,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"of\",\n", + " \"start_time\": 47.44,\n", + " \"end_time\": 47.52,\n", + " \"confidence\": 0.7010328769683838,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"a\",\n", + " \"start_time\": 47.52,\n", + " \"end_time\": 47.6,\n", + " \"confidence\": 0.26516804099082947,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"headache.\",\n", + " \"start_time\": 47.76,\n", + " \"end_time\": 48.16,\n", + " \"confidence\": 0.09578469395637512,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"So\",\n", + " \"start_time\": 48.48,\n", + " \"end_time\": 48.56,\n", + " \"confidence\": 0.14987194538116455,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"how\",\n", + " \"start_time\": 48.72,\n", + " \"end_time\": 48.8,\n", + " \"confidence\": 0.12670308351516724,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"much\",\n", + " \"start_time\": 48.88,\n", + " \"end_time\": 48.96,\n", + " \"confidence\": 0.37054282426834106,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"water\",\n", + " \"start_time\": 49.04,\n", + " \"end_time\": 49.12,\n", + " \"confidence\": 0.13491025567054749,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"are\",\n", + " \"start_time\": 49.36,\n", + " \"end_time\": 49.44,\n", + " \"confidence\": 0.08922003954648972,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"you\",\n", + " \"start_time\": 49.52,\n", + " \"end_time\": 49.6,\n", + " \"confidence\": 0.2524241805076599,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"roughly\",\n", + " \"start_time\": 49.68,\n", + " \"end_time\": 50.0,\n", + " \"confidence\": 0.04612087085843086,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"taking\",\n", + " \"start_time\": 50.08,\n", + " \"end_time\": 50.16,\n", + " \"confidence\": 0.047547876834869385,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"in\",\n", + " \"start_time\": 51.2,\n", + " \"end_time\": 51.28,\n", + " \"confidence\": 0.2185513973236084,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"each\",\n", + " \"start_time\": 51.44,\n", + " \"end_time\": 51.52,\n", + " \"confidence\": 0.12173997610807419,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"day?\",\n", + " \"start_time\": 51.6,\n", + " \"end_time\": 51.68,\n", + " \"confidence\": 0.15826211869716644,\n", + " \"speaker_tag\": 0\n", + " }\n", + " ]\n", + " }\n", + " ],\n", + " \"is_final\": true,\n", + " \"channel_tag\": 1\n", + " },\n", + " {\n", + " \"alternatives\": [\n", + " {\n", + " \"transcript\": \"I've become obsessed with drinking enough water. I have one of those fancy water bottles that have graduated markers on the side. I've also been logging my water intake pretty regularly. On average, I drink about three liters a day. That's excellent. Before I start the routine physical exam, is there anything else you'd like me to know? Anything you'd like to share? What else has been bothering you? \",\n", + " \"confidence\": 0.19451096653938293,\n", + " \"words\": [\n", + " {\n", + " \"word\": \"I've\",\n", + " \"start_time\": 53.68,\n", + " \"end_time\": 53.92,\n", + " \"confidence\": 0.2077348232269287,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"become\",\n", + " \"start_time\": 54.0,\n", + " \"end_time\": 54.08,\n", + " \"confidence\": 0.10347001999616623,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"obsessed\",\n", + " \"start_time\": 54.24,\n", + " \"end_time\": 54.8,\n", + " \"confidence\": 0.1692890226840973,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"with\",\n", + " \"start_time\": 54.8,\n", + " \"end_time\": 54.88,\n", + " \"confidence\": 0.8168550729751587,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"drinking\",\n", + " \"start_time\": 54.96,\n", + " \"end_time\": 55.28,\n", + " \"confidence\": 0.06928665190935135,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"enough\",\n", + " \"start_time\": 55.36,\n", + " \"end_time\": 55.44,\n", + " \"confidence\": 0.04279853776097298,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"water.\",\n", + " \"start_time\": 55.6,\n", + " \"end_time\": 55.68,\n", + " \"confidence\": 0.2244122326374054,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"I\",\n", + " \"start_time\": 56.16,\n", + " \"end_time\": 56.24,\n", + " \"confidence\": 0.11956850439310074,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"have\",\n", + " \"start_time\": 56.4,\n", + " \"end_time\": 56.48,\n", + " \"confidence\": 0.3316974341869354,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"one\",\n", + " \"start_time\": 56.56,\n", + " \"end_time\": 56.64,\n", + " \"confidence\": 0.2192472666501999,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"of\",\n", + " \"start_time\": 56.72,\n", + " \"end_time\": 56.8,\n", + " \"confidence\": 0.4900609850883484,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"those\",\n", + " \"start_time\": 56.88,\n", + " \"end_time\": 56.96,\n", + " \"confidence\": 0.4028575122356415,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"fancy\",\n", + " \"start_time\": 57.12,\n", + " \"end_time\": 57.44,\n", + " \"confidence\": 0.09906727075576782,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"water\",\n", + " \"start_time\": 57.6,\n", + " \"end_time\": 57.68,\n", + " \"confidence\": 0.049076370894908905,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"bottles\",\n", + " \"start_time\": 58.0,\n", + " \"end_time\": 58.48,\n", + " \"confidence\": 0.2664627134799957,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"that\",\n", + " \"start_time\": 58.48,\n", + " \"end_time\": 58.56,\n", + " \"confidence\": 0.22570614516735077,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"have\",\n", + " \"start_time\": 58.72,\n", + " \"end_time\": 58.8,\n", + " \"confidence\": 0.30251458287239075,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"graduated\",\n", + " \"start_time\": 58.88,\n", + " \"end_time\": 59.44,\n", + " \"confidence\": 0.139690563082695,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"markers\",\n", + " \"start_time\": 59.6,\n", + " \"end_time\": 59.92,\n", + " \"confidence\": 0.023367125540971756,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"on\",\n", + " \"start_time\": 60.0,\n", + " \"end_time\": 60.08,\n", + " \"confidence\": 0.26634448766708374,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"the\",\n", + " \"start_time\": 60.16,\n", + " \"end_time\": 60.24,\n", + " \"confidence\": 0.5959072709083557,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"side.\",\n", + " \"start_time\": 60.4,\n", + " \"end_time\": 60.48,\n", + " \"confidence\": 0.17520837485790253,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"I've\",\n", + " \"start_time\": 61.04,\n", + " \"end_time\": 61.28,\n", + " \"confidence\": 0.0869181752204895,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"also\",\n", + " \"start_time\": 61.36,\n", + " \"end_time\": 61.44,\n", + " \"confidence\": 0.12544651329517365,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"been\",\n", + " \"start_time\": 61.52,\n", + " \"end_time\": 61.6,\n", + " \"confidence\": 0.48382851481437683,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"logging\",\n", + " \"start_time\": 61.68,\n", + " \"end_time\": 62.16,\n", + " \"confidence\": 0.036188967525959015,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"my\",\n", + " \"start_time\": 62.24,\n", + " \"end_time\": 62.32,\n", + " \"confidence\": 0.06913705170154572,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"water\",\n", + " \"start_time\": 62.32,\n", + " \"end_time\": 62.4,\n", + " \"confidence\": 0.06303171813488007,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"intake\",\n", + " \"start_time\": 62.64,\n", + " \"end_time\": 62.88,\n", + " \"confidence\": 0.3499637842178345,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"pretty\",\n", + " \"start_time\": 63.04,\n", + " \"end_time\": 63.12,\n", + " \"confidence\": 0.07672327011823654,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"regularly.\",\n", + " \"start_time\": 63.28,\n", + " \"end_time\": 63.76,\n", + " \"confidence\": 0.2083297073841095,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"On\",\n", + " \"start_time\": 63.92,\n", + " \"end_time\": 64.0,\n", + " \"confidence\": 0.17861735820770264,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"average,\",\n", + " \"start_time\": 64.08,\n", + " \"end_time\": 64.4,\n", + " \"confidence\": 0.20065191388130188,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"I\",\n", + " \"start_time\": 64.48,\n", + " \"end_time\": 64.56,\n", + " \"confidence\": 0.16855645179748535,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"drink\",\n", + " \"start_time\": 64.64,\n", + " \"end_time\": 64.88,\n", + " \"confidence\": 0.12234321236610413,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"about\",\n", + " \"start_time\": 64.96,\n", + " \"end_time\": 65.04,\n", + " \"confidence\": 0.09020872414112091,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"three\",\n", + " \"start_time\": 65.12,\n", + " \"end_time\": 65.2,\n", + " \"confidence\": 0.15803150832653046,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"liters\",\n", + " \"start_time\": 65.36,\n", + " \"end_time\": 65.76,\n", + " \"confidence\": 0.1863507628440857,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"a\",\n", + " \"start_time\": 65.84,\n", + " \"end_time\": 65.92,\n", + " \"confidence\": 0.3782702088356018,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"day.\",\n", + " \"start_time\": 65.92,\n", + " \"end_time\": 66.0,\n", + " \"confidence\": 0.5370899438858032,\n", + " \"speaker_tag\": 1\n", + " },\n", + " {\n", + " \"word\": \"That's\",\n", + " \"start_time\": 66.56,\n", + " \"end_time\": 66.8,\n", + " \"confidence\": 0.2986062169075012,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"excellent.\",\n", + " \"start_time\": 66.8,\n", + " \"end_time\": 67.12,\n", + " \"confidence\": 0.10423046350479126,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"Before\",\n", + " \"start_time\": 67.52,\n", + " \"end_time\": 67.6,\n", + " \"confidence\": 0.06178278103470802,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"I\",\n", + " \"start_time\": 67.68,\n", + " \"end_time\": 67.76,\n", + " \"confidence\": 0.32214897871017456,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"start\",\n", + " \"start_time\": 67.92,\n", + " \"end_time\": 68.0,\n", + " \"confidence\": 0.2561351954936981,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"the\",\n", + " \"start_time\": 68.08,\n", + " \"end_time\": 68.16,\n", + " \"confidence\": 0.24796715378761292,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"routine\",\n", + " \"start_time\": 68.24,\n", + " \"end_time\": 68.48,\n", + " \"confidence\": 0.055616796016693115,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"physical\",\n", + " \"start_time\": 68.64,\n", + " \"end_time\": 68.96,\n", + " \"confidence\": 0.21787214279174805,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"exam,\",\n", + " \"start_time\": 69.04,\n", + " \"end_time\": 69.28,\n", + " \"confidence\": 0.5982194542884827,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"is\",\n", + " \"start_time\": 69.44,\n", + " \"end_time\": 69.52,\n", + " \"confidence\": 0.30068957805633545,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"there\",\n", + " \"start_time\": 69.6,\n", + " \"end_time\": 69.68,\n", + " \"confidence\": 0.1378926783800125,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"anything\",\n", + " \"start_time\": 69.76,\n", + " \"end_time\": 69.84,\n", + " \"confidence\": 0.4075137972831726,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"else\",\n", + " \"start_time\": 70.0,\n", + " \"end_time\": 70.08,\n", + " \"confidence\": 0.5971935987472534,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"you'd\",\n", + " \"start_time\": 70.24,\n", + " \"end_time\": 70.56,\n", + " \"confidence\": 0.3605367839336395,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"like\",\n", + " \"start_time\": 70.56,\n", + " \"end_time\": 70.64,\n", + " \"confidence\": 0.7009704113006592,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"me\",\n", + " \"start_time\": 70.72,\n", + " \"end_time\": 70.8,\n", + " \"confidence\": 0.2830003798007965,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"to\",\n", + " \"start_time\": 70.88,\n", + " \"end_time\": 70.96,\n", + " \"confidence\": 0.458041250705719,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"know?\",\n", + " \"start_time\": 70.96,\n", + " \"end_time\": 71.04,\n", + " \"confidence\": 0.1438504159450531,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"Anything\",\n", + " \"start_time\": 71.28,\n", + " \"end_time\": 71.36,\n", + " \"confidence\": 0.14030654728412628,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"you'd\",\n", + " \"start_time\": 71.52,\n", + " \"end_time\": 71.84,\n", + " \"confidence\": 0.10724283009767532,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"like\",\n", + " \"start_time\": 71.84,\n", + " \"end_time\": 71.92,\n", + " \"confidence\": 0.5293093919754028,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"to\",\n", + " \"start_time\": 72.0,\n", + " \"end_time\": 72.08,\n", + " \"confidence\": 0.8176157474517822,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"share?\",\n", + " \"start_time\": 72.08,\n", + " \"end_time\": 72.16,\n", + " \"confidence\": 0.1634613573551178,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"What\",\n", + " \"start_time\": 72.32,\n", + " \"end_time\": 72.4,\n", + " \"confidence\": 0.17356321215629578,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"else\",\n", + " \"start_time\": 72.48,\n", + " \"end_time\": 72.56,\n", + " \"confidence\": 0.17788664996623993,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"has\",\n", + " \"start_time\": 72.72,\n", + " \"end_time\": 72.8,\n", + " \"confidence\": 0.17658747732639313,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"been\",\n", + " \"start_time\": 72.88,\n", + " \"end_time\": 72.96,\n", + " \"confidence\": 0.47056394815444946,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"bothering\",\n", + " \"start_time\": 73.04,\n", + " \"end_time\": 73.36,\n", + " \"confidence\": 0.12016564607620239,\n", + " \"speaker_tag\": 0\n", + " },\n", + " {\n", + " \"word\": \"you?\",\n", + " \"start_time\": 73.44,\n", + " \"end_time\": 73.52,\n", + " \"confidence\": 0.3841295838356018,\n", + " \"speaker_tag\": 0\n", + " }\n", + " ]\n", + " }\n", + " ],\n", + " \"is_final\": true,\n", + " \"channel_tag\": 1\n", + " }\n", + " ],\n", + " \"model_version\": \"parakeet-1-1b-ctc-en-us\"\n", + " }\n", + " ]\n", + "}\n" + ] + } + ], + "source": [ + "# Test 3: Force gRPC route with speaker diarization\n", + "def test_endpoint_grpc_route(audio_file_path, enable_diarization=True, max_speakers=4):\n", + " \"\"\"Test endpoint with forced gRPC route and speaker diarization\"\"\"\n", + " print(f\"Testing gRPC route with speaker diarization: {enable_diarization}\")\n", + " \n", + " try:\n", + " # Read audio file\n", + " with open(audio_file_path, 'rb') as f:\n", + " audio_data = f.read()\n", + " \n", + " # Create multipart form data with diarization parameters\n", + " boundary = f\"----WebKitFormBoundary{uuid.uuid4().hex}\"\n", + " \n", + " # Build multipart payload with additional parameters\n", + " payload_parts = []\n", + " \n", + " # Audio file\n", + " payload_parts.append(f'--{boundary}')\n", + " payload_parts.append('Content-Disposition: form-data; name=\"file\"; filename=\"test.wav\"')\n", + " payload_parts.append('Content-Type: audio/wav')\n", + " payload_parts.append('')\n", + " \n", + " text_part = '\\r\\n'.join(payload_parts) + '\\r\\n'\n", + " \n", + " # Additional parameters\n", + " additional_params = []\n", + " \n", + " # Language code\n", + " additional_params.append(f'--{boundary}')\n", + " additional_params.append('Content-Disposition: form-data; name=\"language_code\"')\n", + " additional_params.append('')\n", + " additional_params.append('en-US')\n", + " \n", + " # Speaker diarization\n", + " additional_params.append(f'--{boundary}')\n", + " additional_params.append('Content-Disposition: form-data; name=\"speaker_diarization\"')\n", + " additional_params.append('')\n", + " additional_params.append('true' if enable_diarization else 'false')\n", + " \n", + " # Max speakers\n", + " additional_params.append(f'--{boundary}')\n", + " additional_params.append('Content-Disposition: form-data; name=\"max_speakers\"')\n", + " additional_params.append('')\n", + " additional_params.append(str(max_speakers))\n", + " \n", + " # Close boundary\n", + " additional_params.append(f'--{boundary}--')\n", + " \n", + " params_part = '\\r\\n'.join(additional_params)\n", + " \n", + " # Combine all parts\n", + " payload = text_part.encode() + audio_data + ('\\r\\n' + params_part).encode()\n", + " \n", + " # Invoke endpoint with gRPC route forced\n", + " response = sm_runtime.invoke_endpoint(\n", + " EndpointName=endpoint_name,\n", + " ContentType=f'multipart/form-data; boundary={boundary}',\n", + " Body=payload,\n", + " CustomAttributes='/invocations/grpc' # Force gRPC route\n", + " )\n", + " \n", + " result = json.loads(response['Body'].read().decode())\n", + " print(f\"\\ngRPC route inference successful!\")\n", + " print(f\"Speaker diarization enabled: {enable_diarization}\")\n", + " print(f\"Max speakers: {max_speakers}\")\n", + " print(f\"Response: {json.dumps(result, indent=2)}\")\n", + " return result\n", + " \n", + " except Exception as e:\n", + " print(f\"gRPC route test failed: {e}\")\n", + " return None\n", + "\n", + "# Run gRPC route test\n", + "grpc_result = test_endpoint_grpc_route(\"data/medical-diarization.wav\")" + ] + }, + { + "cell_type": "markdown", + "id": "a19063f6-b6c0-4de2-a193-e482f26f7406", + "metadata": {}, + "source": [ + "### Terminate endpoint and clean up artifacts" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "e5db083f-4705-4c68-a488-f82da961be4b", + "metadata": { + "execution": { + "iopub.execute_input": "2025-10-22T23:57:34.724389Z", + "iopub.status.busy": "2025-10-22T23:57:34.724136Z", + "iopub.status.idle": "2025-10-22T23:57:35.290269Z", + "shell.execute_reply": "2025-10-22T23:57:35.289820Z", + "shell.execute_reply.started": "2025-10-22T23:57:34.724370Z" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "{'ResponseMetadata': {'RequestId': '4edda3cb-3515-401f-a268-c1ab99fef3ef',\n", + " 'HTTPStatusCode': 200,\n", + " 'HTTPHeaders': {'x-amzn-requestid': '4edda3cb-3515-401f-a268-c1ab99fef3ef',\n", + " 'strict-transport-security': 'max-age=47304000; includeSubDomains',\n", + " 'x-frame-options': 'DENY',\n", + " 'content-security-policy': \"frame-ancestors 'none'\",\n", + " 'cache-control': 'no-cache, no-store, must-revalidate',\n", + " 'x-content-type-options': 'nosniff',\n", + " 'content-type': 'application/x-amz-json-1.1',\n", + " 'date': 'Wed, 22 Oct 2025 23:57:35 GMT',\n", + " 'content-length': '0'},\n", + " 'RetryAttempts': 0}}" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sm.delete_model(ModelName=sm_model_name)\n", + "sm.delete_endpoint_config(EndpointConfigName=endpoint_config_name)\n", + "sm.delete_endpoint(EndpointName=endpoint_name)" + ] + }, + { + "cell_type": "markdown", + "id": "469807ed-9208-49ae-bab6-2498226cda9c", + "metadata": {}, + "source": [ + "**🎉 Congratulations!** You have successfully deployed a production-ready NVIDIA NIM ASR solution on Amazon SageMaker" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c646d658-0c75-45d6-b67c-1a802f7cbc19", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.9" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}