From 57df621f6577136f1676c770076a1ea31309d776 Mon Sep 17 00:00:00 2001 From: Zachary Hoppinen Date: Wed, 14 Jan 2026 12:13:10 -0700 Subject: [PATCH 01/11] adding in ReadMe clarifications and edits from my first hyp3 deployment --- README.md | 71 ++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 47 insertions(+), 24 deletions(-) diff --git a/README.md b/README.md index 1ab2619ea..89093f099 100644 --- a/README.md +++ b/README.md @@ -146,7 +146,9 @@ Once the `github-actions` IAM user has been created, you can create an AWS acces which we will use to deploy HyP3 via CI/CD tooling: 1. Go to AWS console -> IAM -> Users -> github-actions -> security credentials tab -> "create access key". -2. Store the access key ID and secret access key using your team's password manager. +2. Select "Other" for key usage +3. Add tag value to describe the key such as `hyp3-foobar-access` +4. Store the access key ID and secret access key using your team's password manager. You will use them below in "Create the GitHub environment"
@@ -223,13 +225,13 @@ you will need to create an Earthdata Login user for your deployment if you do no Go to AWS console -> Secrets Manager, then: 1. Click the orange "Store a new secret" button -1. For "Secret Type" select "Other type of secret" -1. Enter all required secret key-value pairs. Notably, the keys should be the secret names as listed (case-sensitive) in the [job specs](./job_spec/) that will be deployed -1. Click the orange "Next" button -1. Give the secret the same name that you plan to give to the HyP3 CloudFormation stack when you deploy it (below) -1. Click the orange "Next" button -1. Click the orange "Next" button (we won't configure rotation) -1. Click the orange "Store" button to save the Secret +2. For "Secret Type" select "Other type of secret" +3. Enter all required secret key-value pairs. Notably, the keys should be the secret names as listed (case-sensitive) in the [job specs](./job_spec/) that will be deployed +4. Click the orange "Next" button +5. Give the secret the same name that you plan to give to the HyP3 CloudFormation stack when you deploy it (below) +6. Click the orange "Next" button +7. Click the orange "Next" button (we won't configure rotation) +8. Click the orange "Store" button to save the Secret #### Request SSL cert @@ -239,13 +241,16 @@ even if you're deploying HyP3 to `us-west-2`.* To allow HTTPS connections, HyP3 needs an SSL certificate that is valid for its deployment domain name (URL): -AWS console -> AWS Certificate Manager -> Request a public certificate: -- The domain name should be something like `hyp3-foobar.asf.alaska.edu` -- Choose "DNS validation" +AWS console -> AWS Certificate Manager -> Request certificate:\ +1. Select "Request a public certificate" +2. Click the orange "Next" button +3. Choose a "Fully qualified domain name". Domain name should be something like `hyp3-foobar.asf.alaska.edu` or for a test deployment `hyp3-foobar-test.asf.alaska.edu`. +3. Choose "DNS validation" +4. Copy the "CNAME name" and "CNAME value" -Then add the validation record to +Then copy past the add the validation record to a row of https://gitlab.asf.alaska.edu/operations/puppet/-/edit/production/modules/legacy_dns/files/asf.alaska.edu.db -(see previous records for examples). +in the format ` in CNAME .` (see previous records for examples). ### Create the GitHub environment @@ -254,29 +259,30 @@ https://gitlab.asf.alaska.edu/operations/puppet/-/edit/production/modules/legacy > repository, which is generally only possible for ASF employees on HyP3 development teams. 1. Go to https://github.com/ASFHyP3/hyp3/settings/environments -> New Environment -2. Check "required reviewers" and add the appropriate team(s) or user(s) -3. Change "Deployment branches and tags" to "Selected branches and tags" and +2. Name the environment like your chosen domain name i.e. `hyp3-foobar` or `hyp3-foobar-test` +3. Check "required reviewers" and add the appropriate team(s) or user(s) +4. Change "Deployment branches and tags" to "Selected branches and tags" and - add a deployment branch or tag rule - use "Ref Type: Branch" and write the name of the branch it will be deploying out of. (This is typically `main` for prod deployments, `develop` for test deployments, or a feature branch name for sandbox deployments.) -4. Add the following environment secrets: +5. Add the following environment secrets: - `AWS_REGION` - e.g. `us-west-2` - - `CERTIFICATE_ARN` (ASF and JPL only) - ARN of the AWS Certificate Manager certificate that you created manually + - `CERTIFICATE_ARN` (ASF and JPL only) - ARN of the AWS Certificate Manager certificate that you created manually, e.g. `arn:aws:acm:us-west-2:XXXXXXXXXXXX:certificate/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX` - `CLOUDFORMATION_ROLE_ARN` (ASF only) - part of the `hyp3-ci` stack that you deployed, e.g. `arn:aws:iam::xxxxxxxxxxxx:role/hyp3-ci-CloudformationDeploymentRole-XXXXXXXXXXXXX` - - `SECRET_ARN` - ARN for the AWS Secrets Manager Secret that you created manually + - `SECRET_ARN` - ARN for the AWS Secrets Manager Secret that you created manually, e.g. `arn:aws:secretsmanager:us-west-X:XXXXXXXXXXXX:secret:hyp3-foobar-XXXXXX` - `V2_AWS_ACCESS_KEY_ID` - AWS access key ID: - - ASF: for the `github-actions` user + - ASF: for the `github-actions` user (created in step "Enable CI/CD above") - JPL: for the service user - EDC: created by an ASF developer via Kion - `V2_AWS_SECRET_ACCESS_KEY` - The corresponding secret access key - - `VPC_ID` - ID of the default VPC for this AWS account and region (aws console -> vpc -> your VPCs, e.g. `vpc-xxxxxxxxxxxxxxxxx`) - - `SUBNET_IDS` - Comma delimited list (no spaces) of the default subnets for the VPC specified in `VPC_ID` (aws console -> vpc -> subnets, e.g. `subnet-xxxxxxxxxxxxxxxxx,subnet-xxxxxxxxxxxxxxxxx,subnet-xxxxxxxxxxxxxxxxx,subnet-xxxxxxxxxxxxxxxxx`) + - `VPC_ID` - ID of the default VPC for this AWS account and region (aws console -> VPC -> Your VPCs, e.g. `vpc-xxxxxxxxxxxxxxxxx`) + - `SUBNET_IDS` - Comma delimited list (no spaces) of the default subnets for the VPC specified in `VPC_ID` (aws console -> VPC -> Subnets, e.g. `subnet-xxxxxxxxxxxxxxxxx,subnet-xxxxxxxxxxxxxxxxx,subnet-xxxxxxxxxxxxxxxxx,subnet-xxxxxxxxxxxxxxxxx`) ### Create the HyP3 deployment -You will need to add the deployment to the matrix in an existing GitHub Actions `deploy-*.yml` workflow or create +You will need to add the deployment to the matrix in an existing GitHub Actions `deploy-*.yml` workflow (.github/workflows/) or create a new one for the deployment. If you need to create a new one, we recommend copying one of the -existing workflows, and then updating all of the fields +existing workflows, and then updating all of the fields. as appropriate for your deployment. Also make sure to update the top-level `name` of the workflow and the name of the branch to deploy from. (This is typically `main` for prod deployments, `develop` for test deployments, or a feature branch name for sandbox deployments.) @@ -284,6 +290,9 @@ of the branch to deploy from. (This is typically `main` for prod deployments, `d > If you're deploying from a feature branch, make sure to [protect](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches) > it from accidental deletion. +> [!TIP] +> If your CI/CD workflow fails. Delete the "Rolled Back" stack (AWS Manager -> CloudFormation -> Stacks) before re-running the failed job. + The deployment workflow will run as soon as you merge your changes into the branch specified in the workflow file. ### Finishing touches @@ -296,7 +305,8 @@ Once HyP3 is deployed, there are a few follow on tasks you may need to do for a > This step must be done by an ASF employee. Open a PR adding a line to https://gitlab.asf.alaska.edu/operations/puppet/-/blob/production/modules/legacy_dns/files/asf.alaska.edu.db -for the new custom domain name (AWS console -> api gateway -> custom domain names -> "API Gateway domain name"). +for the new custom domain name (AWS console -> api gateway -> custom domain names -> "API Gateway domain name") of the format +`hyp3-foobar in CNAME .`. Follow similar examples. Ask someone from ASF support to review/merge the PR. @@ -315,6 +325,19 @@ Update the [AWS Accounts and HyP3 Deployments](https://docs.google.com/spreadshe > ``` > Remember to remove this after the DNS PR is merged! +#### Testing and adding user credits to your hyp3 deployment + +After successful deployment and DNS record merging you can test you deployment by accessing the Swagger UI and using the POST /user tab to +check if your user is approved and has credits for running jobs on the deployment. This should automatically add your user to the DynamoDB +table. + +AWS console -> DynamoDB -> Explore items +1. Find the table with format like `hyp3-foobar-UsersTable-XXXXXXXXXXXXX` +2. Edit your user (only present after using the Swagger UI in some way) +3. Add credits if neccessary + +You can then return the Swagger UI and use the POST /jobs to run a test job and confirm it completes. + #### Optional
From 1612c5b30bec416f04a315ea17a50b4dacbb0d5d Mon Sep 17 00:00:00 2001 From: Zachary Hoppinen Date: Wed, 14 Jan 2026 12:29:00 -0700 Subject: [PATCH 02/11] add in attempt at motivation statement --- README.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 89093f099..976387262 100644 --- a/README.md +++ b/README.md @@ -43,6 +43,10 @@ A processing environment for HyP3 Plugins in AWS. ## Deployment +### Why would you set up a hyp3 deployment? + +A HyP3 deployment stack provides a reproducible cloud processing environment that bundles AWS infrastructure, execution logic, and cost controls, enabling scalable, on-demand computation with clear operational costs. + > [!IMPORTANT] > It's not currently possible to deploy HyP3 fully independent of ASF due to our integration with > [ASF Vertex](https://search.alaska.edu). If you'd like your own deployment of HyP3, please open an issue here or @@ -126,7 +130,7 @@ In order to integrate an ASF deployment we'll need: These can be done by deploying the [ASF CI stack](cicd-stacks/ASF-deployment-ci-cf.yml). -*Warning: This stack should only be deployed once per AWS account. This stack also +*Warning: This stack only needs to be deployed once per AWS account. This stack also assumes you are only deploying into a single AWS Region. If you are deploying into multiple regions in the same AWS account, you'll need to adjust the IAM permissions that are limited to a single region.* @@ -149,6 +153,7 @@ which we will use to deploy HyP3 via CI/CD tooling: 2. Select "Other" for key usage 3. Add tag value to describe the key such as `hyp3-foobar-access` 4. Store the access key ID and secret access key using your team's password manager. You will use them below in "Create the GitHub environment" + as `V2_AWS_ACCESS_KEY_ID` and `V2_AWS_SECRET_ACCESS_KEY`.
From 9481980b8db883cbffea7a9804637744425fc1cd Mon Sep 17 00:00:00 2001 From: Zachary Hoppinen Date: Wed, 14 Jan 2026 12:30:27 -0700 Subject: [PATCH 03/11] changelog of readme edits --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index b984027be..520cd999b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - Increased the maximum `stride_for_norm_param_estimation` to 32 from 16 for the `OPERA_DIST_S1` job specification to handle models with a larger input size. +- Changed readme to reflect current state of AWS, motivation for deploying a hyp3 stack, and clarify naming and formatting. ## [10.12.1] From a9b751478dbe5fd4b3812b2f96521c260f0014d9 Mon Sep 17 00:00:00 2001 From: Zachary Hoppinen <56941602+ZachHoppinen@users.noreply.github.com> Date: Mon, 26 Jan 2026 11:06:37 -0700 Subject: [PATCH 04/11] Update README.md Co-authored-by: Joseph H Kennedy --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 976387262..ba822a6ee 100644 --- a/README.md +++ b/README.md @@ -151,7 +151,7 @@ which we will use to deploy HyP3 via CI/CD tooling: 1. Go to AWS console -> IAM -> Users -> github-actions -> security credentials tab -> "create access key". 2. Select "Other" for key usage -3. Add tag value to describe the key such as `hyp3-foobar-access` +3. (Optional)Add tag value to describe the key, such as "For GitHub Actions CI/CD pipelines" 4. Store the access key ID and secret access key using your team's password manager. You will use them below in "Create the GitHub environment" as `V2_AWS_ACCESS_KEY_ID` and `V2_AWS_SECRET_ACCESS_KEY`.
From 7da9ef237166c281b01f5c78faeb76423a4e77e3 Mon Sep 17 00:00:00 2001 From: Zachary Hoppinen <56941602+ZachHoppinen@users.noreply.github.com> Date: Mon, 26 Jan 2026 11:06:52 -0700 Subject: [PATCH 05/11] Update README.md Co-authored-by: Joseph H Kennedy --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index ba822a6ee..ed3ca07a4 100644 --- a/README.md +++ b/README.md @@ -285,7 +285,7 @@ in the format ` in CNAME .` (see previous records for ### Create the HyP3 deployment -You will need to add the deployment to the matrix in an existing GitHub Actions `deploy-*.yml` workflow (.github/workflows/) or create +You will need to add the deployment to the matrix in an existing GitHub Actions `deploy-*.yml` workflow located in the `.github/workflows/` directory, or create a new one for the deployment. If you need to create a new one, we recommend copying one of the existing workflows, and then updating all of the fields. as appropriate for your deployment. Also make sure to update the top-level `name` of the workflow and the name From 94a2dcc3e354fcf51402d969fdec9eef5a446c12 Mon Sep 17 00:00:00 2001 From: Zachary Hoppinen <56941602+ZachHoppinen@users.noreply.github.com> Date: Mon, 26 Jan 2026 11:07:30 -0700 Subject: [PATCH 06/11] Update README.md Co-authored-by: Joseph H Kennedy --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index ed3ca07a4..e88bc1f07 100644 --- a/README.md +++ b/README.md @@ -287,7 +287,7 @@ in the format ` in CNAME .` (see previous records for You will need to add the deployment to the matrix in an existing GitHub Actions `deploy-*.yml` workflow located in the `.github/workflows/` directory, or create a new one for the deployment. If you need to create a new one, we recommend copying one of the -existing workflows, and then updating all of the fields. +existing workflows, and then updating all of the fields as appropriate for your deployment. Also make sure to update the top-level `name` of the workflow and the name of the branch to deploy from. (This is typically `main` for prod deployments, `develop` for test deployments, or a feature branch name for sandbox deployments.) From d3e8dc75bae3c33efc98ab827be9183a803e78cf Mon Sep 17 00:00:00 2001 From: Zachary Hoppinen <56941602+ZachHoppinen@users.noreply.github.com> Date: Mon, 26 Jan 2026 11:07:37 -0700 Subject: [PATCH 07/11] Update README.md Co-authored-by: Joseph H Kennedy --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e88bc1f07..5a273cc6f 100644 --- a/README.md +++ b/README.md @@ -332,7 +332,7 @@ Update the [AWS Accounts and HyP3 Deployments](https://docs.google.com/spreadshe #### Testing and adding user credits to your hyp3 deployment -After successful deployment and DNS record merging you can test you deployment by accessing the Swagger UI and using the POST /user tab to +After successfully deploying HyP3 and your new DNS record has taken effect (or you've edited your local DNS name resolution), you can test your deployment by accessing the Swagger UI and using the POST `/user` tab to check if your user is approved and has credits for running jobs on the deployment. This should automatically add your user to the DynamoDB table. From 467193cb02ec80755ecb8a837ab57013dd4b5278 Mon Sep 17 00:00:00 2001 From: Zachary Hoppinen <56941602+ZachHoppinen@users.noreply.github.com> Date: Mon, 26 Jan 2026 11:07:43 -0700 Subject: [PATCH 08/11] Update README.md Co-authored-by: Joseph H Kennedy --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5a273cc6f..d8438b00e 100644 --- a/README.md +++ b/README.md @@ -341,7 +341,7 @@ AWS console -> DynamoDB -> Explore items 2. Edit your user (only present after using the Swagger UI in some way) 3. Add credits if neccessary -You can then return the Swagger UI and use the POST /jobs to run a test job and confirm it completes. +You can then return the Swagger UI and use the POST `/jobs` to run a test job and confirm it completes. #### Optional From ad639bdded4609a8b243ab4c68c0e930b52e2c3f Mon Sep 17 00:00:00 2001 From: Zachary Hoppinen <56941602+ZachHoppinen@users.noreply.github.com> Date: Mon, 26 Jan 2026 11:08:08 -0700 Subject: [PATCH 09/11] Update README.md Co-authored-by: Joseph H Kennedy --- README.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index d8438b00e..cc985d248 100644 --- a/README.md +++ b/README.md @@ -333,13 +333,11 @@ Update the [AWS Accounts and HyP3 Deployments](https://docs.google.com/spreadshe #### Testing and adding user credits to your hyp3 deployment After successfully deploying HyP3 and your new DNS record has taken effect (or you've edited your local DNS name resolution), you can test your deployment by accessing the Swagger UI and using the POST `/user` tab to -check if your user is approved and has credits for running jobs on the deployment. This should automatically add your user to the DynamoDB -table. +check if your user is approved and has credits for running jobs on the deployment. You will need to be authenticated by either providing an Earthdata Login Bearer Token using the "Authorize" button, or by having a valid `asf-urs` cookie, typically logging into [Vertex](https://search.asf.alaska.edu). Interacting with HyP3 should automatically add your user to the DynamoDB table with the default number of credits (typically 0). -AWS console -> DynamoDB -> Explore items -1. Find the table with format like `hyp3-foobar-UsersTable-XXXXXXXXXXXXX` -2. Edit your user (only present after using the Swagger UI in some way) -3. Add credits if neccessary +To add credits to your (or any) user, log in to the AWS console and navigate to DynamoDB -> Explore items, then: +1. Find the table with a format like `hyp3-foobar-UsersTable-XXXXXXXXXXXXX` +2. Edit your user record (only present after using the Swagger UI in some way) You can then return the Swagger UI and use the POST `/jobs` to run a test job and confirm it completes. From 9bcab8b2e305af2edc2fbcaf1c13a8b9765cdb10 Mon Sep 17 00:00:00 2001 From: Zachary Hoppinen Date: Mon, 26 Jan 2026 11:10:45 -0700 Subject: [PATCH 10/11] addressing Joe's last comment --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index cc985d248..a4c4c4346 100644 --- a/README.md +++ b/README.md @@ -253,9 +253,9 @@ AWS console -> AWS Certificate Manager -> Request certificate:\ 3. Choose "DNS validation" 4. Copy the "CNAME name" and "CNAME value" -Then copy past the add the validation record to a row of +Then create a validation record in https://gitlab.asf.alaska.edu/operations/puppet/-/edit/production/modules/legacy_dns/files/asf.alaska.edu.db -in the format ` in CNAME .` (see previous records for examples). +of the form ` in CNAME `, stripping the `.asf.alaska.edu` from the CNAME name (see previous records for examples). ### Create the GitHub environment From b3e60e7a6922d44b4e6b5f56c625cdb0d78c95f0 Mon Sep 17 00:00:00 2001 From: Joseph H Kennedy Date: Fri, 30 Jan 2026 14:55:27 -0900 Subject: [PATCH 11/11] Fix formatting and enhance clarity in README Corrected formatting and improved clarity in the README. --- README.md | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 0dca78bdc..87ac49f89 100644 --- a/README.md +++ b/README.md @@ -151,7 +151,7 @@ which we will use to deploy HyP3 via CI/CD tooling: 1. Go to AWS console -> IAM -> Users -> github-actions -> security credentials tab -> "create access key". 2. Select "Other" for key usage -3. (Optional)Add tag value to describe the key, such as "For GitHub Actions CI/CD pipelines" +3. (Optional) Add tag value to describe the key, such as "For GitHub Actions CI/CD pipelines" 4. Store the access key ID and secret access key using your team's password manager. You will use them below in "Create the GitHub environment" as `V2_AWS_ACCESS_KEY_ID` and `V2_AWS_SECRET_ACCESS_KEY`.
@@ -237,13 +237,14 @@ Go to AWS console -> Secrets Manager, then: #### Request SSL cert -*Note: For EDC accounts, you should create the cert in the `us-east-1` region -for use with the CloudFront distribution that you will create later, -even if you're deploying HyP3 to `us-west-2`.* +To allow HTTPS connections, HyP3 needs an SSL certificate that is valid for its deployment domain name (URL), which we can request from AWS. -To allow HTTPS connections, HyP3 needs an SSL certificate that is valid for its deployment domain name (URL): +[!NOTE] +> For EDC accounts, you should create the cert in the `us-east-1` region +> for use with the CloudFront distribution that you will create later, +> even if you're deploying HyP3 to `us-west-2`.* -AWS console -> AWS Certificate Manager -> Request certificate:\ +Go to the AWS console -> AWS Certificate Manager -> Request certificate and then: 1. Select "Request a public certificate" 2. Click the orange "Next" button 3. Choose a "Fully qualified domain name". Domain name should be something like `hyp3-foobar.asf.alaska.edu` or for a test deployment `hyp3-foobar-test.asf.alaska.edu`. @@ -252,7 +253,7 @@ AWS console -> AWS Certificate Manager -> Request certificate:\ Then create a validation record in https://gitlab.asf.alaska.edu/operations/puppet/-/edit/production/modules/legacy_dns/files/asf.alaska.edu.db -of the form ` in CNAME `, stripping the `.asf.alaska.edu` from the CNAME name (see previous records for examples). +of the form ` in CNAME `, stripping `.asf.alaska.edu` from the `CNAME_name` (see previous records for examples). ### Create the GitHub environment @@ -329,12 +330,15 @@ Update the [AWS Accounts and HyP3 Deployments](https://docs.google.com/spreadshe #### Testing and adding user credits to your hyp3 deployment -After successfully deploying HyP3 and your new DNS record has taken effect (or you've edited your local DNS name resolution), you can test your deployment by accessing the Swagger UI and using the POST `/user` tab to -check if your user is approved and has credits for running jobs on the deployment. You will need to be authenticated by either providing an Earthdata Login Bearer Token using the "Authorize" button, or by having a valid `asf-urs` cookie, typically logging into [Vertex](https://search.asf.alaska.edu). Interacting with HyP3 should automatically add your user to the DynamoDB table with the default number of credits (typically 0). +After successfully deploying HyP3 and your new DNS record has taken effect (or you've edited your local DNS name resolution), you can test your +deployment by accessing the Swagger UI and using the POST `/user` tab to check if your user is approved and has credits for running jobs on the +deployment. You will need to be authenticated by either providing an Earthdata Login Bearer Token using the "Authorize" button, or by having a +valid `asf-urs` browser cookie, typically obtained by logging into [Vertex](https://search.asf.alaska.edu). Interacting with HyP3 should +automatically add your user to the DynamoDB table with the default number of credits (typically 0). To add credits to your (or any) user, log in to the AWS console and navigate to DynamoDB -> Explore items, then: 1. Find the table with a format like `hyp3-foobar-UsersTable-XXXXXXXXXXXXX` -2. Edit your user record (only present after using the Swagger UI in some way) +2. Edit your user record if present (after using the Swagger UI in some way) or duplicate an existing reccord updaing the `user_id`. You can then return the Swagger UI and use the POST `/jobs` to run a test job and confirm it completes.