Blueprint-uServices · AimanSiddiqui · Jul 21, 2025 · Jul 21, 2025 · Jul 21, 2025
diff --git a/CRISP_ANALYSIS.md b/CRISP_ANALYSIS.md
@@ -0,0 +1,130 @@
+# CRISP Critical Path Analysis Guide
+
+This guide explains how to use the CRISP plugin for critical path analysis in the Blueprint microservices framework, including integration details, experiment setup, and trace analysis.
+
+---
+
+## What is CRISP?
+
+**CRISP** is a Python-based tool for automated critical path analysis of distributed traces. It exposes a FastAPI service that can analyze Jaeger traces and return the critical path and its latency, helping you identify bottlenecks in microservice applications.
+
+**Critical Path Analysis** is the process of finding the slowest (longest-latency) sequence of operations in a distributed trace, which determines the end-to-end response time for a request.
+
+---
+
+## Where and How is CRISP Integrated?
+
+- The CRISP plugin is located in [`plugins/crisp/`](plugins/crisp/).
+- It is integrated into the DeathStarBench Hotel Reservation example (`examples/dsb_hotel`) via the wiring spec files:
+  - [`examples/dsb_hotel/wiring/specs/original.go`](examples/dsb_hotel/wiring/specs/original.go)
+  - [`examples/dsb_hotel/wiring/specs/chain.go`](examples/dsb_hotel/wiring/specs/chain.go)
+  - [`examples/dsb_hotel/wiring/specs/fanin.go`](examples/dsb_hotel/wiring/specs/fanin.go)
+  - [`examples/dsb_hotel/wiring/specs/fanout.go`](examples/dsb_hotel/wiring/specs/fanout.go)
+- In each wiring spec, a CRISP analysis container is added:
+  ```go
+  analysisContainer := crisp.Container(spec, "trace_analysis")
+  // (add environment variable for Jaeger URL, see below)
+  ```
+- The CRISP container is included in the auto-generated `docker-compose.yml` for each experiment.
+
+---
+
+## How to Build and Run Experiments with CRISP
+
+### 1. Build the CRISP Docker Image
+
+```sh
+cd plugins/crisp
+docker build -t crisp:latest .
+```
+
+### 2. Compile the Blueprint Application with Desired Topology
+
+```sh
+cd examples/dsb_hotel/wiring
+go run main.go -w <spec> -o build_<spec>
+```
+Where `<spec>` is one of: `original`, `chain`, `fanin`, `fanout`.
+
+### 3. (Optional) Set Jaeger URL for CRISP
+
+By default, CRISP will look for Jaeger at `http://jaeger_ctr:16686` (the default service name and port in the generated Docker Compose). If you want to override this, set the environment variable in the wiring spec (recommended for automation):
+
+```go
+import "github.com/blueprint-uservices/blueprint/plugins/linuxcontainer"
+...
+crispCtr := crisp.Container(spec, "trace_analysis")
+linuxcontainer.SetEnv(crispCtr, "JAEGER_URL", "http://jaeger_ctr:16686")
+```
+
+Or, edit the generated `docker-compose.yml` to add:
+```yaml
+  environment:
+    - JAEGER_URL=http://jaeger_ctr:16686
+```
+
+### 4. Run the Application
+
+```sh
+cd build_<spec>/docker
+docker compose up
+```
+
+### 5. Generate Workload and Collect Traces
+
+- Use the workload generator included in the build directory (e.g., `wlgen_proc`) to send requests to the frontend service.
+- Traces will be collected by Jaeger.
+
+### 6. Analyze Traces with CRISP
+
+- Access Jaeger UI at [http://localhost:16686](http://localhost:16686) to find trace IDs.
+- Use CRISP's API to analyze a trace:
+  ```sh
+  curl http://localhost:8000/analyze/<trace_id>
+  ```
+- The response will include the critical path and its total duration.
+
+---
+
+## How the Wiring Specs and Docker Compose are Set Up
+
+- The wiring spec files define which services and containers are included in each experiment.
+- When you build a spec, Blueprint generates a `docker-compose.yml` that includes all services, including Jaeger and CRISP.
+- Go-based services are instrumented with tracing plugins and send traces to Jaeger via `JAEGER_DIAL_ADDR`.
+- CRISP reads traces from Jaeger via the query API (`JAEGER_URL`).
+
+---
+
+## Example: Adding JAEGER_URL in the Wiring Spec
+
+```go
+import "github.com/blueprint-uservices/blueprint/plugins/crisp"
+import "github.com/blueprint-uservices/blueprint/plugins/linuxcontainer"
+...
+crispCtr := crisp.Container(spec, "trace_analysis")
+linuxcontainer.SetEnv(crispCtr, "JAEGER_URL", "http://jaeger_ctr:16686")
+```
+
+---
+
+## Troubleshooting
+
+- **CRISP can't connect to Jaeger:**
+  - Make sure the Jaeger service name and port match what CRISP expects (`jaeger_ctr:16686` by default).
+  - Set the `JAEGER_URL` environment variable if needed.
+- **CRISP container not present in Docker Compose:**
+  - Ensure the wiring spec includes the CRISP container.
+- **Manual changes to docker-compose.yml are overwritten:**
+  - Automate environment variable injection in the wiring spec for persistence.
+
+---
+
+## Further Reading
+
+- [CRISP Plugin README](plugins/crisp/README.md)
+- [Blueprint User Manual](docs/manual)
+- [DeathStarBench Hotel Reservation Example](examples/dsb_hotel/README.md)
+
+---
+
+For questions or contributions, see the main [Blueprint README](README.md) or contact the maintainers via Slack or mailing list. 
diff --git a/README.md b/README.md
@@ -28,6 +28,8 @@ See the 📖[Getting Started](docs/manual/gettingstarted.md) page of the User Ma
 📝[Wiring Spec Plugins](plugins) - plugins that come packaged with Blueprint that can be used in wiring specs to modify applications\
 📓[Workflow Spec Backends](runtime/core) - backend interfaces that can be used in workflow specs when developing applications
 
+🔬[CRISP Critical Path Analysis Guide](CRISP_ANALYSIS.md) - details on integrating, running, and analyzing critical path experiments with the CRISP plugin and Blueprint
+
 ### API Documentation on go.dev
 
 🚀[Blueprint Compiler](https://pkg.go.dev/github.com/blueprint-uservices/blueprint/blueprint)\

diff --git a/blueprint/go.mod b/blueprint/go.mod
@@ -1,8 +1,12 @@
 module github.com/blueprint-uservices/blueprint/blueprint
 
-go 1.20
+go 1.22.0
+
+toolchain go1.24.2
 
 require (
-	golang.org/x/exp v0.0.0-20240416160154-fe59bbe5cc7f
-	golang.org/x/mod v0.17.0
+	golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c
+	golang.org/x/mod v0.21.0
 )
+
+replace github.com/blueprint-uservices/blueprint => ../
diff --git a/blueprint/go.sum b/blueprint/go.sum
@@ -1,8 +1,4 @@
-golang.org/x/exp v0.0.0-20230728194245-b0cb94b80691 h1:/yRP+0AN7mf5DkD3BAI6TOFnd51gEoDEb8o35jIFtgw=
-golang.org/x/exp v0.0.0-20230728194245-b0cb94b80691/go.mod h1:FXUEEKJgO7OQYeo8N01OfiKP8RXMtf6e8aTskBGqWdc=
-golang.org/x/exp v0.0.0-20240416160154-fe59bbe5cc7f h1:99ci1mjWVBWwJiEKYY6jWa4d2nTQVIEhZIptnrVb1XY=
-golang.org/x/exp v0.0.0-20240416160154-fe59bbe5cc7f/go.mod h1:/lliqkxwWAhPjf5oSOIJup2XcqJaw8RGS6k3TGEc7GI=
-golang.org/x/mod v0.11.0 h1:bUO06HqtnRcc/7l71XBe4WcqTZ+3AH1J59zWDDwLKgU=
-golang.org/x/mod v0.11.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
-golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA=
-golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
+golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c h1:7dEasQXItcW1xKJ2+gg5VOiBnqWrJc+rq0DPKyvvdbY=
+golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c/go.mod h1:NQtJDoLvd6faHhE7m4T/1IY708gDefGGjR/iUW8yQQ8=
+golang.org/x/mod v0.21.0 h1:vvrHzRwRfVKSiLrG+d4FMl/Qi4ukBCE6kZlTUkDYRT0=
+golang.org/x/mod v0.21.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY=
diff --git a/examples/dsb_hotel/README.md b/examples/dsb_hotel/README.md
@@ -10,9 +10,10 @@ For the most part, the application directly re-uses the original Hotel Reservati
 
 ## Getting started
 
-Prerequisites for this tutorial: 
+Prerequisites for this tutorial:
  * gRPC and protocol buffers compiler are installed - https://grpc.io/docs/protoc-installation/
  * docker is installed
+ * (Optional, for critical path analysis) Python 3 and Docker for building the CRISP plugin
 
 ## Running unit tests prior to compilation
 
@@ -25,28 +26,47 @@ go test
 
 Local unit tests only work on Linux or WSL.
 
-## Compiling the application
+## Compiling the application (including new wiring specs)
 
-To compile the application, we execute `wiring/main.go` and specify which wiring spec to compile. To view options and list wiring specs, run:
+To compile the application, execute `wiring/main.go` and specify which wiring spec to compile. To view options and list wiring specs, run:
 
 ```
 go run wiring/main.go -h
 ```
 
-The following will compile the `original` wiring spec to the directory `build`. This will fail if the pre-requisite grpc compiler is not installed.
+To compile a specific wiring spec (e.g., `original`, `chain`, `fanin`, `fanout`), run:
 
 ```
-go run wiring/main.go -w original -o build
+go run wiring/main.go -w <spec> -o build_<spec>
 ```
 
-If you navigate to the `build` directory, you will now see a number of build artifact.
-* `build/docker` contains docker images for the various containers of the application, as well as a `docker-compose.yml` file for starting and stopping all containers.
-* `build/docker/*`  contain the individual docker images for services, including a Dockerfile and the golang source code.
-* `build/gotests` contains the unit tests that we ran in the previous section.
+**Important:**
+- If you add or modify a wiring spec (e.g., in `wiring/specs/chain.go`), you must recompile using the above command.
+- If you add or update a plugin (e.g., CRISP), you must rebuild its Docker image (see below).
+
+If you navigate to the `build_<spec>` directory, you will now see a number of build artifacts.
+* `build_<spec>/docker` contains docker images for the various containers of the application, as well as a `docker-compose.yml` file for starting and stopping all containers.
+* `build_<spec>/docker/*`  contain the individual docker images for services, including a Dockerfile and the golang source code.
+* `build_<spec>/gotests` contains the unit tests that we ran in the previous section.
+
+## Building the CRISP plugin (for critical path analysis)
+
+If you are using the CRISP plugin for critical path analysis, you must build its Docker image before running the application:
+
+```
+cd plugins/crisp
+docker build -t crisp:latest .
+```
 
 ## Running the application
 
-To run the application, navigate to `build/docker` and run `docker compose up`.  You might see Docker complain about missing environment variables.  Edit the `.env` file in `build/docker` and put the following:
+To run the application, navigate to `build_<spec>/docker` and run:
+
+```
+docker compose up
+```
+
+You might see Docker complain about missing environment variables.  Edit the `.env` file in `build_<spec>/docker` and put the following:
 
 ```
 FRONTEND_SERVICE_HTTP_BIND_ADDR=9000
@@ -71,8 +91,32 @@ USER_SERVICE_GRPC_BIND_ADDR=9007
 
 ## Running tests in compiled application
 
- After running the application, you can run the unit tests against it.
+After running the application, you can run the unit tests against it.
+
+```
+go test -v -profile_service.grpc.dial_addr=localhost:9002 -search_service.grpc.dial_addr=localhost:9006 -recomd_service.grpc.dial_addr=localhost:9004 -geo_service.grpc.dial_addr=localhost:9001 -user_service.grpc.dial_addr=localhost:9007 -rate_service.grpc.dial_addr=localhost:9003 -reserv_service.grpc.dial_addr=localhost:9005 -frontend_service.http.dial_addr=localhost:9000 -jaeger.dial_addr=localhost:14628
+```
+
+## Running different topologies (experiments)
+
+You can experiment with different service topologies by selecting the appropriate wiring spec:
+- `original`: Full DeathStarBench topology
+- `chain`: Frontend → Search → Geo
+- `fanin`: Search → Geo, Profile → Geo
+- `fanout` (star): Frontend calls all backends directly
+
+Compile and run as described above, substituting `<spec>` with the desired topology.
+
+**Note:**
+- You can specify the wiring spec to use by passing the `-w <spec>` flag to `wiring/main.go`.
+- If you are editing the wiring code directly, ensure the correct spec name is set in the wiring file or in your build/run command.
+- Alternatively, you can change the default experiment by editing the spec in `wiring/main.go` (e.g., change `specs.Fanout` to `specs.Chain`, `specs.Original`, or `specs.Fanin`).
+
+## Analyzing traces and critical path
 
- ```
- go test -v -profile_service.grpc.dial_addr=localhost:9002 -search_service.grpc.dial_addr=localhost:9006 -recomd_service.grpc.dial_addr=localhost:9004 -geo_service.grpc.dial_addr=localhost:9001 -user_service.grpc.dial_addr=localhost:9007 -rate_service.grpc.dial_addr=localhost:9003 -reserv_service.grpc.dial_addr=localhost:9005 -frontend_service.http.dial_addr=localhost:9000 -jaeger.dial_addr=localhost:14628
- ```
+- Access Jaeger UI at [http://localhost:16686](http://localhost:16686) to view traces.
+- If using CRISP, send a request to the CRISP API to analyze a trace:
+  ```
+  curl http://localhost:8000/analyze/<trace_id>
+  ```
+- The output will show the critical path and its total latency for the selected trace.
diff --git a/examples/dsb_hotel/tests/go.mod b/examples/dsb_hotel/tests/go.mod
@@ -1,8 +1,8 @@
 module github.com/blueprint-uservices/blueprint/examples/dsb_hotel/tests
 
-go 1.21
+go 1.22.0
 
-toolchain go1.22.1
+toolchain go1.24.2
 
 require github.com/blueprint-uservices/blueprint/runtime v0.0.0-20240619221802-d064c5861c1e
 
@@ -14,12 +14,15 @@ require (
 require (
 	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/hailocab/go-geoindex v0.0.0-20160127134810-64631bfe9711 // indirect
+	github.com/kr/text v0.2.0 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
 	go.mongodb.org/mongo-driver v1.15.0 // indirect
 	go.opentelemetry.io/otel v1.26.0 // indirect
 	go.opentelemetry.io/otel/metric v1.26.0 // indirect
 	go.opentelemetry.io/otel/trace v1.26.0 // indirect
-	golang.org/x/exp v0.0.0-20240416160154-fe59bbe5cc7f // indirect
+	golang.org/x/crypto v0.28.0 // indirect
+	golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c // indirect
+	gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )
 

diff --git a/examples/dsb_hotel/tests/go.sum b/examples/dsb_hotel/tests/go.sum
@@ -1,5 +1,6 @@
 github.com/blueprint-uservices/blueprint/runtime v0.0.0-20240619221802-d064c5861c1e h1:yRCPuCqLAQDDPB5yE+zPLy6cvX9ysAFQGt/hv5S7e+8=
 github.com/blueprint-uservices/blueprint/runtime v0.0.0-20240619221802-d064c5861c1e/go.mod h1:pWgaE60Wfg1GDkFUz17NcxxaFk66/s2RRO5UZG9lCXA=
+github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/go-logr/logr v1.4.1 h1:pKouT5E8xu9zeFC39JXRDukb6JFQPXM5p5I91188VAQ=
@@ -14,6 +15,8 @@ github.com/hailocab/go-geoindex v0.0.0-20160127134810-64631bfe9711 h1:Oi8hPOZX0g
 github.com/hailocab/go-geoindex v0.0.0-20160127134810-64631bfe9711/go.mod h1:+v2qJ3UZe4q2GfgZO4od004F/cMgJbmPSs7dD/ZMUkY=
 github.com/klauspost/compress v1.17.8 h1:YcnTYrq7MikUT7k0Yb5eceMmALQPYBW/Xltxn0NAMnU=
 github.com/klauspost/compress v1.17.8/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw=
+github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI=
+github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
 github.com/montanaflynn/stats v0.7.1 h1:etflOAAHORrCC44V+aR6Ftzort912ZU+YLiSTuV8eaE=
 github.com/montanaflynn/stats v0.7.1/go.mod h1:etXPPgVO6n31NxCd9KQUMvCM+ve0ruNzt6R8Bnaayow=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
@@ -36,15 +39,11 @@ go.opentelemetry.io/otel/metric v1.26.0 h1:7S39CLuY5Jgg9CrnA9HHiEjGMF/X2VHvoXGgS
 go.opentelemetry.io/otel/metric v1.26.0/go.mod h1:SY+rHOI4cEawI9a7N1A4nIg/nTQXe1ccCNWYOJUrpX4=
 go.opentelemetry.io/otel/trace v1.26.0 h1:1ieeAUb4y0TE26jUFrCIXKpTuVK7uJGN9/Z/2LP5sQA=
 go.opentelemetry.io/otel/trace v1.26.0/go.mod h1:4iDxvGDQuUkHve82hJJ8UqrwswHYsZuWCBllGV2U2y0=
-golang.org/x/crypto v0.22.0 h1:g1v0xeRhjcugydODzvb3mEM9SQ0HGp9s/nh3COQ/C30=
-golang.org/x/crypto v0.22.0/go.mod h1:vr6Su+7cTlO45qkww3VDJlzDn0ctJvRgYbC2NvXHt+M=
-golang.org/x/exp v0.0.0-20240416160154-fe59bbe5cc7f h1:99ci1mjWVBWwJiEKYY6jWa4d2nTQVIEhZIptnrVb1XY=
-golang.org/x/exp v0.0.0-20240416160154-fe59bbe5cc7f/go.mod h1:/lliqkxwWAhPjf5oSOIJup2XcqJaw8RGS6k3TGEc7GI=
-golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
-golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
-golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
-golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
-gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
+golang.org/x/crypto v0.28.0 h1:GBDwsMXVQi34v5CCYUm2jkJvu4cbtru2U4TN2PSyQnw=
+golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c h1:7dEasQXItcW1xKJ2+gg5VOiBnqWrJc+rq0DPKyvvdbY=
+golang.org/x/sync v0.8.0 h1:3NFvSEYkUoMifnESzZl15y791HH1qU2xm6eCJU5ZPXQ=
+golang.org/x/text v0.19.0 h1:kTxAhCbGbxhK0IwgSKiMO5awPoDQ0RpfiVYBfK860YM=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
diff --git a/examples/dsb_hotel/wiring/README.md b/examples/dsb_hotel/wiring/README.md
@@ -2,18 +2,30 @@
 
 # wiring
 
-```go
-import "github.com/blueprint-uservices/blueprint/examples/dsb_hotel/wiring"
-```
-
-Package main provides an application for compiling a number of different wiring specs for the Hotel Reservation application from the DeathStarBench suite.
+This directory contains the wiring logic for the Hotel Reservation application. Wiring specs define the topology and deployment of services.
 
 To display options and usage, invoke:
 
 ```
 go run main.go -h
 ```
 
+## Compiling with a specific wiring spec
+
+To compile the application with a specific wiring spec (e.g., `chain`, `fanin`, `fanout`), run:
+
+```
+go run main.go -w <spec> -o build_<spec>
+```
+
+**Note:**
+- You must specify the correct spec name using the `-w` flag, or by editing the wiring file to set the desired experiment.
+- Alternatively, you can change the default experiment by editing the spec in `main.go` (e.g., change `specs.Fanout` to `specs.Chain`, `specs.Original`, or `specs.Fanin`).
+- If you add or modify a wiring spec in `specs/`, you must recompile using the above command.
+- If you add or update a plugin (such as CRISP for critical path analysis), you must rebuild its Docker image (see the main README for details).
+
+See the main README in the parent directory for full instructions on running the application and analyzing traces.
+
 ## Index