From 95fcd53c11761e76cc51a6717abe17a8b75bf590 Mon Sep 17 00:00:00 2001 From: Connor Leech Date: Wed, 26 Mar 2025 02:40:55 -0400 Subject: [PATCH 1/2] docs: updating readme content Signed-off-by: Connor Leech --- README.md | 212 +++++++++++++++++++++++++++--------------------------- 1 file changed, 106 insertions(+), 106 deletions(-) diff --git a/README.md b/README.md index 26bee21..178830a 100644 --- a/README.md +++ b/README.md @@ -1,170 +1,170 @@ # CISO Agent -CISO (Chief Information Security Officer) agents automate compliance assessments using specialized tools. They generate policies (e.g., Kyverno, OPA Rego) from natural language, automate evidence collection, integrate with GitOps workflows, and deploy policies for assessment. Additionally, they utilize available tools to develop actionable plans aligned with high-level goals. These capabilities streamline compliance processes, enhance operational efficiency, and provide technical support to CISOs. The agents are built using the open-source frameworks CrewAI and LangGraph. +The **CISO Agent** (Chief Information Security Officer Agent) automates security and compliance assessments for ITBench scenarios. It leverages large language models (LLMs) to interpret compliance goals, generate policies (e.g., Kyverno, OPA Rego), deploy them, and verify enforcement. The agent streamlines compliance processes, integrates with GitOps workflows, and uses available tools to develop actionable plans aligned with high-level goals. +The CISO Agent is built using the open-source frameworks [CrewAI](https://github.com/joaomdmoura/crewAI) and [LangGraph](https://github.com/langchain-ai/langgraph). -## Prerequisites +--- -- Access to an OpenAI-Compatible LLM service - - tested with `IBM watsonx.ai`, `OpenAI` and `Azure OpenAI Service`. -- [Sample Task Scenario](https://github.com/IBM/ITBench-Scenarios) setup - - This requires 1 Kubernetes cluster and/or 1 RHEL host. For more details, please refer to its [README](https://github.com/IBM/ITBench-Scenarios/tree/main/ciso#readme). -- `python` command (tested with `3.11` ) -- `docker` or `podman` command (tested with docker `26.1.0` and podman `5.1.2` ) +## ๐Ÿ”ง Prerequisites -## Getting started +- Access to an OpenAI-compatible LLM service + (tested with IBM watsonx.ai, OpenAI, Azure OpenAI) +- A running [ITBench scenario](https://github.com/IBM/ITBench-Scenarios) + (requires 1 Kubernetes cluster and/or 1 RHEL host) +- `python` (tested with 3.11) +- `docker` or `podman` (tested with Docker 26.1.0, Podman 5.1.2) -### 1. Setup a Sample Task Scenario on your Kubernetes cluster / RHEL host +--- -Follow the README of sample task scenario and complete `make deploy_bundle` and `make inject_fault` commands for a single scenario. +## ๐Ÿš€ Getting started -Next, run the following command to get `goal` description which is an input to the agent. +### 1. Deploy a scenario + +Follow the [CISO scenario guide](https://github.com/IBM/ITBench-Scenarios/tree/main/ciso#readme) to set up the environment. + +Run: ```bash -$ make get 2>/dev/null | jq -r .goal_template +make deploy_bundle +make inject_fault ``` -Example Output: +Then retrieve the goal (prompt input for the agent): + ```bash -I would like to check if the following condition is satisfiled, given a Kubernetes cluster with `kubeconfig.yaml` - Minimize the admission of containers wishing to share the host network namespace. +make get 2>/dev/null | jq -r .goal_template +``` + +Example goal output: + +``` +I would like to check if the following condition is satisfied, given a Kubernetes cluster with `kubeconfig.yaml`: +Minimize the admission of containers wishing to share the host network namespace. -To check the condition, do the following steps. -- deploy a Kyverno policy to the cluster -- chcek if the policy is correctly deployed. +To check the condition, do the following steps: +- Deploy a Kyverno policy to the cluster +- Check if the policy is correctly deployed -If deploying the policy failed and if you can fix the issue, you will do it and try deploying again. -Once you get a final answer, you can quit the work. +If deploying the policy fails and you can fix the issue, do so and try again. The cluster's kubeconfig is at `{{ kubeconfig }}`. ``` -The above text is an example of ciso task scenarios. +Note: Keep this goal text โ€” you will need it in step 4. -Please keep this goal text for running the agent later. +--- -### 2. Clone this repository +### 2. Clone the repository ```bash -$ git clone https://github.com/IBM/it-bench-ciso-caa-agent.git -$ cd ciso-agent +git clone https://github.com/IBM/itbench-ciso-caa-agent.git +cd itbench-ciso-caa-agent ``` -### 3. Build agent container image +--- -CISO Agent is designed to run as a container. Please build the container image with this command. +### 3. Build the agent container -```bash -$ docker build . -f Dockerfile -t ciso-agent:latest -``` - -If you are using `podman` runtime instead of `docker`, this is the command. +Build the image (using Docker or Podman): ```bash -$ podman build . -f Dockerfile -t ciso-agent:latest +docker build -f Dockerfile -t ciso-agent:latest . +# or +podman build -f Dockerfile -t ciso-agent:latest . ``` -This command could take a several minutes normally. +You only need to rebuild if you update the source code. -**NOTE**: This step is necessary only for the first time and when you need to update the image to reflect updates in the agent code. +--- -### 3. Create `.env` file and set LLM API credentials +### 4. Configure LLM credentials -To run ciso-agent, you need a LLM API access which is compatible with LiteLLM. +The agent supports any provider compatible with [LiteLLM](https://docs.litellm.ai/). Create a `.env` file: -Many LLM services support it, including IBM watsonx.ai, OpenAI and Azure OpenAI Service. +#### i. IBM watsonx.ai -To configure access, create a .env file with the following details. - -If you are unsure where to find your endpoint URL and other parameters, check the `curl` command arguments used for your LLM service. +```env +LLM_BASE_URL= +LLM_API_KEY= +LLM_MODEL_NAME=ibm/granite-3-8b-instruct +WATSONX_PROJECT_ID= +``` -i. For **IBM watsonx.ai** +#### ii. OpenAI -```bash -# .env file -LLM_BASE_URL = # before `/ml/v1/text/generation` -LLM_API_KEY = -LLM_MODEL_NAME = # E.g. "ibm/granite-3-8b-instruct" -WATSONX_PROJECT_ID = +```env +LLM_API_KEY= +LLM_MODEL_NAME=gpt-4o-mini +# LLM_BASE_URL optional for OpenAI ``` -ii. For **OpenAI** -```bash -# .env file -LLM_API_KEY = -LLM_MODEL_NAME = # E.g. "gpt-4o-mini" -# NOTE: The endpoint URL an be omitted for OpenAI -``` +#### iii. Azure OpenAI -iii. For **Azure OpenAI Service** -```bash -# .env file -LLM_BASE_URL = # before `/chat/completions` -LLM_API_KEY = -LLM_MODEL_NAME = -LLM_PARAMS = '{"api-version": ""}' -# NOTE: For Azure OpenAI service, the model to be used is determined by the endpoint URL. -# Thus, here is ignored during LLM calls. +```env +LLM_BASE_URL= +LLM_API_KEY= +LLM_MODEL_NAME= # Ignored, set via endpoint +LLM_PARAMS='{"api-version": ""}' ``` -Any location is fine for `.env` file, but the file needs to be mountable by the agent container later. - -### 4. Start the agent +You can place the `.env` file anywhere, as long as it can be mounted into the container later. -Now, ready to run the agent. +--- -Inputs to the agent is the following 3 things. -- Workdir - - The directory for scenario. Before agent starts, `kubeconfig.yaml` and/or `inventory.ansible.ini` must be planced here. -- `.env` file -- Goal text +### 5. Run the agent -Please run the following command with these inputs. +Prepare: -Note for the input values: -- Replace `` with the actual path for your workdir -- Replace `` with the actual path for your .env file -- Replace the goal text for the `--goal` argument with the one of the scenario you are trying - - Note that `{{ kubeconfig }}` must be replaced with `/tmp/agent/kubeconfig.yaml`. - - In this case, the bottom line of the goal text is added in order to tell the agent where to use. +- `` โ€” Directory containing the scenario context (e.g., `kubeconfig.yaml`) +- `` โ€” Path to your `.env` file +- `` โ€” Goal output from step 1 (with `{{ kubeconfig }}` replaced) -If you use `podman` engine, please replace `docker` with `podman` below. +Run: ```bash -$ docker run --rm -ti --name ciso-agent \ - -v :/tmp/agent \ - -v :/etc/ciso-agent/.env \ - ciso-agent:latest \ - python src/ciso_agent/main.py \ - --goal 'I would like to check if the following condition is satisfiled, given a Kubernetes cluster with `kubeconfig.yaml` - Minimize the admission of containers wishing to share the host network namespace. - -To check the condition, do the following steps. -- deploy a Kyverno policy to the cluster -- chcek if the policy is correctly deployed. +docker run --rm -ti \ + -v :/tmp/agent \ + -v :/etc/ciso-agent/.env \ + ciso-agent:latest \ + python src/ciso_agent/main.py \ + --goal '' \ + --auto-approve +``` -If deploying the policy failed and if you can fix the issue, you will do it and try deploying again. -Once you get a final answer, you can quit the work. +๐Ÿ“Œ Make sure the goal ends with: +``` The cluster's kubeconfig is at `/tmp/agent/kubeconfig.yaml`. -You can use `/tmp/agent` as your workdir.' \ - --auto-approve +You can use `/tmp/agent` as your workdir. ``` -If the inputs and scenario setup are correctly configured, the agent will display logs similar to the image below: +If using Podman, replace `docker` with `podman`. + +--- + +### 6. Example logs + +The agent will output logs like: + +![Start](img/agent_log_example_beginning.png) + +When the task is complete: + +![Result](img/agent_log_example_result.png) -Example agent log at the beginning +Average run time is under 5 minutes depending on LLM latency. -The agent will then continue its work until it achieves the goal. +--- -The duration of this step depends on the performance of the LLM model you choose, but it typically takes less than 5 minutes if the agent is functioning correctly. +## โœ… Evaluation -When the agent displays logs similar to the image below and the Docker/Podman process ends, the agent has completed its task. +After the agent completes its work, evaluate the results using the scenarioโ€™s instructions. -Example agent log for results +Refer to the [CISO scenarios README](https://github.com/IBM/ITBench-Scenarios/tree/main/ciso#readme) for details. -### 5. Evaluation +--- -Once the agent completes its work, you can proceed with the evaluation step for the task scenario. +## ๐Ÿ“ License -Please refer to the README of the task scenario for further details. +This project is licensed under the [Apache License 2.0](LICENSE). From 15737c1b16cd37f05e4d5817fe7e8d41ab274346 Mon Sep 17 00:00:00 2001 From: Connor Leech Date: Thu, 3 Apr 2025 08:12:09 -0400 Subject: [PATCH 2/2] clarity and formatting improvements Signed-off-by: Connor Leech --- README.md | 133 ++++++++++++++++++------------------------------------ 1 file changed, 43 insertions(+), 90 deletions(-) diff --git a/README.md b/README.md index 178830a..d41262d 100644 --- a/README.md +++ b/README.md @@ -1,36 +1,28 @@ # CISO Agent -The **CISO Agent** (Chief Information Security Officer Agent) automates security and compliance assessments for ITBench scenarios. It leverages large language models (LLMs) to interpret compliance goals, generate policies (e.g., Kyverno, OPA Rego), deploy them, and verify enforcement. The agent streamlines compliance processes, integrates with GitOps workflows, and uses available tools to develop actionable plans aligned with high-level goals. +CISO (Chief Information Security Officer) agents automate compliance assessments. They generate policies (e.g., Kyverno, OPA Rego) from natural language prompts, automate evidence collection, integrate with GitOps workflows, and deploy policies to validate compliance. These capabilities streamline compliance processes and enhance operational efficiency. The agents are built using the open-source frameworks [CrewAI](https://github.com/crewAIInc/crewAI) and [LangGraph](https://github.com/langchain-ai/langgraph). -The CISO Agent is built using the open-source frameworks [CrewAI](https://github.com/joaomdmoura/crewAI) and [LangGraph](https://github.com/langchain-ai/langgraph). - ---- - -## ๐Ÿ”ง Prerequisites +## Prerequisites - Access to an OpenAI-compatible LLM service - (tested with IBM watsonx.ai, OpenAI, Azure OpenAI) -- A running [ITBench scenario](https://github.com/IBM/ITBench-Scenarios) + (tested with IBM watsonx.ai, OpenAI, and Azure OpenAI) +- A deployed [ITBench scenario](https://github.com/IBM/ITBench-Scenarios) (requires 1 Kubernetes cluster and/or 1 RHEL host) - `python` (tested with 3.11) - `docker` or `podman` (tested with Docker 26.1.0, Podman 5.1.2) ---- - -## ๐Ÿš€ Getting started +## Getting started ### 1. Deploy a scenario -Follow the [CISO scenario guide](https://github.com/IBM/ITBench-Scenarios/tree/main/ciso#readme) to set up the environment. - -Run: +Follow the [CISO scenario guide](https://github.com/IBM/ITBench-Scenarios/tree/main/ciso#readme): ```bash make deploy_bundle make inject_fault ``` -Then retrieve the goal (prompt input for the agent): +Then get the scenario goal for the agent: ```bash make get 2>/dev/null | jq -r .goal_template @@ -38,7 +30,7 @@ make get 2>/dev/null | jq -r .goal_template Example goal output: -``` +```text I would like to check if the following condition is satisfied, given a Kubernetes cluster with `kubeconfig.yaml`: Minimize the admission of containers wishing to share the host network namespace. @@ -51,120 +43,81 @@ If deploying the policy fails and you can fix the issue, do so and try again. The cluster's kubeconfig is at `{{ kubeconfig }}`. ``` -Note: Keep this goal text โ€” you will need it in step 4. - ---- +Save this goal text for later. -### 2. Clone the repository +### 2. Clone this repository ```bash -git clone https://github.com/IBM/itbench-ciso-caa-agent.git -cd itbench-ciso-caa-agent +git clone https://github.com/IBM/it-bench-ciso-caa-agent.git +cd ciso-agent ``` ---- - ### 3. Build the agent container -Build the image (using Docker or Podman): - ```bash docker build -f Dockerfile -t ciso-agent:latest . -# or +# or, if using podman: podman build -f Dockerfile -t ciso-agent:latest . ``` -You only need to rebuild if you update the source code. - ---- - -### 4. Configure LLM credentials - -The agent supports any provider compatible with [LiteLLM](https://docs.litellm.ai/). Create a `.env` file: +### 4. Configure `.env` with LLM credentials -#### i. IBM watsonx.ai +Create a `.env` file using your LLM service credentials: +- **IBM watsonx.ai** ```env -LLM_BASE_URL= -LLM_API_KEY= -LLM_MODEL_NAME=ibm/granite-3-8b-instruct -WATSONX_PROJECT_ID= +LLM_BASE_URL= +LLM_API_KEY= +LLM_MODEL_NAME= +WATSONX_PROJECT_ID= ``` -#### ii. OpenAI - +- **OpenAI** ```env -LLM_API_KEY= -LLM_MODEL_NAME=gpt-4o-mini -# LLM_BASE_URL optional for OpenAI +LLM_API_KEY= +LLM_MODEL_NAME= ``` -#### iii. Azure OpenAI - +- **Azure OpenAI** ```env -LLM_BASE_URL= -LLM_API_KEY= -LLM_MODEL_NAME= # Ignored, set via endpoint -LLM_PARAMS='{"api-version": ""}' +LLM_BASE_URL= +LLM_API_KEY= +LLM_MODEL_NAME= +LLM_PARAMS='{"api-version": ""}' ``` -You can place the `.env` file anywhere, as long as it can be mounted into the container later. - ---- - ### 5. Run the agent Prepare: -- `` โ€” Directory containing the scenario context (e.g., `kubeconfig.yaml`) -- `` โ€” Path to your `.env` file -- `` โ€” Goal output from step 1 (with `{{ kubeconfig }}` replaced) +- ``: Contains scenario files (`kubeconfig.yaml`, etc.) +- ``: Path to your `.env` +- ``: Scenario goal (replace `{{ kubeconfig }}` with `/tmp/agent/kubeconfig.yaml`) -Run: +Then run: ```bash docker run --rm -ti \ - -v :/tmp/agent \ - -v :/etc/ciso-agent/.env \ + -v :/tmp/agent \ + -v :/etc/ciso-agent/.env \ ciso-agent:latest \ python src/ciso_agent/main.py \ --goal '' \ --auto-approve ``` -๐Ÿ“Œ Make sure the goal ends with: - -``` -The cluster's kubeconfig is at `/tmp/agent/kubeconfig.yaml`. -You can use `/tmp/agent` as your workdir. -``` - -If using Podman, replace `docker` with `podman`. - ---- - -### 6. Example logs - -The agent will output logs like: - -![Start](img/agent_log_example_beginning.png) - -When the task is complete: - -![Result](img/agent_log_example_result.png) - -Average run time is under 5 minutes depending on LLM latency. - ---- +Replace `docker` with `podman` if needed. -## โœ… Evaluation +The agent logs will appear similar to: -After the agent completes its work, evaluate the results using the scenarioโ€™s instructions. +![Start](img/agent_log_example_beginning.png) +(Agent starting) -Refer to the [CISO scenarios README](https://github.com/IBM/ITBench-Scenarios/tree/main/ciso#readme) for details. +![Result](img/agent_log_example_result.png) +(Agent finished) ---- +The run typically takes under 5 minutes. -## ๐Ÿ“ License +### 6. Evaluation -This project is licensed under the [Apache License 2.0](LICENSE). +After the agent finishes, evaluate results using the scenarioโ€™s instructions [here](https://github.com/IBM/ITBench-Scenarios/tree/main/ciso#readme).