feature of perf jedi: Add performance summary command with calendar-based weekly change#89
Conversation
| logger = logging.getLogger(__name__) | ||
|
|
||
| # Default control plane configs used when user doesn't specify a config | ||
| _DEFAULT_CONTROL_PLANE_CONFIGS = [ |
There was a problem hiding this comment.
This is not the real control plane. These are the ones that we often prefer looking at
| } | ||
|
|
||
|
|
||
| def _calculate_period_change( |
There was a problem hiding this comment.
Did you mean percentage change here?
| return text | ||
| if len(text) <= max_len: | ||
| return text | ||
| if max_len <= 3: |
There was a problem hiding this comment.
What do you mean by this case here?
There was a problem hiding this comment.
To avoid slack message display issue beyond max width, metrics names are truncated when it is above certain max_len, then added "..." behind it to indicate truncation.
Adding "..." would be redundant when we require some column to be less than 3 char wide, so I decided to just return the truncated name without "...".
| "get_orion_metrics_with_meta unavailable, falling back to get_orion_metrics: %s", | ||
| e, | ||
| ) | ||
| result = await _call_mcp_tool("get_orion_metrics", {"config_name": config}) |
There was a problem hiding this comment.
What OCP version is going to be used for this call? Looking at the source code it looks like its going to use 4.20 as a hard coded value: https://github.com/jtaleric/orion-mcp/blob/main/utils/utils.py#L266
There was a problem hiding this comment.
okay, in latest commit, I extended the MCP tool get_orion_metrics to accept a version parameter, and pass it from get_metrics when falling back.
|
|
||
| try: | ||
| result = await _call_mcp_tool( | ||
| "get_orion_performance_data", |
There was a problem hiding this comment.
Do we even need this tool, when we have a fallback available? Same in the other places
There was a problem hiding this comment.
good question, I have described the Rationale / Justification of adding new tools in my Orion-MCP PR: cloud-bulldozer/orion-mcp#2
|
@avasilevskii as per your question, I think its fine to make direct mcp tool calls and let them skip agentic workflow if not required. Honestly I am fine with either. Expect for the |
|
needs a readme section |
- Introduced a new `PerformanceData` dataclass to encapsulate performance metrics. - Refactored `_format_metrics_table` and `_format_config_table` functions for improved clarity and functionality. - Updated `get_performance_data` to return `PerformanceData` instances instead of dictionaries. - Enhanced table formatting to conditionally include configuration details based on parameters.
- Added a new section detailing the usage of the Performance Summary feature in BugZooka. - Included examples and notes on configuration and behavior. - Updated mandatory and optional fields to include `JEDI_BOT_SLACK_USER_ID` and performance summary settings.
…oding/BugZooka into option2-config-alias
README.md
Outdated
| **Notes:** | ||
| - If no config is provided, defaults to a curated control-plane config list. | ||
| - `ALL` uses all available Orion configs (fallback list is used if MCP is unavailable). | ||
| - Multiple versions can be provided (e.g. `4.19 4.20`). |
There was a problem hiding this comment.
What do you mean by this? Can you provide an example for performance summary with this scenario?
There was a problem hiding this comment.
ok will add to the usage example
4 participants
Summary
Implements the
performance summarycommand that allows users to request performance metric summaries via Slack. The command fetches telemetry data from Orion and displays formatted tables with Min/Max/Avg statistics and weekly change percentages.Key features:
Usage:
@bot performance summary [ALL|config1.yaml,config2.yaml,config3.yaml...] [version(s)] [number of days]d [verbose]
All arguments are optional(order does not matter), the default lookback and version are 14 days and 4.19.
Default configs are:
"metal-perfscale-cpt-virt-udn-density.yaml",
"trt-external-payload-cluster-density.yaml",
"trt-external-payload-node-density.yaml",
"trt-external-payload-node-density-cni.yaml",
"trt-external-payload-crd-scale.yaml",
"small-scale-udn-l3.yaml",
"med-scale-udn-l3.yaml"
"ALL" option is available, which contains 41 predefined config files in https://github.com/cloud-bulldozer/orion/tree/main/examples.
See more in the demo section.
Changes
bugzooka/analysis/perf_summary_analyzer.py: New module for fetching and formatting performance databugzooka/integrations/slack_socket_listener.py: Handleperformance summarycommand and multi-message responsesTest plan
@BugZooka performance summaryin SlackKnown limitations
Jira ticket
https://issues.redhat.com/browse/PERFSCALE-4422
Demo
@PerfScale Padawan performance summary trt-external-payload-node-density-cni.yaml,trt-external-payload-node-density.yaml 30d verbose