Display perf results as PR comment

[linsword13]

The PR build will create (or update if exists) a comment displaying the perf test durations from the current PR, against the historical data we have.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces an automated system to display performance test results directly within GitHub pull requests. By integrating a new Python script into the existing Cloud Build pipeline, performance metrics from the current PR are now posted as a comment, alongside a comparison to historical average durations retrieved from BigQuery. This enhancement provides immediate and contextual performance feedback to developers, streamlining the review process and helping to identify performance regressions early.

Highlights

• Automated Performance Metrics in PRs: Implemented a new automated step in the Cloud Build pipeline to post performance test results directly as comments on GitHub Pull Requests.
• Historical Data Comparison: The posted performance metrics now include a comparison against historical average durations fetched from BigQuery, providing context for performance changes.
• GitHub API Integration: A new Python script (post_perf_pr_comment.py) was added to interact with the GitHub API, handling the creation and updating of PR comments.
• Secure Token Management: Configured Cloud Build to securely retrieve and use a GITHUB_PR_TOKEN from Google Secret Manager for authenticating GitHub API requests.

Changelog

• share/ramble/cloud-build/ramble-perf-tests.yaml
  • Added a new conditional step to execute post_perf_pr_comment.py if a PR number is available.
  • Included GITHUB_PR_TOKEN in the secretEnv for the build step.
  • Configured GITHUB_PR_TOKEN to be sourced from Google Secret Manager.
• share/ramble/qa/post_perf_pr_comment.py
  • New Python script created to manage GitHub PR comments for performance metrics.
  • Includes logic to fetch historical performance averages from BigQuery.
  • Formats current and historical performance data into a Markdown table.
  • Utilizes the GitHub API to post new comments or update existing ones, identified by a unique marker.

Activity

• No human activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a valuable feature to display performance test results as PR comments, enhancing visibility and feedback for developers. However, the script has several critical security vulnerabilities related to improper handling of command-line arguments, specifically SQL injection in BigQuery queries, credential leakage via URL manipulation (SSRF), and path traversal when reading the metrics file. These vulnerabilities are particularly concerning in a CI/CD environment. Additionally, there are areas for improvement regarding dependency management and hardcoded values, which could affect the maintainability and robustness of the solution.

[linsword13]

Ramble Performance Test Metrics

Test Name	Outcome	Duration (s)	Last 5 Avg (s)
test_analyze_large_file	passed	43.2153	41.8927
test_many_experiments	passed	67.8068	66.8188