feat: Add support for Gemma QAT models #253
Conversation
Adds the necessary configurations to support the Gemma 3 QAT (quantization-aware training) models. This includes: - Adding build steps to cloudbuild.yaml for the 1b, 4b, 12b, and 27b QAT models. - Updating the model name mappings in converter.go to include the new QAT model variants.
Summary of ChangesHello @jethac, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces foundational support for Gemma 3 Quantization-Aware Training (QAT) models within the Cloudrun demo environment. It extends the existing infrastructure to recognize, build, and integrate these optimized model variants, allowing for their seamless deployment and use. The changes primarily involve updating build configurations and model name mappings to accommodate the new QAT models. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
|
I would have updated README.md as well but I guess we would have to internally push new pre-built docker images. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds support for Gemma QAT models by updating the cloudbuild.yaml with new build steps and converter.go with new model mappings. The changes in converter.go look good. For cloudbuild.yaml, I've identified significant code duplication in the new build steps and suggested a refactoring using YAML anchors to improve maintainability. Overall, the changes are in the right direction.
| - id: 'Build gemma3 1b QAT Image' | ||
| name: 'gcr.io/cloud-builders/docker:latest' | ||
| dir: 'Demos/Gemma-on-Cloudrun/' # in CI cloudbuild.yaml's context is the root of the repo. | ||
| env: | ||
| - 'MODEL=gemma3:1b-it-qat' | ||
| script: | | ||
| set -x | ||
| docker build --pull --build-arg MODEL=${MODEL} . -t "${_LOCATION}-docker.pkg.dev/${PROJECT_ID}/${_REPO}/gemma/gemma3-1b-it-qat:latest" | ||
| - id: 'Build gemma3 4b QAT Image' | ||
| name: 'gcr.io/cloud-builders/docker:latest' | ||
| dir: 'Demos/Gemma-on-Cloudrun/' # in CI cloudbuild.yaml's context is the root of the repo. | ||
| env: | ||
| - 'MODEL=gemma3:4b-it-qat' | ||
| script: | | ||
| set -x | ||
| docker build --pull --build-arg MODEL=${MODEL} . -t "${_LOCATION}-docker.pkg.dev/${PROJECT_ID}/${_REPO}/gemma/gemma3-4b-it-qat:latest" | ||
| - id: 'Build gemma3 12b QAT Image' | ||
| name: 'gcr.io/cloud-builders/docker:latest' | ||
| dir: 'Demos/Gemma-on-Cloudrun/' # in CI cloudbuild.yaml's context is the root of the repo. | ||
| env: | ||
| - 'MODEL=gemma3:12b-it-qat' | ||
| script: | | ||
| set -x | ||
| docker build --pull --build-arg MODEL=${MODEL} . -t "${_LOCATION}-docker.pkg.dev/${PROJECT_ID}/${_REPO}/gemma/gemma3-12b-it-qat:latest" | ||
| - id: 'Build gemma3 27b QAT Image' | ||
| name: 'gcr.io/cloud-builders/docker:latest' | ||
| dir: 'Demos/Gemma-on-Cloudrun/' # in CI cloudbuild.yaml's context is the root of the repo. | ||
| env: | ||
| - 'MODEL=gemma3:27b-it-qat' | ||
| script: | | ||
| set -x | ||
| docker build --pull --build-arg MODEL=${MODEL} . -t "${_LOCATION}-docker.pkg.dev/${PROJECT_ID}/${_REPO}/gemma/gemma3-27b-it-qat:latest" |
There was a problem hiding this comment.
These four new build steps are nearly identical, which creates code duplication and makes the configuration harder to maintain. You can use YAML anchors to define a template for the repeated parts.
By defining the script once with an anchor and using an additional TAG environment variable, you can significantly reduce the repetition. This makes the steps cleaner and easier to update in the future. This approach could be extended to template the name and dir fields as well for even less duplication.
- id: 'Build gemma3 1b QAT Image'
name: 'gcr.io/cloud-builders/docker:latest'
dir: 'Demos/Gemma-on-Cloudrun/' # in CI cloudbuild.yaml's context is the root of the repo.
env:
- 'MODEL=gemma3:1b-it-qat'
- 'TAG=gemma3-1b-it-qat'
script: &qat_build_script |
set -x
docker build --pull --build-arg MODEL=${MODEL} . -t "${_LOCATION}-docker.pkg.dev/${PROJECT_ID}/${_REPO}/gemma/${TAG}:latest"
- id: 'Build gemma3 4b QAT Image'
name: 'gcr.io/cloud-builders/docker:latest'
dir: 'Demos/Gemma-on-Cloudrun/' # in CI cloudbuild.yaml's context is the root of the repo.
env:
- 'MODEL=gemma3:4b-it-qat'
- 'TAG=gemma3-4b-it-qat'
script: *qat_build_script
- id: 'Build gemma3 12b QAT Image'
name: 'gcr.io/cloud-builders/docker:latest'
dir: 'Demos/Gemma-on-Cloudrun/' # in CI cloudbuild.yaml's context is the root of the repo.
env:
- 'MODEL=gemma3:12b-it-qat'
- 'TAG=gemma3-12b-it-qat'
script: *qat_build_script
- id: 'Build gemma3 27b QAT Image'
name: 'gcr.io/cloud-builders/docker:latest'
dir: 'Demos/Gemma-on-Cloudrun/' # in CI cloudbuild.yaml's context is the root of the repo.
env:
- 'MODEL=gemma3:27b-it-qat'
- 'TAG=gemma3-27b-it-qat'
script: *qat_build_script1 participant
Adds the necessary configurations to support the Gemma 3 QAT (quantization-aware training) models.
This includes: