From d5c29fd57d5c6e92d58807b32eac82186607299a Mon Sep 17 00:00:00 2001 From: Brook <71849503+Primebrook@users.noreply.github.com> Date: Fri, 7 Feb 2025 19:12:12 +0000 Subject: [PATCH 1/9] release runbook --- RELEASE.md | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 RELEASE.md diff --git a/RELEASE.md b/RELEASE.md new file mode 100644 index 0000000..2121a1f --- /dev/null +++ b/RELEASE.md @@ -0,0 +1,78 @@ +# Release Guide for Exstatic + +## **1. Update Version** +Before creating a new release, ensure that the version number is updated in the following files: + +### **Update `mix.exs`** +Modify the `@version` field: +```elixir +defmodule Exstatic.MixProject do + use Mix.Project + + @version "0.1.1" # Update to the new version + + def project do + [ + app: :exstatic, + version: @version, + elixir: "~> 1.18", + ... + ] + end +end +``` + +--- + +## **2. Update `CHANGELOG.md`** +- Add a new section at the top for the new version. +- Summarize the key changes. + +Example: +```markdown +## [0.1.1] - YYYY-MM-DD +### Added +- Support for additional platforms. +- Improved CI/CD workflows. +- Enhanced documentation. + +### Fixed +- Resolved issue with downloading precompiled NIFs. +``` + +--- + +## **3. Commit Version Bump** +After updating the version, commit the changes: +```bash +git add mix.exs native.ex +git commit -m "Bump version to 0.1.1" +git push origin main +``` + +--- + +## **4. Tag the New Version** +To create a release, tag the new version and push it: +```bash +git tag -a v0.1.1 -m "Release version 0.1.1" +git push origin v0.1.1 +``` +This will trigger the GitHub Actions release workflow, compiling and uploading precompiled NIFs. + +--- + +## **5. Monitor the Release Workflow** +- Go to **GitHub Actions** and check the **Compile & Upload NIFs** workflow. +- Verify that all jobs (for Linux and macOS targets) succeed. + +--- + +## **5. Verify the Release** +Once the workflow completes: +- Go to [GitHub Releases](https://github.com/intellection/exstatic/releases) and confirm that the NIF artifacts are available. +- Download and test the precompiled NIFs using: +```elixir +EXSTATIC_BUILD=false mix rustler_precompiled.download Exstatic.Native --all --print +``` +--- From 41ce57066b073c2c38b60c875f1dcaea260c1d21 Mon Sep 17 00:00:00 2001 From: Brook <71849503+Primebrook@users.noreply.github.com> Date: Sat, 8 Feb 2025 10:55:26 +0000 Subject: [PATCH 2/9] accessing priv repository --- hehe.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 hehe.md diff --git a/hehe.md b/hehe.md new file mode 100644 index 0000000..f8d6b25 --- /dev/null +++ b/hehe.md @@ -0,0 +1,25 @@ +1. Go to GitHub Token Settings +Open GitHub Personal Access Tokens +Click "Generate new token (classic)". +2. Configure Token Permissions +Token Name: +Give it a meaningful name like `exstatic-release-access`. + +Expiration: +Choose "No expiration" or a time period based on your security policy. + +Select Scopes: +For private repositories, you need: + +✅ repo → Full read access to private repositories + +This includes: +repo:status +repo_deployment +public_repo +repo:invite +✅ read:packages + +3. Generate and Copy the Token +Click "Generate token". +Copy the token immediately (you won’t see it again!). From be41739072596bf3940ac9b20848a5269553becb Mon Sep 17 00:00:00 2001 From: Brook <71849503+Primebrook@users.noreply.github.com> Date: Sat, 8 Feb 2025 20:28:34 +0000 Subject: [PATCH 3/9] clearer flow --- RELEASE.md | 114 ++++++++++++++++++++++++----------------------------- 1 file changed, 51 insertions(+), 63 deletions(-) diff --git a/RELEASE.md b/RELEASE.md index 2121a1f..79fc46f 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,78 +1,66 @@ -# Release Guide for Exstatic - -## **1. Update Version** -Before creating a new release, ensure that the version number is updated in the following files: - -### **Update `mix.exs`** -Modify the `@version` field: -```elixir -defmodule Exstatic.MixProject do - use Mix.Project - - @version "0.1.1" # Update to the new version - - def project do - [ - app: :exstatic, - version: @version, - elixir: "~> 1.18", - ... - ] - end -end -``` +# Exstatic Release Process ---- +This document outlines the steps required to release a new version of Exstatic, ensuring that both Elixir and native Rust code are properly updated and distributed via precompiled NIFs. -## **2. Update `CHANGELOG.md`** -- Add a new section at the top for the new version. -- Summarize the key changes. +## **1️⃣ Bump version in `mix.exs`** -Example: -```markdown -## [0.1.1] - YYYY-MM-DD -### Added -- Support for additional platforms. -- Improved CI/CD workflows. -- Enhanced documentation. +After you've made your code changes, update the version in mix.exs (`@version "x.y.z"`) -### Fixed -- Resolved issue with downloading precompiled NIFs. -``` +Now follow the steps below to release the update. ---- +## **2️⃣ Merge Changes to `main`** -## **3. Commit Version Bump** -After updating the version, commit the changes: -```bash -git add mix.exs native.ex -git commit -m "Bump version to 0.1.1" -git push origin main -``` +- Open a **Pull Request** including: + - Code changes (including the version bump in mix.exs) + - A CHANGELOG.md entry for this release. ---- +Once approved and CI passes - **merge to `main`**. + +## **3️⃣ Tag & Create a GitHub Release** + +1. To create a release, check out the latest `origin/main` (this should be what you just merged), tag the new version and push it: + + ```sh + git tag -a vX.Y.Z -m "Release version X.Y.Z" + git push origin vX.Y.Z + ``` + + Ensure the tag (`vX.Y.Z`) matches the version in `mix.exs`, otherwise, precompiled binaries might not align correctly. -## **4. Tag the New Version** -To create a release, tag the new version and push it: -```bash -git tag -a v0.1.1 -m "Release version 0.1.1" -git push origin v0.1.1 +2. **Go to GitHub → Releases → Create New Release** + + - Use `vX.Y.Z` as the tag. + - Copy the corresponding section from CHANGELOG.md as the release notes. + +This triggers the `release.yml` GitHub Action, which: + +- ✅ Builds precompiled NIFs. +- ✅ Uploads them to GitHub Releases. +- ✅ Makes them available for `RustlerPrecompiled`. + +## **4️⃣ Publish to Hex.pm** + +Once the GitHub release is live, publish the package to Hex.pm: + +```sh +mix hex.user auth # Authenticate if not already logged in +mix hex.publish # Publish the package ``` -This will trigger the GitHub Actions release workflow, compiling and uploading precompiled NIFs. ---- +### Important Notes: +- You **must have write access** to Zappi's Hex registry to publish. +- **Ensure you are on the `main` branch**, and it matches `origin/main`, as `mix hex.publish` publishes from your **local copy**, not GitHub. -## **5. Monitor the Release Workflow** -- Go to **GitHub Actions** and check the **Compile & Upload NIFs** workflow. -- Verify that all jobs (for Linux and macOS targets) succeed. +## **6️⃣ Verify Installation** ---- +To confirm users can fetch precompiled NIFs **without needing Rust**, test in a fresh clone: -## **5. Verify the Release** -Once the workflow completes: -- Go to [GitHub Releases](https://github.com/intellection/exstatic/releases) and confirm that the NIF artifacts are available. -- Download and test the precompiled NIFs using: -```elixir -EXSTATIC_BUILD=false mix rustler_precompiled.download Exstatic.Native --all --print +```sh +git clone https://github.com/Intellection/exstatic.git +cd exstatic +mix deps.get +mix compile # Should download precompiled NIFs instead of compiling Rust ``` + +If this works **without recompiling Rust**, the release is successful! 🎉 --- From bd2bc49ff2bffe398fd8b093cfd54938e69857dd Mon Sep 17 00:00:00 2001 From: Brook <71849503+Primebrook@users.noreply.github.com> Date: Sat, 8 Feb 2025 20:35:12 +0000 Subject: [PATCH 4/9] checksum docs --- docs/checksum.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 docs/checksum.md diff --git a/docs/checksum.md b/docs/checksum.md new file mode 100644 index 0000000..006587c --- /dev/null +++ b/docs/checksum.md @@ -0,0 +1,35 @@ +## How the Checksum is Used + +The CI/CD pipeline generates the `checksum-Elixir.Exstatic.Native.exs` file and uses it to verify the integrity of the precompiled NIFs. Here's how it works: + +### Where is the Checksum File Used? + +#### During CI Builds +When the GitHub Actions workflow runs on a new release tag, it: +1. Compiles the Rust NIFs for different architectures. +2. Computes a SHA-256 checksum for each compiled binary. +3. Generates the `checksum-Elixir.Exstatic.Native.exs` file. +4. Uploads the binaries and the checksum file to GitHub Releases. + +#### When Users Install Exstatic +When someone installs Exstatic, RustlerPrecompiled: +1. Downloads the appropriate precompiled NIF. +2. Verifies it against the `checksum-Elixir.Exstatic.Native.exs` file (which is in the release too). +3. If the checksum matches, the NIF is used. +4. If the checksum doesn't (e.g., due to corruption), compilation falls back to building the Rust code locally. + +### Example `checksum-Elixir.Exstatic.Native.exs` File +Below is an example of what the `checksum-Elixir.Exstatic.Native.exs` file looks like for multiple architectures: + +```elixir +%{ + "libexstatic-v0.1.1-nif-2.16-aarch64-apple-darwin.so.tar.gz" => "a2f7de6bae6f0562649f6e1c0a376978623f50a0be17aec92928355a534a906a", + "libexstatic-v0.1.1-nif-2.16-x86_64-apple-darwin.so.tar.gz" => "b5c8d1e2f12a4747d03c4d92e913c8b9d5b8ea2c93a31492a123456789abcdef", + "libexstatic-v0.1.1-nif-2.16-aarch64-unknown-linux-gnu.so.tar.gz" => "c3d7e5f8a1b2c34d56e7f890123abcde0987654321f4e56789abcdef01234567", + "libexstatic-v0.1.1-nif-2.16-x86_64-unknown-linux-gnu.so.tar.gz" => "d4e5f6a7b8c9d01234567890abcdef1234567890abcdef1234567890abcdef12" +} +``` + +Each entry in the map corresponds to a precompiled NIF binary, with its respective SHA-256 checksum. These checksums ensure the integrity of the binaries when downloaded and used in different environments. + +By verifying these checksums, `RustlerPrecompiled` ensures that users receive the correct, untampered precompiled binaries for their system. From 6772dca983f2a28cb5b00b55fc83c99ffb9c9088 Mon Sep 17 00:00:00 2001 From: Brook <71849503+Primebrook@users.noreply.github.com> Date: Sat, 8 Feb 2025 20:46:30 +0000 Subject: [PATCH 5/9] github PAT guide --- docs/github_token_guide.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 docs/github_token_guide.md diff --git a/docs/github_token_guide.md b/docs/github_token_guide.md new file mode 100644 index 0000000..2f95b14 --- /dev/null +++ b/docs/github_token_guide.md @@ -0,0 +1,37 @@ +# Setting Up a GitHub Personal Access Token for Exstatic + +This guide walks you through creating a **GitHub Personal Access Token (PAT)** to allow Exstatic to authenticate and fetch precompiled NIFs from a private repository. + +## 1️⃣ Generate a New Token + +1. **Go to GitHub Token Settings:** + - Navigate to [GitHub Personal Access Tokens](https://github.com/settings/tokens). + - Click **"Generate new token (classic)"**. + +## 2️⃣ Configure Token Permissions + +1. **Token Name:** + - Give it a meaningful name like `exstatic-release-access`. + +2. **Expiration:** + - Choose an expiration date. + +3. **Select Scopes:** + - The following scopes are required for accessing private repositories: + + ✅ **repo** → Full read access to private repositories + ✅ **read:packages** → Read access to GitHub Packages (under write:packages) + +## 3️⃣ Generate and Store the Token + +1. Click **"Generate token"**. +2. **Copy the token** – it will not be shown again. + +--- + +### 4️⃣ Set Environment Variable +1. Exstatic needs this to during installation: + + ```sh + export EXSTATIC_GITHUB_TOKEN="your-token-here" + ``` From 6247a9c9f23220d74f5988440637b07e865cd221 Mon Sep 17 00:00:00 2001 From: Brook <71849503+Primebrook@users.noreply.github.com> Date: Sat, 8 Feb 2025 20:46:52 +0000 Subject: [PATCH 6/9] obselete --- hehe.md | 25 ------------------------- 1 file changed, 25 deletions(-) delete mode 100644 hehe.md diff --git a/hehe.md b/hehe.md deleted file mode 100644 index f8d6b25..0000000 --- a/hehe.md +++ /dev/null @@ -1,25 +0,0 @@ -1. Go to GitHub Token Settings -Open GitHub Personal Access Tokens -Click "Generate new token (classic)". -2. Configure Token Permissions -Token Name: -Give it a meaningful name like `exstatic-release-access`. - -Expiration: -Choose "No expiration" or a time period based on your security policy. - -Select Scopes: -For private repositories, you need: - -✅ repo → Full read access to private repositories - -This includes: -repo:status -repo_deployment -public_repo -repo:invite -✅ read:packages - -3. Generate and Copy the Token -Click "Generate token". -Copy the token immediately (you won’t see it again!). From 3ec549324eb22021d2f8e8984ae5423a84fc60ac Mon Sep 17 00:00:00 2001 From: Brook <71849503+Primebrook@users.noreply.github.com> Date: Sat, 8 Feb 2025 20:51:25 +0000 Subject: [PATCH 7/9] you can go --- docs/checksum.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/checksum.md b/docs/checksum.md index 006587c..563e610 100644 --- a/docs/checksum.md +++ b/docs/checksum.md @@ -31,5 +31,3 @@ Below is an example of what the `checksum-Elixir.Exstatic.Native.exs` file looks ``` Each entry in the map corresponds to a precompiled NIF binary, with its respective SHA-256 checksum. These checksums ensure the integrity of the binaries when downloaded and used in different environments. - -By verifying these checksums, `RustlerPrecompiled` ensures that users receive the correct, untampered precompiled binaries for their system. From 00975d525f54038f9cceaf71e69e224cf840ecbb Mon Sep 17 00:00:00 2001 From: Brook <71849503+Primebrook@users.noreply.github.com> Date: Sat, 8 Feb 2025 21:26:48 +0000 Subject: [PATCH 8/9] forgot checksum generation step in release guide --- RELEASE.md | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/RELEASE.md b/RELEASE.md index 79fc46f..233b9aa 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -38,9 +38,27 @@ This triggers the `release.yml` GitHub Action, which: - ✅ Uploads them to GitHub Releases. - ✅ Makes them available for `RustlerPrecompiled`. -## **4️⃣ Publish to Hex.pm** +## **4️⃣ Generate & Include Checksum File** -Once the GitHub release is live, publish the package to Hex.pm: +**After the GitHub release is live and the CI has built the precompiled NIFs**, you need to generate the checksum file: + +```sh +mix rustler_precompiled.download Exstatic.Native --all --print +``` + +This pulls the NIF binaries from the latest GitHub release and uses them to generate a `checksum-Elixir.Exstatic.Native.exs` file. + +Run the following command to verify the package contents before publishing: + +```sh +mix hex.build --unpack +``` + +You should see `checksum-Elixir.Exstatic.Native.exs` included in th list of files. Don't check the checksum file into version control. + +## **5️⃣ Publish to Hex.pm** + +Once the GitHub release is live and the checksum file is included, publish the package to Hex.pm: ```sh mix hex.user auth # Authenticate if not already logged in @@ -48,7 +66,7 @@ mix hex.publish # Publish the package ``` ### Important Notes: -- You **must have write access** to Zappi's Hex registry to publish. +- You **must have write access** to Zappi's Hex registry to publish (ask Brendon if you don't). - **Ensure you are on the `main` branch**, and it matches `origin/main`, as `mix hex.publish` publishes from your **local copy**, not GitHub. ## **6️⃣ Verify Installation** @@ -63,4 +81,3 @@ mix compile # Should download precompiled NIFs instead of compiling Rust ``` If this works **without recompiling Rust**, the release is successful! 🎉 ---- From 6d2036c84f129d9f8d84eb9c1746f3697bb14315 Mon Sep 17 00:00:00 2001 From: Brook <71849503+Primebrook@users.noreply.github.com> Date: Sat, 8 Feb 2025 21:32:47 +0000 Subject: [PATCH 9/9] improvements in checksum doc --- docs/checksum.md | 56 +++++++++++++++++++++++++++++++----------------- 1 file changed, 36 insertions(+), 20 deletions(-) diff --git a/docs/checksum.md b/docs/checksum.md index 563e610..988a14f 100644 --- a/docs/checksum.md +++ b/docs/checksum.md @@ -1,33 +1,49 @@ -## How the Checksum is Used +# **Checksum in Exstatic** -The CI/CD pipeline generates the `checksum-Elixir.Exstatic.Native.exs` file and uses it to verify the integrity of the precompiled NIFs. Here's how it works: +The `checksum-Elixir.Exstatic.Native.exs` file is used to verify the integrity of the precompiled NIFs when users install Exstatic. This ensures that the correct binaries are downloaded and prevents tampering or corruption. -### Where is the Checksum File Used? +## **How the Checksum is Used** -#### During CI Builds -When the GitHub Actions workflow runs on a new release tag, it: -1. Compiles the Rust NIFs for different architectures. -2. Computes a SHA-256 checksum for each compiled binary. -3. Generates the `checksum-Elixir.Exstatic.Native.exs` file. -4. Uploads the binaries and the checksum file to GitHub Releases. +### **Where is the Checksum File Used?** -#### When Users Install Exstatic -When someone installs Exstatic, RustlerPrecompiled: -1. Downloads the appropriate precompiled NIF. -2. Verifies it against the `checksum-Elixir.Exstatic.Native.exs` file (which is in the release too). -3. If the checksum matches, the NIF is used. -4. If the checksum doesn't (e.g., due to corruption), compilation falls back to building the Rust code locally. +#### **During Release Process** +1. The precompiled NIFs are built by **GitHub Actions** when a new release is created. +2. Once all precompiled binaries are available in the GitHub release, the checksum file must be **manually generated** using: + ```sh + mix rustler_precompiled.download Exstatic.Native --all --print + ``` +3. This command fetches all precompiled binaries and generates a `checksum-Elixir.Exstatic.Native.exs` file. +4. The generated checksum file is **included in the Hex package** but is **not committed to the repository**. -### Example `checksum-Elixir.Exstatic.Native.exs` File +#### **When Users Install Exstatic** +1. RustlerPrecompiled automatically downloads the appropriate precompiled NIF. +2. The NIF's SHA-256 checksum is **verified** against the `checksum-Elixir.Exstatic.Native.exs` file. +3. If the checksum matches, the precompiled binary is used. +4. If the checksum does **not** match (e.g., due to corruption or missing binary), Rustler falls back to compiling the Rust code locally. + +--- + +## **Example `checksum-Elixir.Exstatic.Native.exs` File** Below is an example of what the `checksum-Elixir.Exstatic.Native.exs` file looks like for multiple architectures: ```elixir %{ - "libexstatic-v0.1.1-nif-2.16-aarch64-apple-darwin.so.tar.gz" => "a2f7de6bae6f0562649f6e1c0a376978623f50a0be17aec92928355a534a906a", - "libexstatic-v0.1.1-nif-2.16-x86_64-apple-darwin.so.tar.gz" => "b5c8d1e2f12a4747d03c4d92e913c8b9d5b8ea2c93a31492a123456789abcdef", - "libexstatic-v0.1.1-nif-2.16-aarch64-unknown-linux-gnu.so.tar.gz" => "c3d7e5f8a1b2c34d56e7f890123abcde0987654321f4e56789abcdef01234567", - "libexstatic-v0.1.1-nif-2.16-x86_64-unknown-linux-gnu.so.tar.gz" => "d4e5f6a7b8c9d01234567890abcdef1234567890abcdef1234567890abcdef12" + "libexstatic-v0.1.1-nif-2.16-aarch64-apple-darwin.so.tar.gz" => "sha256:a2f7de6bae6f0562649f6e1c0a376978623f50a0be17aec92928355a534a906a", + "libexstatic-v0.1.1-nif-2.16-x86_64-apple-darwin.so.tar.gz" => "sha256:b5c8d1e2f12a4747d03c4d92e913c8b9d5b8ea2c93a31492a123456789abcdef", + "libexstatic-v0.1.1-nif-2.16-aarch64-unknown-linux-gnu.so.tar.gz" => "sha256:c3d7e5f8a1b2c34d56e7f890123abcde0987654321f4e56789abcdef01234567", + "libexstatic-v0.1.1-nif-2.16-x86_64-unknown-linux-gnu.so.tar.gz" => "sha256:d4e5f6a7b8c9d01234567890abcdef1234567890abcdef1234567890abcdef12" } ``` Each entry in the map corresponds to a precompiled NIF binary, with its respective SHA-256 checksum. These checksums ensure the integrity of the binaries when downloaded and used in different environments. + +--- + +## **Important Notes** +- The checksum file **must be generated manually** after the precompiled NIFs are available in the GitHub release. +- The file must be **included in the Hex package** by ensuring `"checksum-*.exs"` is listed in the `files` field of `mix.exs`. +- You can verify that the checksum file is included before publishing with: + ```sh + mix hex.build --unpack + ``` +- If any **native Rust code changes**, you **must** regenerate the checksum file before the next release.