Skip to content

Enable RustlerPrecompiled to Fetch NIFs from Exstatic (private repo) #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Primebrook merged 7 commits into from
Feb 8, 2025
Merged

Enable RustlerPrecompiled to Fetch NIFs from Exstatic (private repo) #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
18d30cc
release helper
Primebrook Feb 8, 2025
10ff8b3
docs
Primebrook Feb 8, 2025
2bdf0c7
credo
Primebrook Feb 8, 2025
922455a
Tesla > Req
Primebrook Feb 8, 2025
f30b9ce
changelog entry for 0.1.0
Primebrook Feb 8, 2025
84a2813
changelog entry for 0.1.1 release
Primebrook Feb 8, 2025
bea6f10
add links
Primebrook Feb 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 32 additions & 2 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,35 @@
# Changelog

## v0.1.0
All notable changes to this project will be documented in this file.

- Initial release.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [v0.1.1] - 2024-02-07

### Added

- **RustlerPrecompiled Integration:**
- Updated `Exstatic.Native` to use RustlerPrecompiled instead of requiring users to build the Rust code themselves.
- Precompiled NIFs are downloaded from GitHub Releases, reducing the need for local Rust dependencies.
- Defined `nif_versions: ["2.16"]`.

- **GitHub Actions CI/CD for Precompiled NIF Builds (`release.yml`):**
- Builds NIFs for macOS & Linux (ARM and x86_64).
- Uses [`philss/rustler-precompiled-action`](https://github.com/philss/rustler-precompiled-action) for cross-compilation and upload.
- Automatically triggers on:
- New GitHub tags (e.g., `v0.1.0`).
- Main branch pushes.
- PRs that modify native code or the `release.yml` file.
- Uploads precompiled NIFs to GitHub Releases when a tag is pushed.

## [v0.1.0] - 2024-02-07

### Added
Initial Release: Introduced Exstatic, a statistical distribution library for Elixir with native Rust implementations.

[Unreleased]: https://github.com/Intellection/exstatic/compare/v0.1.1...HEAD
[v0.1.1]: https://github.com/Intellection/exstatic/compare/v0.1.1...v0.1.0
[v0.1.0]: https://github.com/Intellection/exstatic/releases/tag/v0.1.0
6 changes: 3 additions & 3 deletions lib/exstatic/native.ex
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
defmodule Exstatic.Native do
@moduledoc false
config = Mix.Project.config()
version = Mix.Project.config()[:version]

use RustlerPrecompiled,
otp_app: :exstatic,
crate: "exstatic",
base_url: "#{config[:source_url]}/releases/download/v#{config[:version]}",
base_url: {Exstatic.ReleaseHelper, :github_release_url!},
force_build: System.get_env("EXSTATIC_BUILD") == "true",
nif_versions: ["2.16"],
version: config[:version],
version: version,
targets: ~w(
aarch64-apple-darwin
x86_64-apple-darwin
Expand Down
97 changes: 97 additions & 0 deletions lib/exstatic/release_helper.ex
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
defmodule Exstatic.ReleaseHelper do
@moduledoc """
A helper module for fetching precompiled assets from a private GitHub release.

This module is designed to work with `RustlerPrecompiled`, allowing it to dynamically
fetch the correct precompiled NIF binaries from a GitHub release using the GitHub API.
It retrieves the asset ID for a given file name and constructs a download URL with the
necessary authentication headers.

## Required Environment Variables

- `EXSTATIC_GITHUB_TOKEN` – A GitHub personal access token (PAT) with access to private repositories.
"""

@config Mix.Project.config()
@tag "v#{@config[:version]}"

@doc """
Retrieves the GitHub asset download URL and authentication headers.

## Parameters

- `file_name` (String.t()): The name of the asset file to download.

## Returns

- `{download_url, headers}` (tuple): A tuple containing the GitHub asset URL and headers.

## Raises

- `RuntimeError`: If the `EXSTATIC_GITHUB_TOKEN` environment variable is missing or if the asset ID cannot be found.

## Example

iex> Exstatic.ReleaseHelper.github_release_url!("libexstatic-v0.1.1-nif-2.16-aarch64-apple-darwin.so.tar.gz")
{"https://api.github.com/repos/Intellection/exstatic/releases/assets/227243824",
[{"Authorization", "token MY_TOKEN"}, {"Accept", "application/octet-stream"}, {"User-Agent", "exstatic-bot"}]}
"""
@spec github_release_url!(String.t()) :: {String.t(), [{String.t(), String.t()}]}
def github_release_url!(file_name) do
token = github_personal_access_token!()

case fetch_github_asset_id(file_name, token) do
{:ok, asset_id} ->
{asset_url(asset_id),
[
{"Authorization", "token #{token}"},
{"Accept", "application/octet-stream"},
{"User-Agent", "exstatic-bot"}
]}

{:error, reason} ->
raise "Failed to fetch GitHub asset ID: #{reason}"
end
end

defp fetch_github_asset_id(file_name, token) do
headers = [
{"Authorization", "token #{token}"},
{"Accept", "application/vnd.github+json"},
{"User-Agent", "exstatic-bot"}
]

case Tesla.get(release_url(), headers: headers) do
{:ok, %Tesla.Env{status: 200, body: release}} ->
release
|> :json.decode()
|> find_asset_id(file_name)

{:ok, %Tesla.Env{status: status, body: body}} ->
{:error, "Failed to fetch releases: #{status} #{inspect(body)}"}

{:error, reason} ->
{:error, reason}
end
end

defp asset_url(asset_id) do
"https://api.github.com/repos/#{@config[:repo]}/releases/assets/#{asset_id}"
end

defp release_url, do: "https://api.github.com/repos/#{@config[:repo]}/releases/tags/#{@tag}"

defp find_asset_id(release, file_name) do
case Enum.find(release["assets"], fn asset -> asset["name"] == file_name end) do
nil -> {:error, "Asset not found: #{file_name} in release #{@tag}"}
asset -> {:ok, asset["id"]}
end
end

defp github_personal_access_token! do
case System.get_env("EXSTATIC_GITHUB_TOKEN") do
nil -> raise "Missing EXSTATIC_GITHUB_TOKEN environment variable."
token -> token
end
end
end
12 changes: 9 additions & 3 deletions mix.exs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@ defmodule Exstatic.MixProject do
use Mix.Project

@version "0.1.1"
@source_url "https://github.com/intellection/exstatic"
@repo "Intellection/exstatic"
@source_url "https://github.com/#{@repo}"

def project do
[
Expand All @@ -18,10 +19,12 @@ defmodule Exstatic.MixProject do
package: package(),
name: "Exstatic",
source_url: @source_url,
repo: @repo,
test_coverage: [
summary: [threshold: 90],
ignore_modules: [
Exstatic.Native
Exstatic.Native,
Exstatic.ReleaseHelper
]
]
]
Expand Down Expand Up @@ -75,7 +78,10 @@ defmodule Exstatic.MixProject do
{:credo, "~> 1.7", only: [:dev, :test], runtime: false},
{:dialyxir, "~> 1.4", only: [:dev, :test], runtime: false},
{:ex_doc, "~> 0.34", only: :dev, runtime: false},
{:sobelow, "~> 0.13", only: [:dev, :test], runtime: false}
{:sobelow, "~> 0.13", only: [:dev, :test], runtime: false},
{:tesla, "~> 1.13.2"},
{:jason, "~> 1.4"},
{:mint, "~> 1.0"}
]
end
end
1 change: 1 addition & 0 deletions mix.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,6 @@
"rustler_precompiled": {:hex, :rustler_precompiled, "0.8.2", "5f25cbe220a8fac3e7ad62e6f950fcdca5a5a5f8501835d2823e8c74bf4268d5", [:mix], [{:castore, "~> 0.1 or ~> 1.0", [hex: :castore, repo: "hexpm", optional: false]}, {:rustler, "~> 0.23", [hex: :rustler, repo: "hexpm", optional: true]}], "hexpm", "63d1bd5f8e23096d1ff851839923162096364bac8656a4a3c00d1fff8e83ee0a"},
"sobelow": {:hex, :sobelow, "0.13.0", "218afe9075904793f5c64b8837cc356e493d88fddde126a463839351870b8d1e", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "cd6e9026b85fc35d7529da14f95e85a078d9dd1907a9097b3ba6ac7ebbe34a0d"},
"telemetry": {:hex, :telemetry, "1.3.0", "fedebbae410d715cf8e7062c96a1ef32ec22e764197f70cda73d82778d61e7a2", [:rebar3], [], "hexpm", "7015fc8919dbe63764f4b4b87a95b7c0996bd539e0d499be6ec9d7f3875b79e6"},
"tesla": {:hex, :tesla, "1.13.2", "85afa342eb2ac0fee830cf649dbd19179b6b359bec4710d02a3d5d587f016910", [:mix], [{:castore, "~> 0.1 or ~> 1.0", [hex: :castore, repo: "hexpm", optional: true]}, {:exjsx, ">= 3.0.0", [hex: :exjsx, repo: "hexpm", optional: true]}, {:finch, "~> 0.13", [hex: :finch, repo: "hexpm", optional: true]}, {:fuse, "~> 2.4", [hex: :fuse, repo: "hexpm", optional: true]}, {:gun, ">= 1.0.0", [hex: :gun, repo: "hexpm", optional: true]}, {:hackney, "~> 1.6", [hex: :hackney, repo: "hexpm", optional: true]}, {:ibrowse, "4.4.2", [hex: :ibrowse, repo: "hexpm", optional: true]}, {:jason, ">= 1.0.0", [hex: :jason, repo: "hexpm", optional: true]}, {:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:mint, "~> 1.0", [hex: :mint, repo: "hexpm", optional: true]}, {:mox, "~> 1.0", [hex: :mox, repo: "hexpm", optional: true]}, {:msgpax, "~> 2.3", [hex: :msgpax, repo: "hexpm", optional: true]}, {:poison, ">= 1.0.0", [hex: :poison, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: true]}], "hexpm", "960609848f1ef654c3cdfad68453cd84a5febecb6ed9fed9416e36cd9cd724f9"},
"toml": {:hex, :toml, "0.7.0", "fbcd773caa937d0c7a02c301a1feea25612720ac3fa1ccb8bfd9d30d822911de", [:mix], [], "hexpm", "0690246a2478c1defd100b0c9b89b4ea280a22be9a7b313a8a058a2408a2fa70"},
}