[Bug]: embed/3 does not emit [:req_llm, :token_usage] telemetry — Step.Usage not attached

[stryderjzw]

Provider (if applicable)

All providers (confirmed: google)

Model (if applicable)

google:gemini-embedding-001

Bug Description

ReqLLM.embed/3 never emits the [:req_llm, :token_usage] telemetry event, unlike generate_object/generate_text. This means embedding calls are invisible to telemetry handlers — costs and token counts are silently dropped.

Reproduction Code

:telemetry.attach("test", [:req_llm, :token_usage], fn _, m, _, _ -> IO.inspect(m, label: "usage") end, nil)

# generate_text fires the event:
ReqLLM.generate_text("google:gemini-2.5-flash", "hello")
# => usage: %{tokens: %{input: 8, output: 12, ...}, total_cost: 0.000012}

# embed never fires the event:
ReqLLM.embed("google:gemini-embedding-001", "hello world")
# => (silence — no telemetry)

Expected Behavior

[:req_llm, :token_usage] fires after every embed/3 call, consistent with all other ReqLLM API calls.

Actual Behavior

No telemetry event. Embedding costs are invisible to any [:req_llm, :token_usage] handler.

Root Cause

embed/3 in lib/req_llm/embedding.ex calls Req.request(request) directly without attaching Step.Usage, unlike the chat/object pipelines which call ReqLLM.Step.Usage.attach(req, model) before executing:

# Current (no telemetry):
{:ok, request} <- provider_module.prepare_request(:embedding, model, text, provider_opts),
{:ok, %Req.Response{...} = response} <- Req.request(request),

# Fix:
{:ok, request} <- provider_module.prepare_request(:embedding, model, text, provider_opts),
{:ok, %Req.Response{...} = response} <- request |> ReqLLM.Step.Usage.attach(model) |> Req.request(),

Note on main branch

We also looked at the unreleased return_usage: true feature on main (added in #444). It reads usage from response.private[:req_llm][:usage], which is also only populated by Step.Usage — so return_usage: true will also always return nil until this is fixed.

Environment

• ReqLLM: 1.6.0
• Elixir: 1.19.1
• OTP: 28

[stryderjzw]

Hey, thanks for the fix in #498! Wanted to flag that it looks like it only covers Vertex AI embeddings. The standard Google AI provider (google:gemini-embedding-001 via ReqLLM.embed/3) still doesn't fire [:req_llm, :token_usage].

Quick repro on v1.7.1:

test_pid = self()

:telemetry.attach("embed-telemetry-test", [:req_llm, :token_usage], fn _event, measurements, metadata, _ ->
  send(test_pid, {:telemetry_fired, measurements, metadata})
end, nil)

{:ok, vector} = ReqLLM.embed("google:gemini-embedding-001", "hello world", dimensions: 768)

result = receive do
  {:telemetry_fired, m, md} -> "FIRED: #{inspect(m)} #{inspect(md)}"
after
  3000 -> "NOT FIRED (waited 3s)"
end

:telemetry.detach("embed-telemetry-test")

{length(vector), result}
# => {768, "NOT FIRED (waited 3s)"}

Happy to open a separate issue if you'd prefer.

[mikehostetler]

Opened PR #530 to fix this by preserving Google embedding usageMetadata through normalization, which restores both return_usage: true and the [:req_llm, :token_usage] telemetry path for standard embeddings.

Verification on the branch:

• mix test test/req_llm/embedding_test.exs
• mix test test/providers/google_test.exs
• mix test
• mix quality

Related upstream tracking issue for the llm_db dotenv-loading bug: agentjido/llm_db#162.