Skip to content

Relay Agent Fails to Start For Clusters Upgraded from 0.2.0 With fingerprint mismatch #77

@estein9825

Description

@estein9825

Expected vs actual behavior

If you attempt to upgrade a paralus server from 0.2.0 with relay version 0.1.2 to 0.2.5 paralus with relay 0.1.5 (starting with 0.1.4), you'll get the error:

"level":"info","ts":"2023-08-16T23:25:59.118Z","caller":"agent/agent.go:171","msg":"Relay Agent::relay agent namespace: paralus-system  fingerprint: 4d63c263-0e39-4451-aa97-6882e339fc30  "}
{"level":"info","ts":"2023-08-16T23:25:59.119Z","caller":"agent/agent.go:394","msg":"Relay Agent::config: &{TemplateToken:cifi3glc3m5b406jcbc0 TemplateName: Scheme:https Mode: Addr:console.paralus.iherbtest.net:443 ClientID:cisqaj3ppcveb4n2bcrg ClientIP:10.7.228.141 Name:relay-agent-557f56bb69-x2rcc PrivateKey:[] CSR:[] Certificate:[] CACertificate:[] ServerHost: ServerPort:0 Fingerprint:4d63c263-0e39-4451-aa97-6882e339fc30} "}
[POST /v2/sentry/bootstrap/{templateToken}/register][500] bootstrapRegisterBootstrapAgentInternalServerError  map[code:2 message:fingerprint mismatch for token cisqaj3ppcveb4n2bcrg]
{"level":"error","ts":"2023-08-16T23:25:59.159Z","caller":"agent/agent.go:397","msg":"Relay Agent::failed to register relay agent error: [POST /v2/sentry/bootstrap/{templateToken}/register][500] bootstrapRegisterBootstrapAgentInternalServerError  map[code:2 message:fingerprint mismatch for token cisqaj3ppcveb4n2bcrg]  ","stacktrace":"github.com/paralus/relay/pkg/agent.registerRelayAgent\n\t/build/pkg/agent/agent.go:397\ngithub.com/paralus/relay/pkg/agent.handleRelayNetworks\n\t/build/pkg/agent/agent.go:606"}

The only way to fix this is to go to the database and update the fingerprint value in the sentry_bootstrap_agent table for the specific token row with the uid from the paralus-system namespace for that client (also found within the log)

Steps to reproduce the bug

  1. Create a cluster in paralus 0.2.0 or 0.2.1 (so the relay agent is at 0.1.3 or lower)
  2. Bootstrap it in and make sure it works
  3. Upgrade the paralus to 0.2.3 or higher (so relay agent is 0.1.4 or higher)
  4. Try to have the client connect and it will fail with the above issue

(optional) If you have ideas on why the bug happens or how it can be solved, please provide it here

This appears to be happening because an existing cluster does not automatically update the fingerprint if it's already bootstrapped into the database. You have to do a new bootstrapping, or update the value manually

  • I've described the bug, included steps to reproduce it, and included my environment setup with all customizations.
  • I'm using the latest version of the project.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingnewNeeds triage

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions