[BUG]: Identify event violates Segment schema (undeclared traits sent)

[NicolasMassart]

What is this about?

The Identify analytics call introduced in PR #24178 sends a very large and unfiltered set of feature flags and remote config data as user traits. This results Identify call not reaching segment (no Identify on https://app.segment.com/consensys-analytics/sources/raw_segment_metamask_mobile_dev/debugger)

Plus if they did, it would be noisy, hard-to-use analytics data and potential performance concerns.

Additionally, Identify event traits must be explicitly declared in the Segment tracking plan schema. Any trait sent that is not defined in the MetaMask Mobile Segment schema is undocumented, unvalidated, and will eventually be filtered out and unavailable in downstream data stores.

Schema reference:
https://github.com/Consensys/segment-schema/blob/main/tracking-plans/metamask-mobile.yaml#L24

Scenario

• GIVEN the app initializes analytics
• WHEN an Identify event is sent
• THEN only meaningful, intentional, schema-declared, and analytics-usable traits should be included

Design

No response

Technical Details

• Current behavior

  • The Identify event includes the full feature flag / remote config object.
  • Payload contains deeply nested objects and large maps (e.g. smartTransactionsNetworks, per-chain configs, allowlists, blocklists).
  • Many traits resolve to [Object] in analytics tools and are not queryable or actionable.
  • Traits are sent without being declared in the Segment tracking plan and will not be retained long term.

• Concerns

  • Analytics noise and unusable traits.
  • Increased payload size for Identify calls.
  • Segment tracking plan contract is not enforced.
  • Traits not declared in the schema will be dropped and unavailable for analysis.

• Suggested direction

  • Explicitly whitelist and send only actively used feature flags.
  • Declare every Identify trait in the Segment schema.
  • Exclude remote config and large structured objects.
  • Add traits individually instead of passing entire objects.
  • Align expectations on schema and usage.

Threat Modeling Framework

• What are we working on?
  Reducing and formalizing Identify analytics payloads to ensure signal > noise and schema compliance.

• What can go wrong?

  • Over-collection of irrelevant or undocumented data
  • Loss of data due to schema filtering
  • Silent schema drift

• What are we going to do about it?

  • Enforce schema alignment
  • Reduce and explicitly define traits
  • Add cross-team review where needed

• Did we do a good job?
  Confirmed when all Identify traits are declared, retained, and usable in analytics.

Acceptance Criteria

• Identify event payload contains only whitelisted traits.
• Every Identify trait is declared in the Segment tracking plan.
• Undeclared traits are not sent.
• Remote config and large nested objects are excluded.
• Traits are queryable and retained in analytics data stores.
• No regression in analytics behavior or app stability.

Stakeholder review needed before the work gets merged

• Engineering (needed in most cases)
• Design
• Product
• QA (automation tests are required to pass before merging PRs but not all changes are covered by automation tests - please review if QA is needed beyond automation tests)
• Security
• Legal
• Marketing
• Management (please specify)
• Data team

References

• PR: feat: add feature-flags to user traits #24178
• Segment schema: https://github.com/Consensys/segment-schema/blob/main/tracking-plans/metamask-mobile.yaml#L24
• Slack thread: https://consensys.slack.com/archives/C087GPTR5HQ/p1767635370333169

[NicolasMassart]

Implementation Proposal: Feature Flag Allowlist

After investigation, the root cause is that the current implementation sends the entire rawFeatureFlags object, which includes:

• Simple boolean/string flags (useful for analytics)
• Complex nested objects (e.g., smartTransactionsNetworks, bridgeConfig, cardSupportedCountries)
• Large arrays and maps (allowlists, blocklists, per-chain configs)
• Remote config data that's not useful for user segmentation

This causes:

1. Payload size violations - Segment rejects or corrupts profiles when payloads are too large
2. Schema violations - Undeclared traits will be filtered out by Segment
3. Unusable data - Complex objects appear as [Object] in analytics tools

---

Proposed Solution: Explicit Allowlist

Approach: Maintain a curated list of feature flags that are:

• Useful for user segmentation
• Declared in Segment schema
• Simple types (boolean, string, number)

Implementation:

Create a new utility file: app/util/metrics/FeatureFlagTraits.ts

/**
 * Allowlist of feature flags to include in Identify traits
 * Each flag must be declared in Segment schema before adding here
 * 
 * Process for adding a new flag:
 * 1. Add trait definition to Segment schema
 * 2. Get schema approval from data team
 * 3. Add flag to this allowlist
 * 4. Update tests
 */
const FEATURE_FLAG_TRAITS_ALLOWLIST = [
  'perpsPerpTradingEnabled',
  'perpsOrderBookEnabled',
  'rewardsEnabled',
  'earnStablecoinLendingEnabled',
  'earnPooledStakingEnabled',
  // Add more as needed, but only after schema declaration
] as const;

/**
 * Extract allowlisted feature flags as simple traits
 * Converts complex flag structures to simple boolean/string values
 * 
 * @param rawFeatureFlags - All feature flags from RemoteFeatureFlagController
 * @returns Filtered traits object with only allowlisted flags as simple values
 */
export const extractFeatureFlagTraits = (
  rawFeatureFlags: Record<string, unknown>,
): Record<string, string | boolean | number> => {
  const traits: Record<string, string | boolean | number> = {};
  
  for (const flagKey of FEATURE_FLAG_TRAITS_ALLOWLIST) {
    const flagValue = rawFeatureFlags[flagKey];
    if (flagValue === undefined) continue;
    
    // Extract simple value from complex structures
    if (typeof flagValue === 'boolean') {
      traits[flagKey] = flagValue;
    } else if (typeof flagValue === 'string') {
      traits[flagKey] = flagValue;
    } else if (typeof flagValue === 'number') {
      traits[flagKey] = flagValue;
    } else if (
      typeof flagValue === 'object' &&
      flagValue !== null &&
      'enabled' in flagValue
    ) {
      // Extract enabled from version-gated flags: { enabled: true, minimumVersion: "7.60.0" }
      traits[flagKey] = (flagValue as { enabled: boolean }).enabled;
    }
  }
  
  return traits;
};

Update FeatureFlagOverrideContext.tsx:

import { extractFeatureFlagTraits } from '../../util/metrics/FeatureFlagTraits';

// Replace the commented-out useEffect with:
useEffect(() => {
  if (rawFeatureFlags && Object.keys(rawFeatureFlags).length > 0) {
    const filteredTraits = extractFeatureFlagTraits(rawFeatureFlags);
    if (Object.keys(filteredTraits).length > 0) {
      addTraitsToUser(filteredTraits);
    }
  }
}, [rawFeatureFlags, addTraitsToUser]);

Benefits:

• ✅ Explicit control over what's sent
• ✅ Easy to audit and maintain
• ✅ Ensures schema compliance
• ✅ Small, predictable payload size
• ✅ Prevents payload size violations

Requirements:

• ⚠️ Requires manual maintenance of allowlist
• ⚠️ Need to update both code and schema when adding flags
• ⚠️ All flags in allowlist must be declared in Segment schema first

---

⚠️ REMINDER: Schema Declaration Process

CRITICAL: Before adding any flag to the allowlist, you MUST follow this process:

1. Add trait definition to Segment schema

   • File: https://github.com/Consensys/segment-schema/blob/main/tracking-plans/metamask-mobile.yaml
   • Add the trait under the Identify event traits section
   • Define the trait type (boolean, string, number) and description

2. Get schema approval from data team

   • Submit PR to segment-schema repository
   • Wait for data team review and approval
   • Ensure trait is merged and deployed

3. Add flag to FEATURE_FLAG_TRAITS_ALLOWLIST

   • Only after schema is approved and merged
   • Add flag key to the allowlist array
   • Update code comments if needed

4. Update tests

   • Add tests for the new flag in traits extraction
   • Verify the flag is correctly extracted and sent

Why this matters:

• Traits not declared in the schema will be filtered out by Segment
• Undeclared traits are not retained in downstream data stores
• This defeats the purpose of sending the trait in the first place
• Schema compliance ensures data is available for analysis

This ensures all traits are declared, validated, and retained in analytics data stores.