Skip to content

Conversation

@seaona
Copy link
Member

@seaona seaona commented Oct 28, 2025

Description

The following specs are flaky at the same point:

  1. Token List Sorting should sort tokens alphabetically and by decreasing balance (failed 2 times, 6 retries) - last log
  2. Token List should not show percentage increase for an ERC20 token without prices available (failed 2 times, 5 retries) - last log
  3. Token Details shows details for an ERC20 token with prices available (failed 2 times, 5 retries) - last log
  4. Send ERC20 it should be possible to send ERC20 token (failed 1 time, 1 retry) - last log

If we login to the Homepage and very fast click Import, the wallet breaks with the error can't convert undefined to object. This is a race condition on the wallet side (issue here).

However, we can fix the test by waiting deterministically for the multichain asset list (whichever we have) to be loaded, before clicking import. This solves the issue and it's mitigated on the page class, so all tests are indirectly more stable because of this now.

See repro:

broken-import.mov

Open in GitHub Codespaces

Changelog

CHANGELOG entry:

Related issues

Fixes:

Manual testing steps

  1. Check ci

Screenshots/Recordings

Before

After

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Note

Ensure e2e token import flows wait for the multichain asset list to load to reduce flakiness.

  • E2E (asset list page object):
    • Add multichainTokenListButton selector.
    • Prepend deterministic waits for multichainTokenListButton before initiating import flows in importCustomTokenByChain, importTokenBySearch, and importMultipleTokensBySearch to ensure asset list is loaded.

Written by Cursor Bugbot for commit 5c4023d. This will update automatically on new commits. Configure here.

@metamaskbot metamaskbot added the team-qa QA team label Oct 28, 2025
@seaona seaona self-assigned this Oct 28, 2025
@metamaskbot
Copy link
Collaborator

metamaskbot commented Oct 28, 2025

✨ Files requiring CODEOWNER review ✨

🧪 @MetaMask/qa (1 files, +7 -0)
  • 📁 test/
    • 📁 e2e/
      • 📁 page-objects/
        • 📁 pages/
          • 📁 home/
            • 📄 asset-list.ts +7 -0

@seaona seaona added flaky tests no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed labels Oct 28, 2025
@seaona seaona marked this pull request as ready for review October 28, 2025 11:42
@seaona seaona requested a review from a team as a code owner October 28, 2025 11:42
@Gudahtt
Copy link
Member

Gudahtt commented Oct 28, 2025

I wonder what the root cause of the failure is. Most initialization steps should be completed by the time the UI loads, I'm not aware of anything else in the background that would be possible to call "too early"

@metamaskbot
Copy link
Collaborator

📊 Page Load Benchmark Results

Current Commit: 3121209 | Date: 10/28/2025

📄 Localhost MetaMask Test Dapp

Samples: 100

Summary

  • pageLoadTime-> current mean value: 1.04s (±52ms) 🟡 | historical mean value: 1.04s ⬇️ (historical data)
  • domContentLoaded-> current mean value: 724ms (±50ms) 🟢 | historical mean value: 729ms ⬇️ (historical data)
  • firstContentfulPaint-> current mean value: 77ms (±12ms) 🟢 | historical mean value: 79ms ⬇️ (historical data)
📈 Detailed Results
Metric Mean Std Dev Min Max P95 P99
pageLoadTime 1.04s 52ms 1.01s 1.33s 1.08s 1.33s
domContentLoaded 724ms 50ms 698ms 999ms 771ms 999ms
firstPaint 77ms 12ms 60ms 188ms 84ms 188ms
firstContentfulPaint 77ms 12ms 60ms 188ms 84ms 188ms
largestContentfulPaint 0ms 0ms 0ms 0ms 0ms 0ms

Results generated automatically by MetaMask CI

@metamaskbot
Copy link
Collaborator

Builds ready [3121209]
UI Startup Metrics (1309 ± 85 ms)
PlatformBuildTypePageMetricMean (ms)Min (ms)Max (ms)Std Dev (ms)P 75 (ms)P 95 (ms)
ChromeBrowserifyStandard HomeuiStartup1309115215558513601462
load111997913298211721260
domContentLoaded111297013218111641249
domInteractive201477111945
firstPaint65892130445511281247
backgroundConnect2352202637238246
firstReactRender31206683447
getState20889102338
initialActions71689720
loadScripts8847301093819361024
setupStore1062231217
BrowserifyPower User HomeuiStartup24071695458777533054587
load1181974175329215611753
domContentLoaded1172969174729315541747
domInteractive301668185168
firstPaint686164171444710221714
backgroundConnect293227793134320793
firstReactRender27243222932
getState23419837452233374
initialActions122107258107
loadScripts919738141225512501412
setupStore1292851228
WebpackStandard HomeuiStartup8517321123798601022
load63958895772646845
domContentLoaded63158294871639830
domInteractive15115281439
firstPaint19754887202203698
backgroundConnect23125182746
firstReactRender27176383136
getState1253041418
initialActions3015348
loadScripts62858093769637819
setupStore1052441215
WebpackPower User HomeuiStartup17981355242832621662428
load71162310751278191075
domContentLoaded679608955101734955
domInteractive221259152459
firstPaint48910410623217881062
backgroundConnect1023130696177306
firstReactRender25223322533
getState19515822414197224
initialActions13159192459
loadScripts67560694597721945
setupStore2081082425108
FirefoxBrowserifyStandard HomeuiStartup14691288190310915221703
load1250111013987713081392
domContentLoaded1250111013987713081391
domInteractive1163336049127204
firstPaint------
backgroundConnect4024148164564
firstReactRender25204852641
getState11426731714
initialActions3126336
loadScripts1225108913777512721363
setupStore1275451217
BrowserifyPower User HomeuiStartup29692563403642832664036
load1470136516898815261689
domContentLoaded1470136516898815251689
domInteractive16411026857238268
firstPaint------
backgroundConnect27060649174394649
firstReactRender473292135092
getState1748733253193332
initialActions12182211082
loadScripts1406127916089314781608
setupStore3861253057125
WebpackStandard HomeuiStartup16561455253415416941897
load14221256177510914851668
domContentLoaded14211256177510914851667
domInteractive1193351281113363
firstPaint------
backgroundConnect4922175255597
firstReactRender2923133132944
getState946110837
initialActions41556412
loadScripts13891215174010814561630
setupStore14774121350
WebpackPower User HomeuiStartup29842418426154534924261
load16421268209825319332098
domContentLoaded16421267209825319322098
domInteractive1537334768182347
firstPaint------
backgroundConnect23748871272437871
firstReactRender42325984959
getState1889525449244254
initialActions13175201775
loadScripts15661246193621217741936
setupStore3761695140169
Bundle size diffs [🚨 Warning! Bundle size has increased!]
  • background: 58 Bytes (0%)
  • ui: -2 Bytes (0%)
  • common: 10 Bytes (0%)

jvbriones
jvbriones previously approved these changes Oct 28, 2025
hjetpoluru
hjetpoluru previously approved these changes Oct 28, 2025
@seaona
Copy link
Member Author

seaona commented Oct 28, 2025

I wonder what the root cause of the failure is. Most initialization steps should be completed by the time the UI loads, I'm not aware of anything else in the background that would be possible to call "too early"

I'm not sure neither 🤔 Is it possible that changes on the multichain work have caused this? ie if the multichain asset list is not yet 'created', trying to Import a token might break the wallet, if we try to access it? This is a guess, but also unsure. I opened a ticket here to investigate on the wallet side: #37303

@seaona seaona dismissed stale reviews from hjetpoluru and jvbriones via 7cb1900 October 28, 2025 13:12

private readonly multichainTokenListButton = {
testId: 'multichain-token-list-button',
};
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Invalid Selector Format Causes Runtime Failures

The multichainTokenListButton selector uses an invalid format. It's defined as { testId: 'multichain-token-list-button' }, but the driver's waitForSelector method doesn't recognize the testId property. It should be formatted as a string selector '[data-testid="multichain-token-list-button"]' to match the existing pattern in this class (see tokenListItem at line 106-107). This will cause runtime failures when waitForSelector(this.multichainTokenListButton) is called in the import methods.

Fix in Cursor Fix in Web

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmm this is not correct 🤔 passing a test id key like this as a selector, does work. @HowardBraham added the logic for it here:

https://github.com/MetaMask/metamask-extension/blob/main/test/e2e/webdriver/driver.js#L238

@seaona seaona changed the title fix: flaky test Token List Sorting should sort tokens alphabetically and by decreasing balance fix: flaky test Token List..., Token Details, Send ERC20because they broke with error can't convert undefined to object Oct 28, 2025
@metamaskbot
Copy link
Collaborator

📊 Page Load Benchmark Results

Current Commit: 5c4023d | Date: 10/28/2025

📄 Localhost MetaMask Test Dapp

Samples: 100

Summary

  • pageLoadTime-> current mean value: 1.03s (±40ms) 🟡 | historical mean value: 1.04s ⬇️ (historical data)
  • domContentLoaded-> current mean value: 717ms (±37ms) 🟢 | historical mean value: 729ms ⬇️ (historical data)
  • firstContentfulPaint-> current mean value: 76ms (±13ms) 🟢 | historical mean value: 78ms ⬇️ (historical data)
📈 Detailed Results
Metric Mean Std Dev Min Max P95 P99
pageLoadTime 1.03s 40ms 1.00s 1.31s 1.08s 1.31s
domContentLoaded 717ms 37ms 694ms 984ms 747ms 984ms
firstPaint 76ms 13ms 56ms 188ms 84ms 188ms
firstContentfulPaint 76ms 13ms 56ms 188ms 84ms 188ms
largestContentfulPaint 0ms 0ms 0ms 0ms 0ms 0ms

Results generated automatically by MetaMask CI

@metamaskbot
Copy link
Collaborator

Builds ready [5c4023d]
UI Startup Metrics (1205 ± 112 ms)
PlatformBuildTypePageMetricMean (ms)Min (ms)Max (ms)Std Dev (ms)P 75 (ms)P 95 (ms)
ChromeBrowserifyStandard HomeuiStartup12051027195611212591339
load1027866163010110791170
domContentLoaded1019848161610210721163
domInteractive201464101846
firstPaint717117162640410511159
backgroundConnect19217530613195207
firstReactRender29175183545
getState1876482036
initialActions61436817
loadScripts8356631414100887983
setupStore1062231117
BrowserifyPower User HomeuiStartup24741961453873531934538
load1224980215434916392154
domContentLoaded1214974214034916312140
domInteractive36161022663102
firstPaint693180111138710421111
backgroundConnect292231731121305731
firstReactRender28243333133
getState24119141265231412
initialActions1116917669
loadScripts961749177931213341779
setupStore16849102449
WebpackStandard HomeuiStartup8617251206858761119
load64758295181649915
domContentLoaded63957794079643907
domInteractive15118691435
firstPaint20259969206194627
backgroundConnect2512139142843
firstReactRender26174773234
getState1263441420
initialActions3019347
loadScripts63657593077640895
setupStore1052031216
WebpackPower User HomeuiStartup19511619287540823712875
load75664811371278601137
domContentLoaded72163510331007741033
domInteractive231541103541
firstPaint4869710373168271037
backgroundConnect11318351126316351
firstReactRender28253942939
getState20318623713208237
initialActions15192251892
loadScripts7166321022977711022
setupStore19755153255
FirefoxBrowserifyStandard HomeuiStartup14571290195110015081622
load1247109514127112961381
domContentLoaded1246109514127112951380
domInteractive1113426140123210
firstPaint------
backgroundConnect4424276324894
firstReactRender25205752538
getState74223913
initialActions3117236
loadScripts1219108013866512691343
setupStore1176061216
BrowserifyPower User HomeuiStartup28962382406940932304069
load14411234171512515701715
domContentLoaded14401226171512615701715
domInteractive17010228972265289
firstPaint------
backgroundConnect22854656162356656
firstReactRender40305674556
getState1589623435174234
initialActions9150121350
loadScripts13761201167413014961674
setupStore37101042659104
WebpackStandard HomeuiStartup16331403216915216771956
load13931223177210714431617
domContentLoaded13921223177210714431616
domInteractive1113043270109281
firstPaint------
backgroundConnect4725191225481
firstReactRender292277102868
getState104122131025
initialActions10121232451
loadScripts13631206174310314081588
setupStore166189211356
WebpackPower User HomeuiStartup29052375406550432994065
load16241329211024618682110
domContentLoaded16231328211024618672110
domInteractive18273495117312495
firstPaint------
backgroundConnect17741870224257870
firstReactRender43301011748101
getState1629223941186239
initialActions2211453929145
loadScripts15721302201421717742014
setupStore4472335972233
Bundle size diffs [🚀 Bundle size reduced!]
  • background: 58 Bytes (0%)
  • ui: -1.59 KiB (-0.02%)
  • common: -448 Bytes (-0.01%)

Copy link
Contributor

@chloeYue chloeYue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the fix looks good to me !

Copy link
Member

@Gudahtt Gudahtt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@itsyoboieltr itsyoboieltr added this pull request to the merge queue Oct 28, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 28, 2025
@HowardBraham HowardBraham added this pull request to the merge queue Oct 28, 2025
Merged via the queue into main with commit a8a6abc Oct 28, 2025
1109 of 1122 checks passed
@HowardBraham HowardBraham deleted the flaky-import-broken branch October 28, 2025 17:49
@github-actions github-actions bot locked and limited conversation to collaborators Oct 28, 2025
@metamaskbot metamaskbot added the release-13.8.0 Issue or pull request that will be included in release 13.8.0 label Oct 28, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

flaky tests no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed release-13.8.0 Issue or pull request that will be included in release 13.8.0 size-XS team-qa QA team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants