Conversation
…acity to send transactions with the moonlight bundle operations to the Stellar network
| const feeScore = Number(bundle.fee) / 1_000_000; // Normalize by dividing by 1M | ||
|
|
||
| // Age: older bundles get higher priority | ||
| // Calculate age in hours, normalize (older = higher score) |
There was a problem hiding this comment.
Why the decision to use hours? Won't this mean that a lot within the same hour will have equal priority?
| /** | ||
| * Error thrown when a bundle is not found in the mempool | ||
| */ | ||
| export class BUNDLE_NOT_FOUND extends PlatformError<{ bundleId: string }> { |
There was a problem hiding this comment.
I'm surprised by all caps? Is that part of the project's style?
| async function parseOperationsFromBundle( | ||
| operationsMLXDR: string[] | ||
| ): Promise<Array<OperationTypes.CreateOperation | OperationTypes.SpendOperation | OperationTypes.DepositOperation | OperationTypes.WithdrawOperation>> { | ||
| const { MoonlightOperation } = await import("@moonlight/moonlight-sdk"); |
There was a problem hiding this comment.
It's been a while and I've totally forgotten all the different JS import types. Is there a reason that we need to import here vs at the top like all the rest of the imports?
| const operations = await Promise.all( | ||
| operationsMLXDR.map((xdr) => MoonlightOperation.fromMLXDR(xdr)) | ||
| ); | ||
| return operations as Array<OperationTypes.CreateOperation | OperationTypes.SpendOperation | OperationTypes.DepositOperation | OperationTypes.WithdrawOperation>; |
There was a problem hiding this comment.
Are these all possible Operation types? Could this be Array<OperationTypes>? Otherwise seems like it should be a type alias for better readability.
| const lowestPriority = findLowestPriorityBundle({ | ||
| bundles: this.bundles, | ||
| currentWeight: this.currentWeight, | ||
| capacity: this.capacity, | ||
| }); | ||
|
|
||
| if (lowestPriority && bundle.priorityScore > lowestPriority.priorityScore) { | ||
| // Replace the lowest priority bundle | ||
| const removedIndex = this.bundles.indexOf(lowestPriority); | ||
| this.bundles.splice(removedIndex, 1); | ||
| this.currentWeight -= lowestPriority.weight; | ||
|
|
||
| this.bundles.push(bundle); | ||
| this.bundles.sort(compareBundlePriority); | ||
| this.currentWeight += bundle.weight; | ||
|
|
||
| return lowestPriority; | ||
| } |
There was a problem hiding this comment.
- Given that the array is sorted by priority would this not always be the last element?
- Shouldn't you still check if
this.currentWeight - lowestPriority.weight + bundle.weight <= this.capacity?
| * Removes a specific bundle by bundleId | ||
| * Returns true if bundle was found and removed, false otherwise | ||
| */ | ||
| removeBundle(bundleId: string): boolean { |
There was a problem hiding this comment.
I need to still see this in the context where it is used but wouldn't you want to return the bundle instead of just deleting it?
|
|
||
| // If bundle still needs to be added, create a new slot | ||
| if (bundleToAdd) { | ||
| const newSlot = new Slot(this.capacity); |
There was a problem hiding this comment.
If I'm understanding this correctly there is no limit on the number of slots. So If the capacity of the mempool is greater than the new bundle it will be guaranteed to be added?
Also then shouldn't this check be done at the beginning of the method since that would mean that it couldn't fit in any of the current slots?
| private removeBundleFromSlot(slot: Slot, bundleId: string): void { | ||
| const removed = slot.removeBundle(bundleId); | ||
| if (!removed) { | ||
| LOG.warn(`Bundle ${bundleId} not found in slot for removal`); |
| * Mempool service for managing transaction queue with slots | ||
| */ | ||
| export class Mempool { | ||
| private slots: Slot[] = []; |
There was a problem hiding this comment.
Bundles within slots are sorted by priority. However, it seems to me that when adding a new bundle it is placed in the first slot with remaining capacity regardless of its priority. Is this the correct design or should the slots themselves be ordered by priority as well?
If it is the latter then I think it might be worth looking into a different datastructure where all bundles are ordered by priority and then when picking the next "slot" is just removing the first x bundles the weights of which add up to the capacity. This way if you have a new bundle with a high priority which would go into the first slot, you wouldn't need to potentially update all the multiple slots if one gets kick out of the first.
| this.slots.push(newSlot); | ||
| LOG.debug(`Bundle ${bundleData.bundleId} added to new slot`); | ||
| } else { | ||
| // Even new slot can't fit (shouldn't happen if capacity is reasonable) |
There was a problem hiding this comment.
Is this capacity statically known? Could it be a constraint on bundles such that they could never have a capacity above the limit? Or is this what you are hoping to increase overtime and is currently limited by the contract execution costs?
| } else { | ||
| // Even new slot can't fit (shouldn't happen if capacity is reasonable) | ||
| LOG.error(`Bundle ${bundleData.bundleId} cannot fit in any slot, weight: ${bundleData.weight}, capacity: ${this.capacity}`); | ||
| throw new E.SLOT_FULL(bundleData.weight, this.capacity); |
There was a problem hiding this comment.
This seems like the incorrect error since really it is that the capacity of the bundle is bigger than the max capacity of any slot.
2 participants
Asynchronous Mempool System Implementation - Phases 1-3
Summary
Initial implementation of an asynchronous mempool system for processing bundles in a queue, with slot-based grouping and priority management.
Implemented Changes
Phase 1: Preparation and Base Structure
FAILEDstatus toTransactionStatusenum0004_add_failed_status_to_transaction.sqlMEMPOOL_SLOT_CAPACITYMEMPOOL_EXPENSIVE_OP_WEIGHTMEMPOOL_CHEAP_OP_WEIGHTMEMPOOL_EXECUTOR_INTERVAL_MSMEMPOOL_VERIFIER_INTERVAL_MSMEMPOOL_TTL_CHECK_INTERVAL_MSPhase 2: Weight and Priority Calculation
calculateBundleWeight()function to calculate weight based on operation typescalculatePriorityScore()function with criteria:SlotBundle,WeightConfig,PriorityScorePhase 3: Mempool Service
Slotclass to manage bundles within a slotMempoolclass to manage transaction queuemempool.service.ts:calculateSlotWeight()canBundleFitInSlot()compareBundlePriority()isBundleExpired()findLowestPriorityBundle()findPendingOrProcessing()method toOperationsBundleRepositoryTechnical Notes
EXPIRED