Skip to content

[Feature Request] API Models for hot and warm tiering actions #21078

@chaitanya588

Description

@chaitanya588

Is your feature request related to a problem? Please describe

This task is to capture the API models needed for capturing the Tiering API (such as hot to warm, warm to hot), status APIs and cancel APIs.

Describe the solution you'd like

REST API Specification for Hot↔Warm Tiering

1. POST /{index}/_tier/warm — Initiate Hot→Warm Tiering

Description: Initiates tiering of an index from the hot tier to the warm tier. This is a cluster-manager-level operation that triggers shard relocation from hot nodes to warm nodes.

Path Parameters:

Parameter Type Required Description
index string Yes Name of the index to tier. Must be a single index (no wildcards, no comma-separated lists). Must not be on the migration blocklist.

Query Parameters:

Parameter Type Required Default Description
timeout time No 30s Timeout for acknowledgement of the tiering request.
cluster_manager_timeout time No 30s Timeout for connecting to the cluster manager node.

Request Body: None

Success Response (200):

{
    "acknowledged": true
}

Failure Response (4xx/5xx):

{
    "error": {
        "root_cause": [
            {
                "type": "tiering_rejection_exception",
                "reason": "<rejection reason>"
            }
        ],
        "type": "tiering_rejection_exception",
        "reason": "<rejection reason>"
    },
    "status": 400
}

Validations:

  1. Index name cannot be null or empty
  2. Exactly one index must be specified (no multi-index support)
  3. Index must not be on the migration blocklist
  4. Cluster must not have global METADATA_WRITE, CREATE_INDEX, or WRITE blocks

2. POST /{index}/_tier/hot — Initiate Warm→Hot Tiering

Description: Initiates tiering of an index from the warm tier back to the hot tier. Triggers shard relocation from warm nodes to hot nodes.

Path Parameters:

Parameter Type Required Description
index string Yes Name of the index to tier. Must be a single index. Must not be on the migration blocklist.

Query Parameters:

Parameter Type Required Default Description
timeout time No 30s Timeout for acknowledgement of the tiering request.
cluster_manager_timeout time No 30s Timeout for connecting to the cluster manager node.

Request Body: None

Success Response (200):

{
    "acknowledged": true
}

Failure Response (4xx/5xx):

{
    "error": {
        "root_cause": [
            {
                "type": "tiering_rejection_exception",
                "reason": "<rejection reason>"
            }
        ],
        "type": "tiering_rejection_exception",
        "reason": "<rejection reason>"
    },
    "status": 400
}

Validations: Same as Hot→Warm tiering.


3. POST /_tier/_cancel/{index} — Cancel Tiering Operation

Description: Cancels an in-progress tiering operation for an index. Useful for recovering from tiering operations stuck in RUNNING_SHARD_RELOCATION state.

Path Parameters:

Parameter Type Required Description
index string Yes Name of the index for which to cancel the in-progress tiering operation. Must be a single index. Must not be on the migration blocklist.

Query Parameters:

Parameter Type Required Default Description
timeout time No 30s Timeout for acknowledgement of the cancel request.
cluster_manager_timeout time No 30s Timeout for connecting to the cluster manager node.

Success Response (200):

{
    "acknowledged": true
}

Failure Response (4xx/5xx):

{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "<validation error>"
            }
        ],
        "type": "illegal_argument_exception",
        "reason": "<validation error>"
    },
    "status": 400
}

Validations:

  1. Index name cannot be null or empty
  2. Exactly one index must be specified
  3. Index must not be on the migration blocklist

4. GET /{index}/_tier — Get Tiering Status for a Specific Index

Path Parameters:

Parameter Type Required Description
index string Yes Name of the index. Must be a single index (no wildcards).

Query Parameters:

Parameter Type Required Default Description
detailed boolean No false When true, includes per-shard relocation details (source_shard_id, relocating_node_id).

Success Response (200) — Default (detailed=false):

{
    "tiering_status": {
        "index": "my-index",
        "state": "RUNNING_SHARD_RELOCATION",
        "source": "hot",
        "target": "warm",
        "start_time": 1719446400000,
        "shard_level_status": {
            "total": 10,
            "succeeded": 4,
            "running": 3,
            "pending": 3
        }
    }
}

Success Response (200) — With detailed=true:

{
    "tiering_status": {
        "index": "my-index",
        "state": "RUNNING_SHARD_RELOCATION",
        "source": "hot",
        "target": "warm",
        "start_time": 1719446400000,
        "shard_level_status": {
            "total": 10,
            "succeeded": 4,
            "running": 3,
            "pending": 3,
            "shard_relocation_status": [
                {
                    "source_shard_id": 2,
                    "relocating_node_id": "node-abc-123"
                },
                {
                    "source_shard_id": 5,
                    "relocating_node_id": "node-def-456"
                },
                {
                    "source_shard_id": 7,
                    "relocating_node_id": "node-ghi-789"
                }
            ]
        }
    }
}

Top-Level Response Fields:

Field Type Description
tiering_status object Root object containing the tiering status for the requested index.

tiering_status Object Fields:

Field Type Description
index string Name of the index.
state string Current tiering state (e.g., RUNNING_SHARD_RELOCATION).
source string Source tier (hot or warm).
target string Target tier (hot or warm).
start_time long Epoch milliseconds when the tiering operation started.
shard_level_status object Shard-level progress counters and optional relocation details.

shard_level_status Object Fields:

Field Type Condition Description
total int Always Total number of shards in the index.
succeeded int Always Shards that have completed relocation and are STARTED on a target-tier node.
running int Always Shards currently in RELOCATING state.
pending int Always Shards that are UNASSIGNED or STARTED but still on a source-tier node.
shard_relocation_status array Only when detailed=true Per-shard relocation details for shards in RELOCATING state.

shard_relocation_status[] Array Element Fields:

Field Type Description
source_shard_id int The shard ID being relocated.
relocating_node_id string The node ID the shard is relocating to.

Shard State Mapping:

succeeded: Shard is STARTED and placed on a node matching the target tier
running: Shard is in RELOCATING state
pending: Shard is UNASSIGNED or STARTED but still on a source-tier node

Failure Response (4xx):

{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "request should contain single index"
            }
        ],
        "type": "illegal_argument_exception",
        "reason": "request should contain single index"
    },
    "status": 400
}

5. GET /_tier/all — List All In-Progress Tiering Operations

Path Parameters: None

Query Parameters:

Parameter Type Required Default Description
target string No None (returns all) Filter by target tier. Valid values: _warm, _hot.
v boolean No false Verbose mode — includes column headers in tabular output.
format string No text Response format: text (tabular), json, yaml, etc.

Request Body: None

Success Response (200) — Default (text/tabular):

index         state                     source target
my-index-1    RUNNING_SHARD_RELOCATION  hot    warm
my-index-2    RUNNING_SHARD_RELOCATION  warm   hot
my-index-3    RUNNING_SHARD_RELOCATION  hot    warm

Success Response (200) — With format=json:

[
    {
        "index": "my-index-1",
        "state": "RUNNING_SHARD_RELOCATION",
        "source": "hot",
        "target": "warm"
    },
    {
        "index": "my-index-2",
        "state": "RUNNING_SHARD_RELOCATION",
        "source": "warm",
        "target": "hot"
    }
]

Response Fields (per entry):

Field Type Description
index string Name of the index undergoing tiering.
state string Current tiering state (e.g., RUNNING_SHARD_RELOCATION).
source string Source tier (hot or warm).
target string Target tier (hot or warm).

Related component

Storage

Describe alternatives you've considered

No response

Additional context

Reference: #21016

Contributors: @skumawat2025 @abhita

Metadata

Metadata

Assignees

No one assigned

    Labels

    StorageIssues and PRs relating to data and metadata storageenhancementEnhancement or improvement to existing feature or requestuntriaged

    Type

    No type

    Projects

    Status

    🆕 New

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions