router: make full map_callrw with split args #628

mrForza · 2025-12-15T14:42:08Z

This patch introduces a new way of map_callrw execution by which we can pass some arguments to all storages and split buckets' arguments to those storages that have at least one bucket of bucket_ids. To achieve this we introduce a new string option - mode to map_callrw api.

@TarantoolBot document

Title: vshard: mode option for router.map_callrw()

This string option regulates on which storages the user function will be executed via map_callrw. Possible values:

full - the user function will be executed on all storages in
cluster.
partial - the user function will be executed on certain sotrages on
which buckets from bucket_ids were found.

After that changes map_callrw works in 4 different scenarios depending on mode and bucket_ids options:

<mode = 'full', bucket_ids = nil> - user function will be executed
with args on all storages.
<mode = 'full', bucket_ids = {[bid_1] = {b_arg_1}, ...}> - storages
that have at least one bucket of bucket_ids will execute user
function with args and additional buckets' arguments. Other storages
will execute user function only with args.
<mode = 'partial', bucket_ids = {bid_1, ...}> - user function will be
executed with args on storages that have at least one bucket of
bucket_ids.
<mode = 'partial', bucket_id = {[bid_1] = {b_arg_1}, ...}> - the same
as the 3rd scenario but buckets' arguments (b_arg_1, ..., b_arg_n) will
be added as additional arguments to user function.

Also now map_callrw ends with error in cases of <mode = 'full', bucket_ids = {1, 2, ...}> and <mode = 'partial', bucket_ids = nil>.

Closes #559

Gerold103

Thanks 🔥! I sure hope it will help the customers 😁.

Gerold103 · 2025-12-18T21:42:52Z

vshard/router/init.lua

+    local grouped_args_added = false
    for id, rs in pairs(replicasets) do
        if grouped_args ~= nil then
            -- It's cheaper to push and then pop, rather then deepcopy
            -- arguments table for every call.
-            table.insert(args, grouped_args[id])
+            local rs_grouped_args = grouped_args[id]
+            table.insert(args, rs_grouped_args)
+            grouped_args_added = rs_grouped_args ~= nil
        end
        local res, err = rs:callrw('vshard.storage._call', func_args, opts_map)
-        if grouped_args ~= nil then
+        if grouped_args_added then
            table.remove(args)
+            grouped_args_added = false
        end


You won't need this flag I think if you would change it to something as:

for ... do local rs_args = grouped_args and grouped_args[id] or nil if rs_args then table.insert(args, grouped_args[id]) end -- do the call if rs_args then table.remove(args) end end

The code style is a bit relaxed, but you get the idea.

Gerold103 · 2025-12-18T21:44:21Z

vshard/router/init.lua

+        local mode = bucket_ids and MAP_CALLRW_PARTIAL or MAP_CALLRW_FULL
+        return mode, nil


You don't need to return trailing nils in Lua. Can be simplified to return bucket_ids and MAP_CALLRW_PARTIAL or MAP_CALLRW_FULL.

Gerold103 · 2025-12-18T21:46:08Z

vshard/router/init.lua

    if opts then
+        mode, err = router_set_map_callrw_mode(opts.mode, opts.bucket_ids)
+        if err then
+            return nil, err, nil


Same here. No need for a trailing nil.

Gerold103 · 2025-12-18T21:47:40Z

vshard/router/init.lua

+    local mode = MAP_CALLRW_FULL
    if opts then
+        mode, err = router_set_map_callrw_mode(opts.mode, opts.bucket_ids)
+        if err then
+            return nil, err, nil
+        end
        timeout = opts.timeout or consts.CALL_TIMEOUT_MIN
        do_return_raw = opts.return_raw


Here you assign mode and then overwrite it when opts are provided. Lets not do an overwrite. And instead simply declare the empty mode variable, and with given opts do what you do now, and in the else branch assign it to the "full" default. Then there won't be an overwrite, which in turn might save us a tiny bit of cpu.

Gerold103 · 2025-12-18T21:55:06Z

vshard/router/init.lua

+--    4) <mode = 'partial', bucket_id = {[bid_1] = {b_arg_1}, ...}> - the same
+--       as the 3rd scenario but buckets' arguments (b_arg_1, ..., b_arg_n)
+--       will be added as additional arguments to user function.


This here and in the commit message might be slightly misleading. It looks like the user should expect the bucket arguments to be passed as positional arguments like func(all_args, bucket1_arg, bucket2_arg, ...). But instead it is more like func(all_args, bucket_args_map). I suggest to clarify this.

Gerold103 · 2025-12-18T21:56:11Z

The commit message doesn't mention the ticket. You need to write Closes #559 right before the TarantoolBot doc request.

@TarantoolBot

This patch introduces a new way of `map_callrw` execution by which we can pass some arguments to all storages and split buckets' arguments to those storages that have at least one bucket of `bucket_ids`. To achieve this we introduce a new string option - `mode` to `map_callrw` api. Closes tarantool#559 @TarantoolBot document Title: vshard: `mode` option for `router.map_callrw()` This string option regulates on which storages the user function will be executed via `map_callrw`. Possible values: 1) `full` - the user function will be executed on all storages in cluster. 2) `partial` - the user function will be executed on certain sotrages on which buckets from `bucket_ids` were found. After that changes `map_callrw` works in 4 different scenarios depending on `mode` and `bucket_ids` options: 1) `<mode = 'full', bucket_ids = nil>` - user function will be executed with args on all storages. 2) `<mode = 'full', bucket_ids = bucket_args_map>` - storages that have at least one bucket of bucket_ids will execute user function with args and additional buckets' arguments. Other storages will execute user function only with args. 3) `<mode = 'partial', bucket_ids = numeric_buckets_map>` - user function will be executed with args on storages that have at least one bucket of bucket_ids. 4) `<mode = 'partial', bucket_ids = bucket_args_map>` - the same as the 3rd scenario but buckets' arguments will be added as additional arguments to user function. Annotations: bucket_args_map = `{[bid_1] = {b_arg_1}, ..., [bid_n] = {b_arg_n}}` numeric_buckets_map = `{bucket_id_1, ..., bucket_id_n}` Also now `map_callrw` ends with error in cases of `<mode = 'full', bucket_ids = {1, 2, ...}>` and `<mode = 'partial', bucket_ids = nil>`.

Serpentian

Thank you for the patch! It looks great, but I'm afraid, it's a little bit broken now, let's fix and test the new mode with broken cache on the router

Serpentian · 2025-12-22T13:16:13Z

test/router-luatest/map_callrw_test.lua

        res, err = ivshard.router.map_callrw('assert', {'a'},
            {bucket_ids = {5ULL, 6ULL}})
        ilt.assert(res)
        ilt.assert_not(err)


Typo in the doc request:

the user function will be executed on certain sotrages on

on certain storages

Serpentian · 2025-12-23T13:30:41Z

vshard/router/init.lua

+--       of bucket_ids.
+--    4) <mode = 'partial', bucket_ids = bucket_args_map> - the same as the 3rd
+--       scenario but buckets' arguments will be added as additional arguments
+--       to user function.


Comment and doc request:

Please, let's describe all available modes in the doc request and the comment above, the API is very difficult, it's better to explicitly mention every available behavior. What we're missing now is the description, when mode is not provided. It's default value is a little bit tricky (full, when no bucket_ids and partial otherwise).

Serpentian · 2025-12-23T13:35:09Z

vshard/router/init.lua

+        return nil, lerror.make('Router can\'t execute map_callrw with ' ..
+                                '\'full\' mode and numeric bucket_ids')
+    end
+    return opts_mode, nil


Nil in the end is not needed

Serpentian · 2025-12-23T13:51:55Z

vshard/router/init.lua

+        local rs_args = grouped_args and grouped_args[id] or nil
+        -- It's cheaper to push and then pop, rather then deepcopy
+        -- arguments table for every call.
+        if rs_args then table.insert(args, rs_args) end


Nit: let's preserve the old style with proper indentation of the conditions. It's easier to look at the code and minimizes the diff, which will look as follows:

diff --git a/vshard/router/init.lua b/vshard/router/init.lua index b6baeb3..a0be400 100644 --- a/vshard/router/init.lua +++ b/vshard/router/init.lua @@ -981,13 +981,14 @@ local function replicasets_map_reduce(replicasets, rid, func, args, -- local func_args = {'storage_map', rid, func, args} for id, rs in pairs(replicasets) do - if grouped_args ~= nil then + local rs_args = grouped_args and grouped_args[id] + if rs_args ~= nil then -- It's cheaper to push and then pop, rather then deepcopy -- arguments table for every call. table.insert(args, grouped_args[id]) end local res, err = rs:callrw('vshard.storage._call', func_args, opts_map) - if grouped_args ~= nil then + if rs_args ~= nil then table.remove(args) end if res == nil then

Serpentian · 2025-12-23T13:52:12Z

vshard/router/init.lua

-            -- arguments table for every call.
-            table.insert(args, grouped_args[id])
-        end
+        local rs_args = grouped_args and grouped_args[id] or nil


or nil is not needed in the end

Serpentian · 2025-12-23T14:13:03Z

vshard/router/init.lua

            router_ref_storage_all(router, timeout)
    end
    if timeout then
+        if plain_bucket_ids then


This won't work unfortunately. Previously the router_group_map_callrw_args was invoked right after router_ref_storage_by_buckets, now it can be invoked also after router_ref_storage_all. The first one finds the buckets in the cluster, updates the router map cache and only because of that we could use the router_group_map_callrw_args function. But the router_ref_storage_all doesn't do that.

Currently, the newly added mode (mode = 'full' + split args) doesn't work, when the cache on a router is broken, the arguments will be sent to the wrong instances. Let's won't forget to add the test case for that

mrForza requested a review from Gerold103 December 15, 2025 15:02

mrForza assigned Gerold103 Dec 15, 2025

mrForza requested a review from Serpentian December 18, 2025 10:11

mrForza assigned Serpentian Dec 18, 2025

Gerold103 reviewed Dec 18, 2025

View reviewed changes

Gerold103 assigned mrForza and unassigned Gerold103 Dec 18, 2025

mrForza force-pushed the mrforza/gh-559-full-map-call-rw-with-split-args branch from 1f4e9b4 to 11e76dc Compare December 22, 2025 12:37

mrForza assigned Serpentian and unassigned Serpentian and mrForza Dec 23, 2025

Serpentian reviewed Dec 23, 2025

View reviewed changes

Serpentian assigned mrForza and unassigned Serpentian Dec 23, 2025

		local mode = bucket_ids and MAP_CALLRW_PARTIAL or MAP_CALLRW_FULL
		return mode, nil

router: make full map_callrw with split args #628

Are you sure you want to change the base?

router: make full map_callrw with split args #628

Uh oh!

Conversation

mrForza commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Gerold103 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Gerold103 commented Dec 18, 2025

Uh oh!

Serpentian left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mrForza commented Dec 15, 2025 •

edited

Loading