Add command to dump dynamic config values (#421) #8676

vaibhavyadav-dev · 2025-11-21T10:05:01Z

What changed?

Added an admin command that dump all frontend dynamic config values,

Why?

Operators currently cannot easily inspect frontend runtime dynamic configuration, dumping values aids troubleshooting, incident response, and verification after config changes.

How did you test it?

Closes #421

vaibhavyadav-dev · 2025-11-21T11:32:21Z

@bergundy I think It's better to add DefaultKey in this handler .

Do let me know what you think,
Thanks

dnr

please regenerate with the correct versions of protoc/protoc-gen-go to avoid the unrelated generated file changes

dnr · 2025-11-24T23:16:28Z

common/dynamicconfig/collection.go

 	}
 }

+func (c *Collection) GetClient() Client {


is this used anywhere? I'd prefer not exposing the underlying Client if we don't really need to

dnr · 2025-11-24T23:22:10Z

cmd/tools/gendynamicconfig/dynamic_config.tmpl

 	return err
 }
+func (s {{$P.Name}}TypedConstrainedDefaultSetting[T]) DefaultValue() any {
+	return s.cdef


this feels weird, s.cdef is a []TypedConstrainedValue[T], which is a type that's going to be pretty hard/awkward for any client code to use. I suppose you're only using it serialize to json or something, so it doesn't really matter what the type is?

dnr · 2025-11-24T23:24:16Z

common/dynamicconfig/file_based_client.go

+				hasDefaultConstraint = true
+			}
+		}
+		// add default value if not present


this feels kind of weird. why do we need to insert the default values here? first of all, it doesn't work for settings with multiple constrained defaults. second, it's using more memory for something we don't actually need for looking up settings. why can't the code to retrieve settings just have logic to get the default from the registry instead of doing it here? it needs that anyway to get the default for a setting that's not preset in the file at all, right?

dnr · 2025-11-24T23:25:34Z

common/dynamicconfig/registry.go

 	globalRegistry registry
 )

+func GetDefaultValueForKey(k Key) ConstrainedValue {


as above, this just doesn't make any sense for settings with constrained default values, there is no single default.

dnr · 2025-11-24T23:25:56Z

config/dynamicconfig/development-sql.yaml

  - value: true
 history.hostLevelCacheMaxSize:
-  - value: 8192
+  - value: 8000


why is this changed?

My bad, I did this to test my changes and forgot revert it.

dnr · 2025-11-24T23:28:59Z

proto/internal/temporal/server/api/dynamicconfig/v1/message.proto

+
+message ConstrainedValue {
+  Constraints constraints = 1;
+  google.protobuf.Struct value = 2;


hmm, shouldn't it be a google.protobuf.Value instead? most configs are scalars, there are structs and lists but they're not the majority.

I'm not sure it should be Value, should we just dump the struct as JSON? It could be simpler. But Value would be better than Struct, agree.

dnr · 2025-11-24T23:29:48Z

service/frontend/admin_handler.go

 	"go.temporal.io/server/api/adminservice/v1"
 	clusterspb "go.temporal.io/server/api/cluster/v1"
 	commonspb "go.temporal.io/server/api/common/v1"
+	dc "go.temporal.io/server/api/dynamicconfig/v1"


import should be named dynamicconfigspb ("spb" for "server proto buf")

dnr · 2025-11-24T23:30:51Z

service/frontend/admin_handler.go

+		}
+
+		protoValues := make([]*dc.ConstrainedValue, 0)
+		for _, ConstrainedValue := range dcEntry {


use lowercase for variable names

Suggested change

for _, ConstrainedValue := range dcEntry {

for _, cv := range dcEntry {

dnr · 2025-11-24T23:31:30Z

service/frontend/admin_handler.go

+			dcEntry = append(dcEntry, defaultValue)
+		}
+
+		protoValues := make([]*dc.ConstrainedValue, 0)


Suggested change

protoValues := make([]*dc.ConstrainedValue, 0)

protoValues := make([]*dc.ConstrainedValue, 0, len(dcEntry))

or assign to specific indexes. or use util.MapSlice.

dnr · 2025-11-24T23:33:39Z

proto/internal/temporal/server/api/adminservice/v1/request_response.proto

+}
+
+message GetDynamicConfigurationsResponse {
+  repeated HostConfig host_config = 1;


why repeated? do we allow querying multiple hosts at once?

Not now, but do have plan to make so.

This was my idea, so you can compare dynamic configs across hosts. This will be a future capability though.

vaibhavyadav-dev · 2025-11-25T17:28:34Z

Thanks for the review @dnr
I have a question about the approach since It doesn't work for settings with multiple constrained defaults, and inserting defaults at load time is bad.

I'm considering two alternatives.

Only display values that are explicitly configured in the dynamic config file. If a setting isn't present, don't show it.
Retrieve defaults at display time from the registry based on the actual constraints being queried.

Which approach would you prefer? I'm leaning toward first one as it's simpler and provides value for operators troubleshooting config issues.

bergundy · 2025-12-03T23:31:03Z

common/dynamicconfig/registry.go

 	return globalRegistry.settings[strings.ToLower(k.String())]
 }

+func GetDefaultValues(k Key) []ConstrainedValue {


Suggested change

func GetDefaultValues(k Key) []ConstrainedValue {

// DefaultValue returns the default value defined for the setting of the given key.

func DefaultValue(k Key) []ConstrainedValue {

But I'm not even convinced we want this.

bergundy · 2025-12-03T23:33:35Z

proto/internal/temporal/server/api/dynamicconfig/v1/message.proto

+
+message ConstrainedValue {
+  Constraints constraints = 1;
+  google.protobuf.Value value = 2;


I wonder if it would be simpler to give back a serialized JSON object here.

bergundy · 2025-12-03T23:39:05Z

service/frontend/admin_handler.go

+
+		// get global default value
+		if dcEntry == nil {
+			dcEntry = dynamicconfig.GetDefaultValues(k)


Do we even want to give back the default value? I was thinking we would return the "raw" values without deserializing them. I'm not sure if the default values are always JSON serializable.

I can look into this a bit more but @dnr can speak to it better than I can.

bergundy · 2025-12-03T23:42:17Z

service/frontend/admin_handler.go

+
+		protoValues := make([]*dynamicconfigspb.ConstrainedValue, 0, len(dcEntry))
+		for _, cv := range dcEntry {
+			s, err := structpb.NewValue(normalizeValue(cv.Value))


This isn't ideal, we should be able to return all of the values back. Users of this API will be surprised to not get responses back.

Reading the implementation of NewValue I'm not convinced that it will be able to convert everything that we support, which is why I think returning a JSON serialized representation is probably a better option.
We should also test this API with various complex configs to see that we return what is expected.

bergundy · 2025-12-03T23:45:51Z

proto/internal/temporal/server/api/dynamicconfig/v1/message.proto

+	string namespace = 1;
+	string namespace_id = 2;
+	string task_queue_name = 3;
+	int32 task_queue_type = 4;


We should be able to use
TaskQueueType from temporal/api/enums/v1/task_queue.proto and TaskType from temporal/server/api/enums/v1/task.proto.

bergundy · 2025-12-03T23:47:51Z

service/frontend/admin_handler_test.go

+	dynamicConfigClient.OverrideValue("matching.rps", 20)
+	dynamicConfigClient.OverrideValue("history.rps", 30)
+	dynamicConfigClient.OverrideValue("limit.maxIDLength", 255)
+	dynamicConfigClient.OverrideValue("history.enableTransitionHistory", true)


Test more complex configs here. Anything that has slices or maps or is a more complex struct. There are a few of those in dynamicconfig/constants.go.

bergundy · 2025-12-03T23:48:21Z

tools/tdbg/tdbg_commands.go

+		{
+			Name: "config",
+			Usage: "Command to get dynamic config values",
+			Aliases: []string{"cfg"},


Suggested change

Aliases: []string{"cfg"},

bergundy · 2025-12-03T23:48:31Z

tools/tdbg/tdbg_commands.go

+	return []*cli.Command{
+		{
+			Name:    "get",
+			Aliases: []string{"g"},


Suggested change

Aliases: []string{"g"},

bergundy · 2025-12-03T23:49:03Z

tools/tdbg/tdbg_commands.go

+			Flags: []cli.Flag{
+				&cli.StringSliceFlag{
+					Name:  FlagKey,
+					Usage: "Enter keys",


Suggested change

Usage: "Enter keys",

Usage: "Dynamic config keys to get",

bergundy · 2025-12-03T23:49:29Z

tools/tdbg/commands.go

+
+    keys := c.StringSlice(FlagKey)
+    if len(keys) == 0 {
+        return fmt.Errorf("flag %q is required", FlagKey)


Suggested change

return fmt.Errorf("flag %q is required", FlagKey)

return fmt.Errorf("at least one %q flag is required", FlagKey)

vaibhavyadav-dev requested review from a team as code owners November 21, 2025 10:05

vaibhavyadav-dev mentioned this pull request Nov 21, 2025

Add command to dump frontend dynamic config values #8214

Closed

5 tasks

vaibhavyadav-dev changed the title ~~feat: add command to dump dynamic config values (#421)~~ Add command to dump dynamic config values (#421) Nov 21, 2025

dnr reviewed Nov 24, 2025

View reviewed changes

vaibhavyadav-dev closed this Nov 28, 2025

vaibhavyadav-dev force-pushed the added-tdbg-command-to-dump-dynamic-config-values branch from abb41c8 to 509df74 Compare November 28, 2025 18:15

feat: Added tdbg command to dump dynamic config value

96a5562

vaibhavyadav-dev reopened this Nov 28, 2025

bergundy reviewed Dec 3, 2025

View reviewed changes

	for _, ConstrainedValue := range dcEntry {
	for _, cv := range dcEntry {

	protoValues := make([]*dc.ConstrainedValue, 0)
	protoValues := make([]*dc.ConstrainedValue, 0, len(dcEntry))

	func GetDefaultValues(k Key) []ConstrainedValue {
	// DefaultValue returns the default value defined for the setting of the given key.
	func DefaultValue(k Key) []ConstrainedValue {

	return fmt.Errorf("flag %q is required", FlagKey)
	return fmt.Errorf("at least one %q flag is required", FlagKey)

Add command to dump dynamic config values (#421) #8676

Are you sure you want to change the base?

Add command to dump dynamic config values (#421) #8676

Conversation

vaibhavyadav-dev commented Nov 21, 2025

What changed?

Why?

How did you test it?

Uh oh!

vaibhavyadav-dev commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dnr left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vaibhavyadav-dev commented Nov 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vaibhavyadav-dev commented Nov 21, 2025 •

edited

Loading