problem

[Ulricalang]

Hello, I have a question. When I was running the residual 34, there was a nan value in the global test accuracy. I don't know where there was a problem.

[YMJS-Irfan]

Hello, I have a question. When I was running the residual 34, there was a nan value in the global test accuracy. I don't know where there was a problem.

Thanks for focusing on our work! This issue hasn't happened in our experimental settings; the detailed description can see in our CVPR paper.
And for some scenarios where larger C can lead to the gradient explosion phenomenon, there was a nan value in the global test accuracy. This issue may be caused by a large variance magnitude of DP random noise due to large C.

Some explanation and experimental setup can be found in Section 6.1 of our CVPR paper :

[Ulricalang]

Is the global_test_loss value too large because of this problem?

[YMJS-Irfan]

Is the global_test_loss value too large because of this problem?

Yes, maybe this issue causes. We also found this phenomenon under some hyper-parameter settings.

[Ulricalang]

I also want to ask, I tried to add some layers to the model, which was normal during training, but the loss value was unusually large during testing. Is it possible that this is also related to the noise value? Did you try it during the test?

[YMJS-Irfan]

I also want to ask, I tried to add some layers to the model, which was normal during training, but the loss value was unusually large during testing. Is it possible that this is also related to the noise value? Did you try it during the test?

In our existing experiments, we only use the standard backbone for verifying the effectiveness of DP-FedSAM.
And if the training process is normal, I think it may be impossible for unusually large loss during the testing process. Maybe, could you check some detailed code or change some important hyper-parameters?

[Ulricalang]

Thank you very much for your answer. I wish you all the best.

[YMJS-Irfan]

I also want to ask, I tried to add some layers to the model, which was normal during training, but the loss value was unusually large during testing. Is it possible that this is also related to the noise value? Did you try it during the test?

Furthermore, we find that this issue may possibly occur in the EMNIST data. Maybe you can try it on CIFAR-10/100.
Best wish.

[ssissw]

Hello, I have a question. When I was running the residual 34, there was a nan value in the global test accuracy. I don't know where there was a problem.

Thanks for focusing on our work! This issue hasn't happened in our experimental settings; the detailed description can see in our CVPR paper. And for some scenarios where larger C can lead to the gradient explosion phenomenon, there was a nan value in the global test accuracy. This issue may be caused by a large variance magnitude of DP random noise due to large C.

Some explanation and experimental setup can be found in Section 6.1 of our CVPR paper :

I wonder why the var of noise is (σ^2 * C^2 / m), since the sensitivity of each client update is bounded by (C / m), may the var should be (σ^2 * C^2 / m^2)?