Issues with temperature #13
Description
Hello @cpmpercussion,
Here I am back again to ask you a new question about your code! This time, regarding temperature sampling.
My reasoning tells me that temperature sampling for 'mu' should be as close to 1 as possible, as the selection of one gaussian ought not to discriminate the others. When it comes to temperature sampling for sigma, I guess that we could make use of values of sigma_temp as close to 0 as possible, but I cannot tell the exact reason why.
I did some grid search for different values of sigma_temp and temp and my results seem to be better when both of them are as small as possible. But when I set both sigma_temp and temp to 1e-4, I receive the following regression line as output (my outputs are normalized between -1 and 1, and the blue line represents the expected -target values):
Does it exist a clear 'rule of thumb' for this model about how to perform temperature sampling for sigma and mu values? Do you find any reason why I am having this weird shape as output?
Many thanks in advance, and may you have a nice day!