Does Transformer Model perform better on more data? (How to avoid straight line prediction)

[nathanmaulding]

I know in the original post the author states that the straight line prediction is reasonable for this amount of data, but I wanted to ask if anyone has had success using this model in predicting more accurately than the straight line?

If so, what data was used? If not, what aspects of the model have others looked to change to create better predictions? data shuffling?

Thanks in advance,
Nate