2015-09-02 GLM 2.0.4 Ice on and off comparison

[lawinslow]

Using GLM for ice-on and -off is the mda.lakes holy grail.

The Goal

Shuter et al 2013 used a simple multi-linear regression approach and got these stats

Ice on

• -2.4 day bias (observed - predicted)
• 8.4 day precision (standard deviation(observed - predicted))
• 8.7 mean absolute error

Ice off

• -3.0 day bias (observed - predicted)
• 7.4 day precision (standard deviation(observed - predicted))
• 5.2 mean absolute error

Interesting that they get better ice-off estimates. Probably the opposite of what we'll get. (edit: or not)

GLM v2.0.4 Results

Ice on

• 3.3 day bias (observed - predicted)
• 11.1 day precision (standard deviation(observed - predicted))
• 7.4 mean absolute error

Ice off

• 11.0 day bias (observed - predicted)
• 8.7 day precision (standard deviation(observed - predicted))
• 11.9 mean absolute error

Ok, we're getting there. It has been far worse. Ice on isn't great, but I have some anecdotal evidence from Dale that it can be fairly good. Ice off is all about the bias. If we can get rid of the 11 day bias, then we could get mean absolute error down to 6.5.

Also, here are some plots.

Note: below is ice off, ignore title

[jordansread]

looking very promising. NLDAS w/ wind bias removed?

[lawinslow]

Straight NLDAS. didn't correct wind, but I can

[jordansread]

just curious what the reference point is/was. We should discuss tomorrow, but this seems pretty positive. Another thing to consider is how error in either end (off vs on) propagates to the thermal metrics of interest.

[lawinslow]

Model tweaks

• Ice/snow evaporation improved
• Max albedo bumped from 0.7 to 0.8

GLM v2.0.4-AlbedoBump Results

Ice on

• 3.5 day bias (observed - predicted)
• 11.2 day precision (standard deviation(observed - predicted))
• 7.6 mean absolute error

(Interesting that these results change a little)

Ice off

• 1.1 day bias (observed - predicted)
• 8.1 day precision (standard deviation(observed - predicted))
• 6.2 mean absolute error

Significant improvement.

[jordansread]

wow, quite a jump in the I-O bias. Was the albedo or the evap change more important to that?

[lawinslow]

All albedo. I think I'm going to move forward switching to full runs instead of chained. What do you think?

[jordansread]

Yep. Does it require evap mass flux to be off?

[lawinslow]

Nope. Evap is now on, unlike before. Though granted, Winter Evap is pretty low. I didn't evaluate carefully, but I doubt it made a huge difference.

[lawinslow]

Note, earlier results were from a small increase to max snow albedo. Now improved implementation in GLM v3.1.0 - alpha. Using snow_albedo_factor = 1.1 we get similar results.

> cat('MAE of GLM Overall, on and off date/time')
MAE of GLM Overall, on and off date/time
> cat('Mean Absolute Errors\n')
Mean Absolute Errors
> mean(abs(both_ice$glm_doy - both_ice$doy))
[1] 7.011064
> mean(abs(both_ice[both_ice$on.off=='on',]$glm_doy - both_ice[both_ice$on.off=='on',]$doy))
[1] 7.639442
> mean(abs(both_ice[both_ice$on.off=='off',]$glm_doy - both_ice[both_ice$on.off=='off',]$doy))
[1] 6.542348
> 
> cat('STD of GLM Overall, on and off date/time')
STD of GLM Overall, on and off date/time
> cat('Standard Deviation\n')
Standard Deviation
> sd((both_ice$glm_doy - both_ice$doy))
[1] 9.586225
> sd((both_ice[both_ice$on.off=='on',]$glm_doy - both_ice[both_ice$on.off=='on',]$doy))
[1] 11.33798
> sd((both_ice[both_ice$on.off=='off',]$glm_doy - both_ice[both_ice$on.off=='off',]$doy))
[1] 8.038161
> 
> cat('Bias of GLM Overall, on and off date/time')
Bias of GLM Overall, on and off date/time
> cat('Bias\n')
Bias
> mean((both_ice$glm_doy - both_ice$doy))
[1] -3.23234
> mean((both_ice[both_ice$on.off=='on',]$glm_doy - both_ice[both_ice$on.off=='on',]$doy))
[1] -3.5
> mean((both_ice[both_ice$on.off=='off',]$glm_doy - both_ice[both_ice$on.off=='off',]$doy))
[1] -3.032689
> 
> cat('MAE of empirical model overall, on and off date/time')
MAE of empirical model overall, on and off date/time
> cat('Mean Absolute Errors\n')
Mean Absolute Errors
> mean(abs(both_ice$emp_doy - both_ice$doy))
[1] 6.18383
> mean(abs(both_ice[both_ice$on.off=='on',]$emp_doy - both_ice[both_ice$on.off=='on',]$doy))
[1] 6.709163
> mean(abs(both_ice[both_ice$on.off=='off',]$emp_doy - both_ice[both_ice$on.off=='off',]$doy))
[1] 5.791976
> 
> 
> cat('R^2')
R^2
> tmp = both_ice[both_ice$on.off == 'off', ]
> summary(lm(tmp$emp_doy ~ tmp$glm_doy))$r.squared
[1] 0.7406314
> summary(lm(tmp$doy ~ tmp$glm_doy))$r.squared
[1] 0.7147607
> summary(lm(tmp$doy ~ tmp$emp_doy))$r.squared
[1] 0.7732483