Skip to content

Make median and quantile consistent to base R#254

Open
hcirellu wants to merge 6 commits intomainfrom
median
Open

Make median and quantile consistent to base R#254
hcirellu wants to merge 6 commits intomainfrom
median

Conversation

@hcirellu
Copy link
Collaborator

median and quantile have unexpected results when comparing it with integer transfered to integer64.

main:

median(as.integer64(c(1, 3)))
# integer64
# [1] 3

PR:

median(as.integer64(c(1, 3)))
# integer64
# [1] 2

I had to fix two existing tests.

Closes #247

@hcirellu
Copy link
Collaborator Author

I also had to change the optimizer64 in a way that the comparing quantile is called with type=7, which is the default in R. In addition I needed to add a custom round function, that does not round to the next even number for the conversion from double to integer64.
This way it meets the integer64 logic of calculating the in-between values in sortqtl.integer64 and orderqtl.integer64 where the resulting value is calculated by the interpolation of the neighboring values, i.e.:

neighboring_values[1L,] + (neighboring_values[2L,] - neighboring_values[1L,])*(sel%%1)

This boils down to:

as.integer64(1L) + as.integer64(2L - 1L)*0.5
# integer64
# [1] 2

which is different to coercing the double result to integer64 for comparison:

as.integer64(1L + (2L - 1L)*0.5)
# integer64
# [1] 1

Is this ok? Or should the calculation in sortqtl.integer64 and orderqtl.integer64 be changed to meet the requirement that median(as.integer(1:2)) equals as.integer64(median(1:2)), which seems inconsistent with the interpolation of neighboring values using +, -, and * for integer64?
WDYT?

@hcirellu hcirellu marked this pull request as ready for review January 27, 2026 16:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

median diverges of base when length of vector is even

1 participant