Skip to content

[Animal] Maintaining herd size#2622

Open
matthew7838 wants to merge 153 commits intodevfrom
maintain-herd
Open

[Animal] Maintaining herd size#2622
matthew7838 wants to merge 153 commits intodevfrom
maintain-herd

Conversation

@matthew7838
Copy link
Collaborator

@matthew7838 matthew7838 commented Oct 21, 2025

Context

Issue(s) closed by this pull request: closes #1684 and #357

What

Goal of this PR is to implement changes to the culling logic so that "low production" culls (now called "oversupply culls") are a reaction to how many heifers are calving in and the resulting size of the cow herd. Changes include:

  • Eliminate “low production” DNB culls and cull_milk_production input

  • Eliminate heiferIII sales

  • Update daily herd size checks to user-defined interval of herd size checks

  • For herd size checks, compare Cows to Herd Num

  • Update buy_threshold and sell_threshold to be user defined

  • Update interval at which herd size is checked to be user defined

  • If number of cows < buy_threshold: buy springers

  • If number of cows > sell threshold: sell cows

  • Identify cow(s) to sell:

    • Cows with > 60 DIM and < 180 days pregnant are eligible
    • Rank eligible cows by milk production
    • Remove bottom cows until reach threshold value
  • All herd inventory numbers (herd statistics?) daily reflect the update

  • All outputs reflect the update

    • Cows_sold
    • Cow_herd_exits
    • (others)

Updates to M305 / ME305 (parity-adjusted M305) will be addressed in a separate PR to be revisited later.

Additional information in "helpful info" doc here: https://docs.google.com/document/d/1O_j24ktBA2-56nYlZOdZxhbiPJfdaeDK3jfLvXBuQuU/edit?usp=sharing

Why

How

Test plan

@matthew7838 matthew7838 self-assigned this Oct 21, 2025
@github-actions
Copy link
Contributor

Current Coverage: %

Mypy errors on maintain-herd branch: 1678
Mypy errors on dev branch: 1678
No difference in error counts

@github-actions
Copy link
Contributor

🚨 Please update the changelog. This PR cannot be merged until changelog.md is updated.
🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.
🚨 Flake8 linting errors were found. Please fix the linting issues.
🚨 Some tests have failed.

@github-actions
Copy link
Contributor

Current Coverage: %

Mypy errors on maintain-herd branch: 1678
Mypy errors on dev branch: 1678
No difference in error counts

@github-actions
Copy link
Contributor

🚨 Please update the changelog. This PR cannot be merged until changelog.md is updated.
🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.
🚨 Flake8 linting errors were found. Please fix the linting issues.
🚨 Some tests have failed.

@jadamchick
Copy link
Contributor

Thanks Matthew! Just noting after discussion this morning - this PR will implement the initial logic to switch from pull to push culling model (available heifers push out low performing cows at user-defined interval) using a first-pass / placeholder metric for ranking cows on performance. Not intended to comprehensively address everything in the design doc referenced in the issue; there will be more work to follow.

@github-actions
Copy link
Contributor

Current Coverage: %

Mypy errors on maintain-herd branch: 1678
Mypy errors on dev branch: 1678
No difference in error counts

@github-actions
Copy link
Contributor

🚨 Please update the changelog. This PR cannot be merged until changelog.md is updated.
🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.
🚨 Flake8 linting errors were found. Please fix the linting issues.
🚨 Some tests have failed.

@github-actions
Copy link
Contributor

Current Coverage: %

Mypy errors on maintain-herd branch: 1684
Mypy errors on dev branch: 1678
6 more errors on maintain-herd branch

@github-actions
Copy link
Contributor

🚨 Please update the changelog. This PR cannot be merged until changelog.md is updated.
🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.
🚨 Flake8 linting errors were found. Please fix the linting issues.
🚨 Some tests have failed.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 3, 2025

Current Coverage: %

Mypy errors on maintain-herd branch: 1716
Mypy errors on dev branch: 1710
6 more errors on maintain-herd branch

@github-actions
Copy link
Contributor

github-actions bot commented Nov 3, 2025

🚨 Please update the changelog. This PR cannot be merged until changelog.md is updated.
🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.
🚨 Flake8 linting errors were found. Please fix the linting issues.
🚨 Some tests have failed.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2025

Current Coverage: %

Mypy errors on maintain-herd branch: 1711
Mypy errors on dev branch: 1710
1 more errors on maintain-herd branch

@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2025

🚨 Please update the changelog. This PR cannot be merged until changelog.md is updated.
🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.
🚨 Flake8 linting errors were found. Please fix the linting issues.
🚨 Some tests have failed.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2025

Current Coverage: %

Mypy errors on maintain-herd branch: 1711
Mypy errors on dev branch: 1710
1 more errors on maintain-herd branch

@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2025

🚨 Please update the changelog. This PR cannot be merged until changelog.md is updated.
🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.
🚨 Flake8 linting errors were found. Please fix the linting issues.
🚨 Some tests have failed.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2025

Current Coverage: %

Mypy errors on maintain-herd branch: 1711
Mypy errors on dev branch: 1710
1 more errors on maintain-herd branch

@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2025

🚨 Please update the changelog. This PR cannot be merged until changelog.md is updated.
🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.
🚨 Flake8 linting errors were found. Please fix the linting issues.
🚨 Some tests have failed.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2025

Current Coverage: %

Mypy errors on maintain-herd branch: 1711
Mypy errors on dev branch: 1710
1 more errors on maintain-herd branch

@github-actions
Copy link
Contributor

github-actions bot commented Nov 4, 2025

🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.
🚨 Flake8 linting errors were found. Please fix the linting issues.
🚨 Some tests have failed.

@github-actions
Copy link
Contributor

Current Coverage: 99%

Mypy errors on maintain-herd branch: 1248
Mypy errors on dev branch: 1249
1 fewer errors on maintain-herd branch

@github-actions
Copy link
Contributor

🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.

matthew7838 and others added 3 commits March 11, 2026 23:22
Co-authored-by: jadamchick <146877580+jadamchick@users.noreply.github.com>
@github-actions
Copy link
Contributor

Current Coverage: 99%

Mypy errors on maintain-herd branch: 1248
Mypy errors on dev branch: 1249
1 fewer errors on maintain-herd branch

@github-actions
Copy link
Contributor

🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.

@github-actions
Copy link
Contributor

Current Coverage: 99%

Mypy errors on maintain-herd branch: 1248
Mypy errors on dev branch: 1249
1 fewer errors on maintain-herd branch

@github-actions
Copy link
Contributor

🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.

Copy link
Collaborator

@allisterakun allisterakun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Nice work, Matthew! Just remember to move the changelog entry to the Next Version Updates section

Please also update the E2E expected results!!!

Copy link
Contributor

@jadamchick jadamchick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviously I've done plenty of commenting already, but want to document here that I think this PR is doing what it is intended to do! Illustrated most clearly by running the example_freestall herd with a higher and lower male_calf_rate. Baseline is 0.53, so I ran it at 0.3 and 0.7 to see how the herd responded to having more or fewer calves entering the system.

In comparison to the baseline (MCR = 53%)... (supporting results in google sheet here: link)

Lower MCR (0.3):

  • More calves + heifers
  • Higher total cows (at or above sell threshold on average)
  • More cows sold for oversupply & higher herd turnover rate
  • Lower average parity / higher % of the herd in L1
  • Lower milk allocation factor (more heifer and cow sales, even though more milk sold)
  • Attributes of the cows sold for oversupply: more milk (though still less than herd average); more % L1; more pregnant

Higher MCR (0.7):

  • Fewer calves + heifers
  • Lower turnover rate
  • More open cows, much higher average days in milk (longer lactation)
  • Much lower milk production per cow
  • Fewer calves born
  • Fewer cows sold for oversupply
  • Fewer cows sold for "acute culls" (by-parity cull prob) -- because lactation lengths are so long
  • Higher milk allocation factor (fewer animals sold at every stage - calves, heifers, cows; even though less milk sold)
  • Attributes of the cows sold for oversupply: zero milk, very high days in milk

What is going on with the higher MCR herd? Two interacting factors:
a) A low number of heifers drives low turnover, so more cows have to stay in the herd even though they're not making much milk
b) This herd has a pretty aggressive DNB cutoff (180 days in milk) -- so a lot of cows are getting marked as DNB cows, meaning that they are no longer being bred; however they have no replacement available so do not get sold either, so they are staying in the herd but without getting pregnant and having another calf and starting a new lactation. So fewer babies are being born, and less milk is getting made, and that turns into a vicious cycle with even fewer heifers available in the future.

I also ran this for several pilot herds on dev vs. this branch, and they showed similar trends. Most showed only minor changes, and if they were net buying or selling a bunch of heiferIII's every year (on dev), then the whole herd would be off-balance on this branch, similar to the trends seen above.

An interesting side effect is that there is less impact caused by a discrepancy in mature body weight between the initial animal population aka replacement herd and the animal input value. I saw this when trying to figure out why, for some herds, there was a big difference in MBW between this branch and dev. Note that in herds 5 and 9, they bought 24-34% of their herd per year as replacement heifers on dev, and only 1-4% on this branch given the inputs I used. This difference resulted in a difference of ~50 kg between the herd on dev vs. on this branch.

Herd MBW dev (last yr) MBW mhs (last year) MBW input   heifers bought/yr, dev heifers bought/yr, mhs total cows, dev total cows, mhs heifers bought as % of cow herd, dev heifers bought as % of cow herd, mhs
5 699 645 614   12.3 2.1 51 49 24% 4%
6 728 726 726   10.9 0 118 122 9% 0%
7 741 740 740   26.4 0 2197 2232 1% 0%
8 748 748 748   10.7 0 492 496 2% 0%
9 638 584 546   20.1 0.7 59 58 34% 1%
10 572 563 545   0 0 3510 3605 0% 0%

@jadamchick
Copy link
Contributor

ps I do give my approval, but would also like to give the rest of the animal team a chance to look it over and would prefer another SME review besides myself, since I've been elbow-deep in this for quite a while...

@github-actions
Copy link
Contributor

Current Coverage: 99%

Mypy errors on maintain-herd branch: 1248
Mypy errors on dev branch: 1249
1 fewer errors on maintain-herd branch

@github-actions
Copy link
Contributor

🚨 Unauthorized changes detected in protected files. Please remove these changes if they are not intended.

@gmg228
Copy link
Contributor

gmg228 commented Mar 13, 2026

Review of changes:

  • "Eliminate “low production” DNB culls and cull_milk_production input
  • Eliminate heiferIII sales
  • Update daily herd size checks to user-defined interval of herd size checks
  • For herd size checks, compare Cows to Herd Num
  • Update buy_threshold and sell_threshold to be user defined
  • Update interval at which herd size is checked to be user defined
  • If number of cows < buy_threshold → buy springers
  • If number of cows > sell threshold → sell cows
  • Identify cow(s) to sell:
  • Cows with > 60 DIM and < 180 days pregnant are eligible "

These updates are comprehensive, but may leave open some opportunities to continue to improve this logic. Confirming that there may still be future plans to address DNBs based on repro and production?

Otherwise, I think that this is ready to merge.

@matthew7838
Copy link
Collaborator Author

Review of changes:

  • "Eliminate “low production” DNB culls and cull_milk_production input
  • Eliminate heiferIII sales
  • Update daily herd size checks to user-defined interval of herd size checks
  • For herd size checks, compare Cows to Herd Num
  • Update buy_threshold and sell_threshold to be user defined
  • Update interval at which herd size is checked to be user defined
  • If number of cows < buy_threshold → buy springers
  • If number of cows > sell threshold → sell cows
  • Identify cow(s) to sell:
  • Cows with > 60 DIM and < 180 days pregnant are eligible "

These updates are comprehensive, but may leave open some opportunities to continue to improve this logic. Confirming that there may still be future plans to address DNBs based on repro and production?

Otherwise, I think that this is ready to merge.

I believe so, with the ME305-based culling currently right next in the pipeline. @jadamchick Just to confirm again, after the final few changes, our plan is still to incorporate the ME305, right?

@jadamchick
Copy link
Contributor

Thanks Geneva!

These updates are comprehensive, but may leave open some opportunities to continue to improve this logic. Confirming that there may still be future plans to address DNBs based on repro and production?

The current logic (in this PR):

  • cows become DNB's when they pass the number of days designated by do_not_breed_time and are not yet pregnant
  • how is a cow treated differently once she is designated DNB? the only difference is that she is not eligible to be bred (and so never gets pregnant, has another calf, or starts another lactation)

Would love to hear what specific future plans you are thinking of / would suggest?
One option, that might be more similar to how many real dairies designate DNB's, would be to make the cutoff for triggering a DNB flag to be based on milk production rather than DIM. But we haven't discussed that with any of the animal team.

I don't think it makes sense to prioritize DNB's for culling, because they are often making quite a bit more milk than other cows in the herd (especially with some of these herds htat have very early DNB cutoffs, such as 150 or 180 DIM). When I used print statements to look at the cows being sold in these situations, it was sometimes DNB's making more milk than the herd average milk. That said, the updates planned in #2693 would allow a user to protect all pregnant cows and all cows with fewer DIM than the DNB cutoff if they want to, thereby limiting culls to only the DNB cows if that is the desired behavior (or using the pregnant & DIM dials to implement a less extreme set of eligibility criteria)

I do wonder about adding a warning that alerts the user when there are cows whose milk production has dropped to 0 kg per day (in some of those off-balance herds with DNB's just hanging around forever but not enough replacement heifers to sell them) -- to draw the user's attention to the fact that they need to improve their inputs. However, I don't want to have to print a warning for every cow every day. @matthew7838 would it be possible to print a warning at the end of each simday in which there is at least 1 lactating cow with 0 kg/d milk production that day, and have the warning say what the simday is and how many lactating cows had 0 milk production? (or other ideas along those lines?)

I believe so, with the ME305-based culling currently right next in the pipeline. @jadamchick Just to confirm again, after the final few changes, our plan is still to incorporate the ME305, right?

yep! first to implement the customized culling inputs here #2693, and then to do some variations of M305, ME305, energy corrected milk, etc. that users could choose to use for their cull_ranking_criteria

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Animal] Maintaining Herd Size by Animal Culling

5 participants