refactor: Move Mergekit integration to experimental submodule

[behroozazarkhalili]

Summary

Resolves #4395

This PR moves all Mergekit-related functionality to the experimental submodule (trl.experimental.mergekit) as part of v1.0 preparations. The Mergekit integration has seen low usage and may be deprecated or removed in future versions.

Changes

Module Structure

• Created trl/experimental/mergekit/ module with:
  • mergekit_utils.py - Core mergekit utilities (moved from trl/mergekit_utils.py)
  • callbacks.py - MergeModelCallback class (extracted from trl/trainer/callbacks.py)
  • __init__.py - Module exports

Removed from Main API

• Removed MergeModelCallback from trl/__init__.py exports
• Removed MergeModelCallback from trl/trainer/__init__.py exports
• Removed MergeModelCallback class from trl/trainer/callbacks.py
• Removed unused imports (is_mergekit_available, MergeConfig, merge_models, upload_model_to_hf)

Updated Imports

• Updated tests/test_callbacks.py to import from trl.experimental.mergekit
• All tests continue to work with experimental module

Documentation

• Updated docs/source/callbacks.md with:
  • Removed MergeModelCallback from main callbacks section
  • Added "Experimental Callbacks" section explaining the move
  • Provided migration example for users

Migration Guide

Before

from trl import MergeModelCallback
from trl.mergekit_utils import MergeConfig

config = MergeConfig()
merge_callback = MergeModelCallback(config)

After

from trl.experimental.mergekit import MergeConfig, MergeModelCallback

config = MergeConfig()
merge_callback = MergeModelCallback(config)

Rationale

As noted in issue #4395:

"I'm not seeing any usage for this part of the library, we may remove it in the future."

Moving to the experimental submodule:

• Signals potential deprecation to users
• Maintains backward compatibility through experimental namespace
• Allows for removal in v1.0 or later if usage doesn't materialize
• Keeps main API focused on core, well-used features

Testing

• All existing tests continue to pass with updated imports
• @require_mergekit decorator still works correctly
• No changes to functionality, only module location

Breaking Change

⚠️ This is a breaking change for users importing MergeModelCallback from the main trl module. Users will need to update their imports to use trl.experimental.mergekit instead.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

I'd prefer having a dedicated section under Experimental instead of(like BEMA)

[behroozazarkhalili]

@qgallouedec I've restructured the MergeModelCallback documentation as requested.

Changes made:

• ✅ Created dedicated documentation page: docs/source/merge_model_callback.md
• ✅ Added to Experimental section in _toctree.yml (alphabetically sorted after GSPO)
• ✅ Removed inline "Experimental Callbacks" section from callbacks.md

Pattern followed: Matches BEMA's structure with dedicated page under Experimental section, including:

• Feature description
• Complete usage example
• Note about experimental status
• API reference

The MergeModelCallback now has its own dedicated page in the Experimental section, just like BEMA for Reference Model.

[qgallouedec]

conflict resolving was too hard, I reopened a new one