Adding SWE-Smith support

[MarcCote]

This pull request introduces several enhancements and bug fixes to the debug_gym repository, including the addition of a new agent class, support for a new environment, and various utility improvements. Key changes include the implementation of the AgentSolution class, the addition of the SWESmithEnv environment, and updates to configuration loading and container setup logic.

New Features

• New Agent Class: Introduced AgentSolution, a new agent class in debug_gym/agents/solution_agent.py, which includes methods for debugging tasks, applying patches, and evaluating results.
• New Environment: Added SWESmithEnv in debug_gym/gym/envs/swe_smith.py, which supports the SWE-Smith benchmark. This includes dataset loading, Docker image handling, and task setup logic.

Environment Enhancements

• Environment Selection: Updated select_env in debug_gym/gym/envs/__init__.py to include SWESmithEnv as a selectable environment.
• Dataset Loading: Modified dataset loading in SWEBenchEnv to support dynamic splits.

Utility Improvements

• Configuration Loading: Enhanced load_config in debug_gym/agents/utils.py to allow parameter extension using the extend action.
• Container Setup: Updated container setup in debug_gym/gym/terminal.py to use network_mode="host" and simplified container renaming logic.

Bug Fixes

• PDB Tool Output: Fixed output handling in use method of debug_gym/gym/tools/pdb.py to correctly adjust messages when the program exits via sys.exit().

Miscellaneous

• Manifest Update: Included YAML configuration files in the package manifest (MANIFEST.in).
• Imports: Added missing imports for DEBUG_GYM_CONFIG_DIR in debug_gym/init_llm_config.py.

[Copilot]

Pull Request Overview

This PR introduces support for a new environment (SWESmithEnv) and a new agent (AgentSolution) aimed at debugging tasks for the SWE-Smith dataset while also updating existing environments and terminal configurations. Key changes include:

• Adding new utility functions in scripts/run.py for grouping and printing problems.
• Introducing SWESmithEnv with Docker‐based repository cloning, image pulling, and custom task initialization.
• Adding a new agent (AgentSolution) that applies a gold patch and then evaluates the task.

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
scripts/run.py	Added functions for common prefix, grouping, pretty-printing and updated dataset selection logic.
scripts/config_swesmith.yaml	New YAML configuration for the SWE-Smith environment with minor comments.
debug_gym/utils.py	New constants for config and cache directories created at startup.
debug_gym/init_llm_config.py	Switched to using the new DEBUG_GYM_CONFIG_DIR constant.
debug_gym/gym/terminal.py	Updated container setup by adding network_mode and commenting out container renaming.
debug_gym/gym/envs/swe_smith.py	New environment with repository cloning, image pulling, dataset splitting and task setup.
debug_gym/gym/envs/swe_bench.py	Updated to use configurable dataset split instead of hardcoded “test”.
debug_gym/gym/envs/env.py	Modified entrypoint preparation logic to handle specific cases (e.g. “uv run”).
debug_gym/gym/envs/configs/swe_smith.yaml	New config file for task splits in the SWE-Smith environment.
debug_gym/gym/envs/init.py	Added import and environment selection case for “swesmith”.
debug_gym/agents/solution_agent.py	New debugging agent that applies a “gold patch” and evaluates its effect.
debug_gym/agents/init.py	Included the new AgentSolution in the agent registry.
MANIFEST.in	Updated to include YAML configuration files for environments.

Comments suppressed due to low confidence (2)

scripts/config_swesmith.yaml:6

• There is a spelling mistake in the comment ('lillst' should be 'list'). Fixing this improves clarity.

# problems: "all"  # lillst of problems, e.g., ["spulec__freezegun.5f171db0.combine_file__f3rcc5ea"]

debug_gym/agents/solution_agent.py:38

• [nitpick] The f-string already interpolates the variables, so the appended .format() call is redundant and may confuse readers. Remove the .format(info.score) part.

self.logger.info(f"Score: {info.score}/{info.max_score} ({info.score/info.max_score:.1%}) [Best: {highscore}]".format(info.score))

[Copilot]

Pull Request Overview

This PR adds support for the SWE-Smith benchmark by introducing a new environment and agent, along with various utility and configuration updates.

• Implement AgentSolution to apply gold patches and evaluate directly
• Add SWESmithEnv with dataset loading, Docker handling, and split support
• Enhance scripts, container setup, and config loading logic; fix PDB exit handling

Reviewed Changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
scripts/run.py	Add problem‐grouping utilities; adjust dataset split logic
scripts/config_swesmith.yaml	New SWE-Smith config template
debug_gym/utils.py	Define and create cache/config directories at import
debug_gym/init_llm_config.py	Use DEBUG_GYM_CONFIG_DIR for default dest path
debug_gym/gym/tools/pdb.py	Fix PDB sys.exit() message handling
debug_gym/gym/terminal.py	Enable host networking; simplify container naming
debug_gym/gym/envs/swe_smith.py	New SWESmithEnv implementation
debug_gym/gym/envs/swe_bench.py	Update import paths and dynamic split support
debug_gym/gym/envs/env.py	Adjust entrypoint preparation for uv and python
debug_gym/gym/envs/init.py	Register SWESmithEnv in select_env
debug_gym/agents/utils.py	Allow argparse extend action for params
debug_gym/agents/solution_agent.py	Add solution agent to apply patches and eval
debug_gym/agents/init.py	Import AgentSolution
MANIFEST.in	Include YAML configs in package manifest

Comments suppressed due to low confidence (2)

debug_gym/gym/terminal.py:456

• [nitpick] Remove commented-out code for renaming/reloading the container if it's no longer needed, to reduce clutter.

#        # container.rename(container_name)

debug_gym/agents/solution_agent.py:45

• This f-string uses nested double quotes and will cause a syntax error. Use single quotes inside or escape the inner quotes, for example:
  f"git -C {self.env.working_dir} apply {getattr(self.env, 'git_apply_args', '')} -".

command = f"git -C {self.env.working_dir} apply {getattr(self.env, "git_apply_args", "")} -"

[xingdi-eric-yuan]

Need a short tutorial on how to run official swe-smith tasks and customized tasks generated from swe-smith, this will include the use of the solution agent.