[BUG] Context already been set

[apomalyn]

Description of the issue

When running mutmut3 I always end up with the same error during the stats step:

RuntimeError: context has already been set

During the stats step on the first

from mutmut.__main__ import record_trampoline_hit

mutmut.__main__ is loaded and so set_start_method('fork') is executed a second time.

Whatever the project I use, as long as I use mutmut >3, I always get that error. I've tried even with a dummy project (CRUD of FastAPI), and always end up with the same issue. I'm out of ideas of what to do, so I'm asking here if anyone has an idea of what is happening here.

Specs

OS: MacOS 15.7 (M4 pro)
Python: 3.13, 3.14
mutmut: 3.4.0
pytest: 9.0.2
Project layout: flat

[Otto-AA]

Hi, can you try it with a very simple repo, e.g. https://github.com/Otto-AA/mini-repro-repo ?

And/or can you share your dummy project? Maybe it's FastAPI related?

[Otto-AA]

During the stats step on the first
from mutmut.main import record_trampoline_hit
mutmut.main is loaded and so set_start_method('fork') is executed a second time.

Do you mean, that it is loaded twice within mutmut, or once within your application and once within mutmut? I guess it could be a problem if FastAPI sets the runtime and we try to do the same.

[apomalyn]

Hi there! I was thinking of building the simplest project I could. It's not FastAPI related (unfortunately)

project
├── app
│   └── service.py
├── tests
│   └── test_service.py
└── pyproject.toml

pyproject.toml

[build-system]
requires = ["setuptools>=80.0.0"]
build-backend = "setuptools.build_meta"

[tool.setuptools.packages.find]
where = ["."]
include = ["app", "app.*"]

[project]
name = "app"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
    "requests==2.32.5",
]

[project.optional-dependencies]
test = [
    "pytest>=9.0.2",
    "mutmut>=3.3.0"
]

[tool.pytest.ini_options]
pythonpath = ["app"]
testpaths = ["tests"]

[tool.mutmut]
debug = true
paths_to_mutate = ["app"]
tests_dir = ["tests"]

service.py

from logging import getLogger
import requests

logger = getLogger(__name__)

class Service:
    _timeout: int = 10
    base_url: str = "https://api.test.com"

    def load_file(self, url: str) -> str:
        try:
            if not url.startswith(self.base_url):
                logger.error("URL %s is not within the allowed API domain.", url)
                raise ValueError(f"URL %s is not within the allowed API domain.")
            response = requests.get(url, timeout=self._timeout)
            response.raise_for_status()
            return response.text
        except requests.RequestException as e:
            logger.error("Error loading file from URL %s: %s", url, e)
            raise e

test_service.py

from unittest.mock import patch, Mock

import pytest

from app.service import Service
from requests import Response


@patch("app.service.requests.get", return_value=Mock(spec=Response, text="File content here"))
def test_load_file_success(mock_get):
    instance = Service()

    url = f"https://api.test.com/raw/file.txt"
    result = instance.load_file(url)

    mock_get.assert_called_with(url, timeout=10)
    assert result == "File content here"


@patch("app.service.requests.get")
def test_load_file_invalid_base(mock_get):
    instance = Service()

    with pytest.raises(ValueError):
        url = f"https://invalid.com/raw/file.txt"
        instance.load_file(url)

    mock_get.assert_not_called()

And here is the full error in case that can help you:

Full error

⠋ Generating mutants
     also copying tests
     also copying test
     also copying setup.cfg
     also copying pyproject.toml
app/service.py
app/__init__.py

    done in 150ms

⠙ Running stats
     python -m pytest  "-vv" "--rootdir=." "--tb=native" "-x" "-q" "tests"
============================= test session starts ==============================
platform darwin -- Python 3.12.12, pytest-9.0.2, pluggy-1.6.0 -- /Users/X/code/PythonProject/.venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/X/code/PythonProject/mutants
configfile: pyproject.toml
collecting ... collected 2 items

tests/test_service.py::test_load_file_success FAILED                     [ 50%]

=================================== FAILURES ===================================
____________________________ test_load_file_success ____________________________
Traceback (most recent call last):
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/_pytest/runner.py", line 353, in from_call
    result: TResult | None = func()
                             ^^^^^^
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/_pytest/runner.py", line 245, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
    yield
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/_pytest/runner.py", line 179, in pytest_runtest_call
    item.runtest()
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/_pytest/python.py", line 1720, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
    result = testfunction(**testargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/X/.local/share/mise/installs/python/3.12.12/lib/python3.12/unittest/mock.py", line 1396, in patched
    return func(*newargs, **newkeywargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/X/code/PythonProject/tests/test_service.py", line 14, in test_load_file_success
    result = instance.load_file(url)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/X/code/PythonProject/mutants/app/service.py", line 370, in load_file
    result = _mutmut_trampoline(object.__getattribute__(self, "xǁServiceǁload_file__mutmut_orig"), object.__getattribute__(self, "xǁServiceǁload_file__mutmut_mutants"), args, kwargs, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/X/code/PythonProject/mutants/app/service.py", line 22, in _mutmut_trampoline
    from mutmut.__main__ import record_trampoline_hit
  File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/mutmut/__main__.py", line 978, in <module>
    set_start_method('fork')
  File "/Users/X/.local/share/mise/installs/python/3.12.12/lib/python3.12/multiprocessing/context.py", line 247, in set_start_method
    raise RuntimeError('context has already been set')
RuntimeError: context has already been set
=========================== short test summary info ============================
FAILED tests/test_service.py::test_load_file_success - RuntimeError: context ...
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
============================== 1 failed in 0.07s ===============================
    exit code 1

failed to collect stats. runner returned 1

FYI I've tested with python 3.12, 3.13, and 3.14

[Otto-AA]

Thanks for sharing the example. I could not reproduce locally on Ubuntu 25.10 (python 3.14.0; uv for installing dependencies).

When you add the following print statement in .venv/lib/python3.14/site-packages/mutmut/__main__.py:

# ....

# used to copy the global mutmut.config to subprocesses
print('set_start_method', ''.join(traceback.format_stack()).replace('\n', '\t'))
set_start_method('fork')
START_TIMES_BY_PID_LOCK = Lock()

# ...

And then run mutmut run | grep 'set_start_method'. Do you see one "set_start_method" + stacktrace, or two of them? Normally, you should only see one of them, as python does not import the same file twice usually (but caches it in sys.modules). I'd be confused if it would execute mutmuts __main__ twice, but who knows 🤷

And to be sure: Here's your repository with pinned versions:

git clone git@github.com:Otto-AA/mutmut-repro.git
cd mutmut-repro
uv run mutmut run
# ...
# ⠇ 24/24  🎉 8 🫥 0  ⏰ 0  🤔 0  🙁 16  🔇 0
# 107.63 mutations/second

[apomalyn]

Hi, can you try it with a very simple repo, e.g. https://github.com/Otto-AA/mini-repro-repo ?

And/or can you share your dummy project? Maybe it's FastAPI related?

Seems like it's working with this project. Maybe it's a flat layout issue? 🤔 I don't really know uv so maybe it's setuptools the problem.

And then run mutmut run | grep 'set_start_method'. Do you see one "set_start_method" + stacktrace, or two of them? Normally, you should only see one of them, as python does not import the same file twice usually (but caches it in sys.modules). I'd be confused if it would execute mutmuts __main__ twice, but who knows 🤷

Here is what I see:

set_start_method   File "/Users/X/code/PythonProject/.venv/bin/mutmut", line 5, in <module>          from mutmut.__main__ import cli       File "<frozen importlib._bootstrap>", line 1360, in _find_and_load   File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked     File "<frozen importlib._bootstrap>", line 935, in _load_unlocked    File "<frozen importlib._bootstrap_external>", line 999, in exec_module         File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed    File "/Users/X/code/PythonProject/.venv/lib/python3.12/site-packages/mutmut/__main__.py", line 979, in <module>         print('set_start_method', ''.join(traceback.format_stack()).replace('\n', '\t')) 

And to be sure: Here's your repository with pinned versions:

git clone git@github.com:Otto-AA/mutmut-repro.git
cd mutmut-repro
uv run mutmut run

...

⠇ 24/24 🎉 8 🫥 0 ⏰ 0 🤔 0 🙁 16 🔇 0

107.63 mutations/second

I wasn't expecting 24 mutation tests from this simple method 😂 Let me try running it locally

[apomalyn]

And to be sure: Here's your repository with pinned versions:

Oh wait, seems like it's working now? 🤔 Maybe a cache issue? I tried with uv and it works, then did the same thing I normally do:

python -m venv .venv
source .venv/bin/activate
pip install . '.[test]'
python -m mutmut run

And it's working on the dummy project 🤷

[apomalyn]

Maybe a cache/python issue 🤷 Let me try on the "true" project and hope it works because I can't share this one 😅

[apomalyn]

🎉 🎉 It's working (kinda) 🎉 🎉
Now I get KeyError: 'MUTANT_UNDER_TEST' during the stats steps. I had a test that cleared the os.environ dict

I will try to figure out what it happening and if I can make a dummy project that reproduce this error and so create an issue if I can't resolve it. (if you have any ideas I'm all ears!)

Thank you again!

[apomalyn]

I figured out the issue...I'm running python -m mutmut run while mutmut run works correctly (except the 'MUTANT_UNDER_TEST' issue), python -m mutmut run ends up with a context already been set issue.

I can't reproduce it with a dummy project for now, I will close this issue for now and try to find a way to reproduce it. In the full project I do see multiple time set_start_method called (with the grep from early).

In the meantime thank you for your time

[apomalyn]

Alright found a way to reproduce it!

1. Delete the mutants folder
2. Run python -m mutmut run

(so simple to reproduce but took me some time to find it --') The fix is pretty simple:

if get_start_method(allow_none=True) is None:
    set_start_method('fork')

Or:

if __name__ == "__main__":
    set_start_method('fork')

I can open a PR for it if you want.

[Otto-AA]

Thanks, I could reproduce it with that. I'm not sure if the fix is really that simple.

The cause

A much smaller main.py to show the reason:

# __main__.py
import sys
print('mutmut.__main__ in sys.modules? ', 'mutmut.__main__' in sys.modules)

Running:

• python -m mutmut run outputs "mutmut.main in sys.modules? False"
• mutmut run outputs "mutmut.main in sys.modules? True"

The first one directly executes the module, which apparently does not cache it into sys.modules. The second one executes .venv/bin/mutmut which is a small wrapper that imports mutmut and thus caches it in sys.modules:

#!/home/.../.venv/bin/python
# -*- coding: utf-8 -*-
import sys
from mutmut.__main__ import cli
if __name__ == "__main__":
    if sys.argv[0].endswith("-script.pyw"):
        sys.argv[0] = sys.argv[0][:-11]
    elif sys.argv[0].endswith(".exe"):
        sys.argv[0] = sys.argv[0][:-4]
    sys.exit(cli())

When sys.modules includes mutmut.__main__ it won't be re-executed. When it's not cached there, then importing mutmut.__main__ will re-execute it. This is the problem you observed.

The fix?

mutmut uses many global variables. If we execute it twice, I could imagine that the run method uses different global variables than the from mutmut.__main__ import record_trampoline_hit method. Currently, it probably would work because all important globals are defined in __init__.py.

If you want to make a PR, I think we should (1) ensure that set_start_method is only called once, as you proposed and (2) also emits a warnings.warn message that mutmut run is the preferred method to run mutmut, if mutmut.__main__ is not cached in sys.modules.

Alternative fix

I guess we could do sys.modules['mutmut.__main__'] = sys.modules['__main__'] to manually cache it. I'm not sure if there would be downsides to this, but it feels a bit hacky.

[Otto-AA]

For background info on the issue: https://peps.python.org/pep-0499/ (not implemented fix for this in python)

[apomalyn]

I will try to see if there is an impact and add the warning.

If there is an impact, I think a simple doc update so we know it's a known issue and mutmut should not be used as a module should be enough.

Once again, thank you for your time and work on mutmut!

I'm impatient to get the next release so we can easily extract the stats on the CI!