Skip to content

Fix: Shut down the server with one keyboard interrupt (#94)#8936

Open
lstein wants to merge 2 commits intoinvoke-ai:mainfrom
lstein:lstein/feature/elegant-shutdown
Open

Fix: Shut down the server with one keyboard interrupt (#94)#8936
lstein wants to merge 2 commits intoinvoke-ai:mainfrom
lstein:lstein/feature/elegant-shutdown

Conversation

@lstein
Copy link
Collaborator

@lstein lstein commented Mar 2, 2026

Summary

This PR cleans up what happens when you ^C the running invokeai-web server.

Before

Two ^C needed, and terminal looks lilke this:

^C[2026-02-26 18:56:13,606]::[ModelInstallService]::INFO --> Installer thread 139253501650624 exiting                                                                                                         
Traceback (most recent call last):                                                                      
  File "/home/lstein/invokeai-lstein/.venv/bin/invokeai-web", line 12, in <module>                                                                                                                              
    sys.exit(run_app())                                                                                 
             ^^^^^^^^^                                                                                  
  File "/home/lstein/Projects/InvokeAI-lstein/invokeai/app/run_app.py", line 103, in run_app                                                                                                                    
    loop.run_until_complete(server.serve())                                                                                                                                                                     
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py", line 678, in run_until_complete                                                            
    self.run_forever()                                                                                                                                                                                          
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py", line 645, in run_forever                                                                   
    self._run_once()                                                                                                                                                                                            
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py", line 1999, in _run_once                                                                    
    handle._run()                                                                                                                                                                                               
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/events.py", line 88, in _run                                                                                
    self._context.run(self._callback, *self._args)                                                                                                                                                              
  File "/home/lstein/invokeai-lstein/.venv/lib/python3.12/site-packages/uvicorn/server.py", line 70, in serve                                                                                                   
    with self.capture_signals():                                                                                                                                                                                
         ^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                 
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 144, in __exit__                                                                               
    next(self.gen)                                                                                                                                                                                              
  File "/home/lstein/invokeai-lstein/.venv/lib/python3.12/site-packages/uvicorn/server.py", line 331, in capture_signals                                                                                        
    signal.raise_signal(captured_signal)   
KeyboardInterrupt                                                                                                                                                                      
 ^C
 Exception ignored in: <module 'threading' from '/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/threading.py'>                                                            
Traceback (most recent call last):                                                                                                                                                                              
  File "/home/lstein/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/threading.py", line 1624, in _shutdown                                                                              
    lock.acquire()                                                                                                                                                                                              
KeyboardInterrupt:                                                                                                                                                                                              

After:

Only one ^C needed, and terminal shows this:

^C
[2026-02-26 19:35:26,931]::[ModelInstallService]::INFO --> Installer thread 127057625994944 exiting
[2026-02-26 19:35:26,956]::[InvokeAI]::INFO --> InvokeAI shutting down...

Related Issues / Discussions

QA Instructions

  1. Start the server, run a few generations, then send a keyboard interrupt with ^C.
  2. Observe the terminal log messages, and see if server stops without additional intervention.

Merge Plan

Simple merge.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

* Initial plan

* Handle KeyboardInterrupt in run_app to allow single Ctrl+C shutdown

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

* Force os._exit(0) on KeyboardInterrupt to avoid hanging on background threads

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

Fix graceful shutdown to wait for download/install worker threads (#102)

* Initial plan

* Replace os._exit(0) with ApiDependencies.shutdown() on KeyboardInterrupt

Instead of immediately force-exiting the process on CTRL+C, call
ApiDependencies.shutdown() to gracefully stop the download and install
manager services, allowing active work to complete or cancel cleanly
before the process exits.

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

* Make stop() idempotent in download and model install services

When CTRL+C is pressed, uvicorn's graceful shutdown triggers the FastAPI
lifespan which calls ApiDependencies.shutdown(), then a KeyboardInterrupt
propagates from run_until_complete() hitting the except block which tries
to call ApiDependencies.shutdown() a second time.

Change both stop() methods to return silently (instead of raising) when
the service is not running. This handles:
- Double-shutdown: lifespan already stopped the services
- Early interrupt: services were never fully started

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

Fix shutdown hang on session processor thread lock (#108)

* Initial plan

* Fix shutdown hang: wake session processor thread on stop() and mark daemon

Co-authored-by: lstein <111189+lstein@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
@lstein lstein assigned lstein and JPPhoto and unassigned lstein Mar 2, 2026
@github-actions github-actions bot added python PRs that change python files services PRs that change app services labels Mar 2, 2026
@lstein lstein changed the title Fix: Kill the server with one keyboard interrupt (#94) Fix: Shut down the server with one keyboard interrupt (#94) Mar 2, 2026
@JPPhoto
Copy link
Collaborator

JPPhoto commented Mar 2, 2026

Sorry, this is still waiting on a lock somewhere and Invoke doesn't shut down - same as in the last PR.

…generation hang (#112)

Fix: cancel pending asyncio tasks before loop.close() to suppress destroyed-task warnings
Fix: suppress stack trace when dispatching events after event loop is closed on shutdown
Fix: cancel in-progress generation on stop() to prevent core dump during mid-flight Ctrl+C

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
@github-actions github-actions bot added the python-tests PRs that change python tests label Mar 2, 2026
Copy link
Collaborator

@JPPhoto JPPhoto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Lots of belt and suspenders action here but I think that's what's needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python PRs that change python files python-tests PRs that change python tests services PRs that change app services

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants