Skip to content

Delete organization fails with 500 #235

@noctarius

Description

@noctarius
2026-02-20T14:46:02+0000 - INFO - uvicorn.access - 10.244.0.80:40214 - "DELETE /organizations/01KHXR3YZCAPKXPS3WG1HD1SSH/ HTTP/1.1" 500
2026-02-20T14:46:02+0000 - ERROR - uvicorn.error - Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 578, in _executemany
    return await self._connection.executemany(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, seq_of_parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/asyncpg/connection.py", line 401, in executemany
    return await self._executemany(command, args, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/asyncpg/connection.py", line 1997, in _executemany
    result, _ = await self._do_execute(
                ^^^^^^^^^^^^^^^^^^^^^^^
        query, executor, timeout, record_class=record_class
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/asyncpg/connection.py", line 2033, in _do_execute
    result = await executor(stmt, None)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 266, in bind_execute_many
asyncpg.exceptions.UniqueViolationError: duplicate key value violates unique constraint "uq_limit_global"
DETAIL:  Key (entity_type, resource)=(org, milli_vcpu) already exists.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1936, in _exec_single_context
    self.dialect.do_executemany(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        cursor,
        ^^^^^^^
    ...<2 lines>...
        context,
        ^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/engine/default.py", line 949, in do_executemany
    cursor.executemany(statement, parameters)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 590, in executemany
    return self._adapt_connection.await_(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._executemany(operation, seq_of_parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 582, in _executemany
    self._handle_exception(error)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 513, in _handle_exception
    self._adapt_connection._handle_exception(error)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 797, in _handle_exception
    raise translated_error from error
sqlalchemy.dialects.postgresql.asyncpg.AsyncAdapt_asyncpg_dbapi.IntegrityError: <class 'asyncpg.exceptions.UniqueViolationError'>: duplicate key value violates unique constraint "uq_limit_global"
DETAIL:  Key (entity_type, resource)=(org, milli_vcpu) already exists.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/uvicorn/protocols/http/h11_impl.py", line 410, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        self.scope, self.receive, self.send
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/fastapi/applications.py", line 1134, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/applications.py", line 107, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/cors.py", line 87, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.13/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 716, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 736, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 290, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/fastapi/routing.py", line 119, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.13/site-packages/fastapi/routing.py", line 105, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/fastapi/routing.py", line 424, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<3 lines>...
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/fastapi/routing.py", line 312, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/simplyblock/vela/api/organization/__init__.py", line 204, in delete
    await session.commit()
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 1000, in commit
    await greenlet_spawn(self.sync_session.commit)
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 203, in greenlet_spawn
    result = context.switch(value)
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 2030, in commit
    trans.commit(_to_root=True)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "<string>", line 2, in commit
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 1311, in commit
    self._prepare_impl()
    ~~~~~~~~~~~~~~~~~~^^
  File "<string>", line 2, in _prepare_impl
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 1286, in _prepare_impl
    self.session.flush()
    ~~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 4331, in flush
    self._flush(objects)
    ~~~~~~~~~~~^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 4466, in _flush
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 4427, in _flush
    flush_context.execute()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/orm/unitofwork.py", line 466, in execute
    rec.execute(self)
    ~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/orm/unitofwork.py", line 642, in execute
    util.preloaded.orm_persistence.save_obj(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self.mapper,
        ^^^^^^^^^^^^
        uow.states_for_mapper_hierarchy(self.mapper, False, False),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        uow,
        ^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/orm/persistence.py", line 85, in save_obj
    _emit_update_statements(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        base_mapper,
        ^^^^^^^^^^^^
    ...<3 lines>...
        update,
        ^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/orm/persistence.py", line 912, in _emit_update_statements
    c = connection.execute(
        statement, multiparams, execution_options=execution_options
    )
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/sql/elements.py", line 527, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 2363, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1936, in _exec_single_context
    self.dialect.do_executemany(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        cursor,
        ^^^^^^^
    ...<2 lines>...
        context,
        ^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/engine/default.py", line 949, in do_executemany
    cursor.executemany(statement, parameters)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 590, in executemany
    return self._adapt_connection.await_(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._executemany(operation, seq_of_parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 582, in _executemany
    self._handle_exception(error)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 513, in _handle_exception
    self._adapt_connection._handle_exception(error)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 797, in _handle_exception
    raise translated_error from error
sqlalchemy.exc.IntegrityError: (sqlalchemy.dialects.postgresql.asyncpg.IntegrityError) <class 'asyncpg.exceptions.UniqueViolationError'>: duplicate key value violates unique constraint "uq_limit_global"
DETAIL:  Key (entity_type, resource)=(org, milli_vcpu) already exists.
[SQL: UPDATE resourcelimit SET org_id=$1::UUID WHERE resourcelimit.id = $2::UUID]
[parameters: [(None, UUID('019c7b81-fc57-363e-2391-23f3a3ade082')), (None, UUID('019c7b81-fc5a-f48a-6606-e1fe42bc8c86')), (None, UUID('019c7b81-fc5a-f48a-6606-e1fe42bc8c87')), (None, UUID('019c7b81-fc5b-9afd-026c-daad9973f03e')), (None, UUID('019c7b81-fc5c-48f9-ccf1-6dc48d4c459f'))]]
(Background on this error at: https://sqlalche.me/e/20/gkpj)
2026-02-20T14:46:03+0000 - INFO - httpx - HTTP Request: DELETE https://api.cloudflare.com/client/v4/zones/797ef508b44ef106b7ccc3708169a337/dns_records/b5dd35766b63d693f77406e6d958b7dc "HTTP/1.1 200 OK"
2026-02-20T14:46:03+0000 - INFO - simplyblock.vela.deployment - Deleted Cloudflare DNS CNAME record 01khxqnx5zv205rzc0f6zmjse6.demo.vela.run (id=b5dd35766b63d693f77406e6d958b7dc)
2026-02-20T14:46:03+0000 - INFO - httpx - HTTP Request: DELETE https://api.cloudflare.com/client/v4/zones/797ef508b44ef106b7ccc3708169a337/dns_records/0206fd689f894625eeef59076be3fc43 "HTTP/1.1 200 OK"
2026-02-20T14:46:03+0000 - INFO - simplyblock.vela.deployment - Deleted Cloudflare DNS CNAME record db.01khxqnx5zv205rzc0f6zmjse6.demo.vela.run (id=0206fd689f894625eeef59076be3fc43)
2026-02-20T14:46:03+0000 - INFO - simplyblock.vela.deployment - Deleted autoscaler VM vela-autoscaler-vm in namespace vela-01khxqnx5zv205rzc0f6zmjse6
2026-02-20T14:46:03+0000 - INFO - simplyblock.vela.deployment.grafana - Deleting Grafana objects branch=01KHXQNX5ZV205RZC0F6ZMJSE6
2026-02-20T14:46:03+0000 - INFO - httpx - HTTP Request: DELETE https://demo.vela.run/grafana/api/dashboards/uid/01KHXQNX5ZV205RZC0F6ZMJSE6 "HTTP/1.1 200 OK"
2026-02-20T14:46:03+0000 - INFO - simplyblock.vela.deployment.grafana - Deleted Grafana dashboard uid=01KHXQNX5ZV205RZC0F6ZMJSE6
2026-02-20T14:46:03+0000 - INFO - httpx - HTTP Request: GET https://demo.vela.run/grafana/api/search?tag=01KHXQNX5ZV205RZC0F6ZMJSE6&type=dash-db "HTTP/1.1 200 OK"

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingvela-controllerIssue is related to vela-controller

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions