Issue: Unclear Domain and Port 3003 Communication Preventing Service from Working

[BertrandakaBurt]

Description:

I am encountering an error in my logs related to the domain kichat.korn.local and port 3003. The application is trying to connect to this domain on port 3003, but there is no service running on this port, and I am unsure what the domain represents or why the connection is being attempted. This is preventing the service from working as expected. None of the feature work for me.

Error Details:

ConnectionError` at /update_files/update/
HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7b0c9110fa70>: Failed to establish a new connection: [Errno 111] Connection refused'))

Request` Method: POST
Request URL: http://10.255.255.40:8000/update_files/update/

Django Version: 5.1.7
Exception Type: ConnectionError
Exception Value:
HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7b0c9110fa70>: Failed to establish a new connection: [Errno 111] Connection refused'))

Exception Location: /usr/local/lib/python3.12/site-packages/requests/adapters.py, line 700, in send
Observations:
The domain kichat.korn.local and port 3003 are being referenced, but no service is running on port 3003.

This is causing a ConnectionError and preventing the service from working.

I am unsure of the purpose of this domain and port configuration.

What I need help with:
Clarification on what kichat.korn.local is and why the application is attempting to connect to it.

Investigation into why port 3003 is being used when no service is running on it.

Guidance on how to resolve this issue and prevent the application from attempting to connect to an unreachable service.
Thanks,
Also,
I am on Ubuntu desktop 22.04.5 LTS

Logs
Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
any_folder | response = self._make_request(
any_folder | ^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
any_folder | raise new_e
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
any_folder | self._validate_conn(conn)
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
any_folder | conn.connect()
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
any_folder | self.sock = sock = self._new_conn()
any_folder | ^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
any_folder | raise NewConnectionError(
any_folder | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x79382c3c9d60>: Failed to establish a new connection: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
any_folder | resp = conn.urlopen(
any_folder | ^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
any_folder | retries = retries.increment(
any_folder | ^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
any_folder | raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c3c9d60>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | During handling of the above exception, another exception occurred:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/django_q/worker.py", line 103, in worker
any_folder | res = f(*task["args"], **task["kwargs"])
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/app/check_for_files/tasks.py", line 552, in main
any_folder | sort_files.sort_files()
any_folder | File "/app/check_for_files/sort_files.py", line 52, in sort_files
any_folder | response = requests.get("https://kichat.korn.local:3003/api/v1/workspaces", headers=self.headers_json, timeout=10, verify=False).json()
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 73, in get
any_folder | return request("get", url, params=params, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 59, in request
any_folder | return session.request(method=method, url=url, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
any_folder | resp = self.send(prep, **send_kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
any_folder | r = adapter.send(request, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
any_folder | raise ConnectionError(e, request=request)
any_folder | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c3c9d60>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | Failed 'check_for_files.tasks.main' (carpet-fish-butter-mississippi) - HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c183860>: Failed to establish a new connection: [Errno 111] Connection refused')) : Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
any_folder | sock = connection.create_connection(
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
any_folder | raise err
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
any_folder | sock.connect(sa)
any_folder | ConnectionRefusedError: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
any_folder | response = self._make_request(
any_folder | ^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
any_folder | raise new_e
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
any_folder | self._validate_conn(conn)
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
any_folder | conn.connect()
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
any_folder | self.sock = sock = self._new_conn()
any_folder | ^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
any_folder | raise NewConnectionError(
any_folder | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x79382c183860>: Failed to establish a new connection: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
any_folder | resp = conn.urlopen(
any_folder | ^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
any_folder | retries = retries.increment(
any_folder | ^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
any_folder | raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c183860>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | During handling of the above exception, another exception occurred:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/django_q/worker.py", line 103, in worker
any_folder | res = f(*task["args"], **task["kwargs"])
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/app/check_for_files/tasks.py", line 552, in main
any_folder | sort_files.sort_files()
any_folder | File "/app/check_for_files/sort_files.py", line 52, in sort_files
any_folder | response = requests.get("https://kichat.korn.local:3003/api/v1/workspaces", headers=self.headers_json, timeout=10, verify=False).json()
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 73, in get
any_folder | return request("get", url, params=params, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 59, in request
any_folder | return session.request(method=method, url=url, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
any_folder | resp = self.send(prep, **send_kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
any_folder | r = adapter.send(request, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
any_folder | raise ConnectionError(e, request=request)
any_folder | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c183860>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | Done Updating, task was successful. Found 0 files to add, 0 files that changed and 0 files that got deleted.
any_folder | Done Updating, task was successful. Found 0 files to add, 0 files that changed and 0 files that got deleted.
any_folder | Done Updating, task was successful. Found 0 files to add, 0 files that changed and 0 files that got deleted.
any_folder | Failed 'check_for_files.tasks.main' (kilo-cardinal-may-one) - HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c189490>: Failed to establish a new connection: [Errno 111] Connection refused')) : Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
any_folder | sock = connection.create_connection(
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
any_folder | raise err
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
any_folder | sock.connect(sa)
any_folder | ConnectionRefusedError: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
any_folder | response = self._make_request(
any_folder | ^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
any_folder | raise new_e
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
any_folder | self._validate_conn(conn)
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
any_folder | conn.connect()
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
any_folder | self.sock = sock = self._new_conn()
any_folder | ^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
any_folder | raise NewConnectionError(
any_folder | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x79382c189490>: Failed to establish a new connection: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
any_folder | resp = conn.urlopen(
any_folder | ^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
any_folder | retries = retries.increment(
any_folder | ^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
any_folder | raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c189490>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | During handling of the above exception, another exception occurred:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/django_q/worker.py", line 103, in worker
any_folder | res = f(*task["args"], **task["kwargs"])
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/app/check_for_files/tasks.py", line 552, in main
any_folder | sort_files.sort_files()
any_folder | File "/app/check_for_files/sort_files.py", line 52, in sort_files
any_folder | response = requests.get("https://kichat.korn.local:3003/api/v1/workspaces", headers=self.headers_json, timeout=10, verify=False).json()
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 73, in get
any_folder | return request("get", url, params=params, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 59, in request
any_folder | return session.request(method=method, url=url, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
any_folder | resp = self.send(prep, **send_kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
any_folder | r = adapter.send(request, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
any_folder | raise ConnectionError(e, request=request)
any_folder | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c189490>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | Failed 'check_for_files.tasks.main' (uniform-hot-lemon-green) - HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c3cbe90>: Failed to establish a new connection: [Errno 111] Connection refused')) : Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
any_folder | sock = connection.create_connection(
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
any_folder | raise err
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
any_folder | sock.connect(sa)
any_folder | ConnectionRefusedError: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
any_folder | response = self._make_request(
any_folder | ^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
any_folder | raise new_e
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
any_folder | self._validate_conn(conn)
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
any_folder | conn.connect()
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
any_folder | self.sock = sock = self._new_conn()
any_folder | ^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
any_folder | raise NewConnectionError(
any_folder | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x79382c3cbe90>: Failed to establish a new connection: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
any_folder | resp = conn.urlopen(
any_folder | ^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
any_folder | retries = retries.increment(
any_folder | ^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
any_folder | raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c3cbe90>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | During handling of the above exception, another exception occurred:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/django_q/worker.py", line 103, in worker
any_folder | res = f(*task["args"], **task["kwargs"])
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/app/check_for_files/tasks.py", line 552, in main
any_folder | sort_files.sort_files()
any_folder | File "/app/check_for_files/sort_files.py", line 52, in sort_files
any_folder | response = requests.get("https://kichat.korn.local:3003/api/v1/workspaces", headers=self.headers_json, timeout=10, verify=False).json()
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 73, in get
any_folder | return request("get", url, params=params, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 59, in request
any_folder | return session.request(method=method, url=url, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
any_folder | resp = self.send(prep, **send_kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
any_folder | r = adapter.send(request, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
any_folder | raise ConnectionError(e, request=request)
any_folder | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c3cbe90>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | Failed 'check_for_files.tasks.main' (floor-yellow-magnesium-five) - HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c183980>: Failed to establish a new connection: [Errno 111] Connection refused')) : Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
any_folder | sock = connection.create_connection(
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
any_folder | raise err
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
any_folder | sock.connect(sa)
any_folder | ConnectionRefusedError: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
any_folder | response = self._make_request(
any_folder | ^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
any_folder | raise new_e
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
any_folder | self._validate_conn(conn)
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
any_folder | conn.connect()
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
any_folder | self.sock = sock = self._new_conn()
any_folder | ^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
any_folder | raise NewConnectionError(
any_folder | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x79382c183980>: Failed to establish a new connection: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
any_folder | resp = conn.urlopen(
any_folder | ^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
any_folder | retries = retries.increment(
any_folder | ^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
any_folder | raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c183980>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | During handling of the above exception, another exception occurred:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/django_q/worker.py", line 103, in worker
any_folder | res = f(*task["args"], **task["kwargs"])
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/app/check_for_files/tasks.py", line 552, in main
any_folder | sort_files.sort_files()
any_folder | File "/app/check_for_files/sort_files.py", line 52, in sort_files
any_folder | response = requests.get("https://kichat.korn.local:3003/api/v1/workspaces", headers=self.headers_json, timeout=10, verify=False).json()
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 73, in get
any_folder | return request("get", url, params=params, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 59, in request
any_folder | return session.request(method=method, url=url, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
any_folder | resp = self.send(prep, **send_kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
any_folder | r = adapter.send(request, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
any_folder | raise ConnectionError(e, request=request)
any_folder | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c183980>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | queueing from default
any_folder | Process-1b056549f79a4888a14d2c0553acb298 processing lion-black-timing-skylark 'check_for_files.tasks.main' [check_files_schedule]
any_folder | Process-d99181439b5b4cc4bc9466c306cda455 processing washington-october-stream-magnesium 'check_for_files.tasks.main' [check_files_schedule]
any_folder | Done Updating, task was successful. Found 0 files to add, 0 files that changed and 0 files that got deleted.
any_folder | Done Updating, task was successful. Found 0 files to add, 0 files that changed and 0 files that got deleted.
any_folder | Failed 'check_for_files.tasks.main' (washington-october-stream-magnesium) - HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c188200>: Failed to establish a new connection: [Errno 111] Connection refused')) : Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
any_folder | sock = connection.create_connection(
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
any_folder | raise err
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
any_folder | sock.connect(sa)
any_folder | ConnectionRefusedError: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
any_folder | response = self._make_request(
any_folder | ^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
any_folder | raise new_e
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
any_folder | self._validate_conn(conn)
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
any_folder | conn.connect()
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
any_folder | self.sock = sock = self._new_conn()
any_folder | ^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
any_folder | raise NewConnectionError(
any_folder | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x79382c188200>: Failed to establish a new connection: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
any_folder | resp = conn.urlopen(
any_folder | ^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
any_folder | retries = retries.increment(
any_folder | ^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
any_folder | raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c188200>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | During handling of the above exception, another exception occurred:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/django_q/worker.py", line 103, in worker
any_folder | res = f(*task["args"], **task["kwargs"])
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/app/check_for_files/tasks.py", line 552, in main
any_folder | sort_files.sort_files()
any_folder | File "/app/check_for_files/sort_files.py", line 52, in sort_files
any_folder | response = requests.get("https://kichat.korn.local:3003/api/v1/workspaces", headers=self.headers_json, timeout=10, verify=False).json()
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 73, in get
any_folder | return request("get", url, params=params, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 59, in request
any_folder | return session.request(method=method, url=url, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
any_folder | resp = self.send(prep, **send_kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
any_folder | r = adapter.send(request, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
any_folder | raise ConnectionError(e, request=request)
any_folder | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c188200>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | Failed 'check_for_files.tasks.main' (lion-black-timing-skylark) - HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c188380>: Failed to establish a new connection: [Errno 111] Connection refused')) : Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
any_folder | sock = connection.create_connection(
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
any_folder | raise err
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
any_folder | sock.connect(sa)
any_folder | ConnectionRefusedError: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
any_folder | response = self._make_request(
any_folder | ^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
any_folder | raise new_e
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
any_folder | self._validate_conn(conn)
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
any_folder | conn.connect()
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
any_folder | self.sock = sock = self._new_conn()
any_folder | ^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
any_folder | raise NewConnectionError(
any_folder | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x79382c188380>: Failed to establish a new connection: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
any_folder | resp = conn.urlopen(
any_folder | ^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
any_folder | retries = retries.increment(
any_folder | ^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
any_folder | raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c188380>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | During handling of the above exception, another exception occurred:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/django_q/worker.py", line 103, in worker
any_folder | res = f(*task["args"], **task["kwargs"])
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/app/check_for_files/tasks.py", line 552, in main
any_folder | sort_files.sort_files()
any_folder | File "/app/check_for_files/sort_files.py", line 52, in sort_files
any_folder | response = requests.get("https://kichat.korn.local:3003/api/v1/workspaces", headers=self.headers_json, timeout=10, verify=False).json()
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 73, in get
any_folder | return request("get", url, params=params, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 59, in request
any_folder | return session.request(method=method, url=url, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
any_folder | resp = self.send(prep, **send_kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
any_folder | r = adapter.send(request, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
any_folder | raise ConnectionError(e, request=request)
any_folder | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c188380>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | queueing from default
any_folder | Process-159bc9371b66427c82f13cef39ca13ea processing edward-texas-sink-fish 'check_for_files.tasks.main' [check_files_schedule]
any_folder | Done Updating, task was successful. Found 0 files to add, 0 files that changed and 0 files that got deleted.
any_folder | Failed 'check_for_files.tasks.main' (edward-texas-sink-fish) - HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c181700>: Failed to establish a new connection: [Errno 111] Connection refused')) : Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
any_folder | sock = connection.create_connection(
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
any_folder | raise err
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
any_folder | sock.connect(sa)
any_folder | ConnectionRefusedError: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
any_folder | response = self._make_request(
any_folder | ^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
any_folder | raise new_e
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
any_folder | self._validate_conn(conn)
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
any_folder | conn.connect()
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
any_folder | self.sock = sock = self._new_conn()
any_folder | ^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
any_folder | raise NewConnectionError(
any_folder | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x79382c181700>: Failed to establish a new connection: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
any_folder | resp = conn.urlopen(
any_folder | ^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
any_folder | retries = retries.increment(
any_folder | ^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
any_folder | raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c181700>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | During handling of the above exception, another exception occurred:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/django_q/worker.py", line 103, in worker
any_folder | res = f(*task["args"], **task["kwargs"])
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/app/check_for_files/tasks.py", line 552, in main
any_folder | sort_files.sort_files()
any_folder | File "/app/check_for_files/sort_files.py", line 52, in sort_files
any_folder | response = requests.get("https://kichat.korn.local:3003/api/v1/workspaces", headers=self.headers_json, timeout=10, verify=False).json()
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 73, in get
any_folder | return request("get", url, params=params, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 59, in request
any_folder | return session.request(method=method, url=url, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
any_folder | resp = self.send(prep, **send_kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
any_folder | r = adapter.send(request, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
any_folder | raise ConnectionError(e, request=request)
any_folder | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c181700>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | Broker in an atomic transaction
any_folder | Enqueued [default] 33
any_folder | Pushed ('pluto-uranus-seven-august', 'efb207ebcd474b29b5f1e86e2b520276')
any_folder | Process-05e448e6db7347e7a773dfca422944a9 created task pluto-uranus-seven-august from schedule [check_files_schedule]
any_folder | queueing from default
any_folder | Process-8a1606e91c8c457a9b33abf2ea12c0d6 processing pluto-uranus-seven-august 'check_for_files.tasks.main' [check_files_schedule]
any_folder | Done Updating, task was successful. Found 0 files to add, 0 files that changed and 0 files that got deleted.
any_folder | Failed 'check_for_files.tasks.main' (pluto-uranus-seven-august) - HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c3cbf20>: Failed to establish a new connection: [Errno 111] Connection refused')) : Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
any_folder | sock = connection.create_connection(
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
any_folder | raise err
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
any_folder | sock.connect(sa)
any_folder | ConnectionRefusedError: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
any_folder | response = self._make_request(
any_folder | ^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
any_folder | raise new_e
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
any_folder | self._validate_conn(conn)
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
any_folder | conn.connect()
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
any_folder | self.sock = sock = self._new_conn()
any_folder | ^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
any_folder | raise NewConnectionError(
any_folder | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x79382c3cbf20>: Failed to establish a new connection: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
any_folder | resp = conn.urlopen(
any_folder | ^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
any_folder | retries = retries.increment(
any_folder | ^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
any_folder | raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c3cbf20>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | During handling of the above exception, another exception occurred:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/django_q/worker.py", line 103, in worker
any_folder | res = f(*task["args"], **task["kwargs"])
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/app/check_for_files/tasks.py", line 552, in main
any_folder | sort_files.sort_files()
any_folder | File "/app/check_for_files/sort_files.py", line 52, in sort_files
any_folder | response = requests.get("https://kichat.korn.local:3003/api/v1/workspaces", headers=self.headers_json, timeout=10, verify=False).json()
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 73, in get
any_folder | return request("get", url, params=params, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 59, in request
any_folder | return session.request(method=method, url=url, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
any_folder | resp = self.send(prep, **send_kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
any_folder | r = adapter.send(request, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 700, in send
any_folder | raise ConnectionError(e, request=request)
any_folder | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c3cbf20>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | queueing from default
any_folder | Process-d99181439b5b4cc4bc9466c306cda455 processing wisconsin-virginia-paris-pasta 'check_for_files.tasks.main' [check_files_schedule]
any_folder | Done Updating, task was successful. Found 0 files to add, 0 files that changed and 0 files that got deleted.
any_folder | Failed 'check_for_files.tasks.main' (wisconsin-virginia-paris-pasta) - HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c3252b0>: Failed to establish a new connection: [Errno 111] Connection refused')) : Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
any_folder | sock = connection.create_connection(
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
any_folder | raise err
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
any_folder | sock.connect(sa)
any_folder | ConnectionRefusedError: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
any_folder | response = self._make_request(
any_folder | ^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
any_folder | raise new_e
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
any_folder | self._validate_conn(conn)
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
any_folder | conn.connect()
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 704, in connect
any_folder | self.sock = sock = self._new_conn()
any_folder | ^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
any_folder | raise NewConnectionError(
any_folder | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x79382c3252b0>: Failed to establish a new connection: [Errno 111] Connection refused
any_folder |
any_folder | The above exception was the direct cause of the following exception:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
any_folder | resp = conn.urlopen(
any_folder | ^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
any_folder | retries = retries.increment(
any_folder | ^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
any_folder | raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='kichat.korn.local', port=3003): Max retries exceeded with url: /api/v1/workspaces (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x79382c3252b0>: Failed to establish a new connection: [Errno 111] Connection refused'))
any_folder |
any_folder | During handling of the above exception, another exception occurred:
any_folder |
any_folder | Traceback (most recent call last):
any_folder | File "/usr/local/lib/python3.12/site-packages/django_q/worker.py", line 103, in worker
any_folder | res = f(*task["args"], **task["kwargs"])
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/app/check_for_files/tasks.py", line 552, in main
any_folder | sort_files.sort_files()
any_folder | File "/app/check_for_files/sort_files.py", line 52, in sort_files
any_folder | response = requests.get("https://kichat.korn.local:3003/api/v1/workspaces", headers=self.headers_json, timeout=10, verify=False).json()
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 73, in get
any_folder | return request("get", url, params=params, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/api.py", line 59, in request
any_folder | return session.request(method=method, url=url, **kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
any_folder | File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
any_folder | resp = self.send(prep, **send_kwargs)
any_folder | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[MrMarans]

Hi, first off thanks for using my Project. I'm happy for my first issue.

This error can be ignored currently. This is from the file "sort_files.py" and it's trying to call a local URL from my workplace (it's the url we use for anythinLLM), where I tested that. It's not using the url you put in into the .env file currently and I missed to change that.

That feature is currently still under production and I simply forgot I was on main branch when creating that. That feature sorts embedded files into folders corresponding to their workspaces they are used in. It's missing the pinning feature for now (since you have to unpin and un-embed first before one can move files into new folders), that's why its not done yet, but still made it falsely into the main branch.

That error shouldn't stop you from using the other features tho. I'll fix that once I'm back from my current vacation, which will be in exactly one week.

[MrMarans]

If you are doing the sort_files=false this error shouldn't come again. But I can't test, cause vacation,don't have a laptop with me.

[BertrandakaBurt]

Hi Thanks you for your fast response,
I disabled sort_files and it did indeed remove these errors

But for some reason whenever new files appear it does not directly embed in llm. Would you have a suggestion for me.
Thank you

heres my docker compose file
services:
web:
build:
context: .
container_name: any_folder
image: any_folder
volumes:
- static_volume:/app/static
- storage_volume:/app/storage
- database_volume:/app/database
- /home/user/Desktop/data/not_live_logs:/app/AnythingLLM
env_file:
- .env
environment:
- DJANGO_SETTINGS_MODULE=main.settings
- PYTHONPATH=/app
- DATABASE_DIR=/app/database
- STORAGE_DIR=/app/storage
- HOST_FOLDER=/app/AnythingLLM
- PORT=8000
ports:
- "8000:8000"
extra_hosts:
- "host.docker.internal:host-gateway"
tty: true
stdin_open: true
restart: always
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"

volumes:
static_volume:
storage_volume:
database_volume:

heres my env file:
ANYTHING_LLM_API=secret

ANYTHING_LLM_URL=http://10.255.255.40:3001

USE_CRON=true
Enable scheduled checking
CHECK_FILES_CRON=*/1 * * * *
Check every minute

Image description (optional)
OLLAMA_URL=http://10.255.255.40:11434/api/generate
IMAGE_DESCRIPTION_ACTIVATE=true
IMAGE_DESCRIPTION_MODEL=gemma3:4b
IMAGE_DESCRIPTION_LANGUAGE=english

SORT_FILES=false
DELETE_UNUSED_FOLDERS=false

The files I want to embed are json but i tested it with simple text files too and it did not work
Logs:
Done Updating, task was successful. Found 0 files to add, 0 files that changed and 0 files that got deleted.

[MrMarans]

Could you try to go to http://localhost:8000/ and press the "Full Upload and Cleaning" button? Maybe the files are too many and too big so embedding takes too long for a 1 minute CRON schedule, which let's the progress cancel to start a new Task from CRON. That's just a guess tho

Using the button will do the same as each minute with your Cron scheduler, but cancelling isn't possible that way.
If this doesn't work then I currently can't further help for that regard. You'll have to wait for the end of my vacation for me to check further

[BertrandakaBurt]

Hi,
I tried going to localhost:8000 and clicked the full upload button same thing is happening:

Processed 'check_for_files.tasks.main' (cup-four-cup-ohio) any_folder | Uploading files any_folder | Done Updating, task was successful. Found 0 files to add, 0 files that changed and 0 files that got deleted.

I also went into the container and when I ls I can see all the files its supposed to emdeb:
/app/AnythingLLM$ ls
test.json
test.txt

But they do not emdeb either when trying to do it manually or automatically.

Thanks for your help,
Enjoy your vacation

[BertrandakaBurt]

Hi,
I had issues with the original project it wasnt detecting, going in the database, uploading and embedding

First, the new files werent getting dectected and the logs were returning :

Processed 'check_for_files.tasks.main' (cup-four-cup-ohio) any_folder | Uploading files any_folder | Done Updating, task was successful. Found 0 files to add, 0 files that changed and 0 files that got deleted.```

After that, The files would not go in the database.So the logs were returning :

Processed 'check_for_files.tasks.main' (cup-four-cup-ohio) any_folder | Uploading files any_folder | Done Updating, task was successful. Found 1 files to add, 0 files that changed and 0 files that got deleted.

but it would not go in the database

After that, the files were uploading to my anythingllm instance but not embedding,

And finally, the files were sucessfully automatically embedding into the anythingllm workspace

To find the fixes that worked for me i had an AI help me debug it and this is the final version of tasks.py it came up with if you want to look at it :

import os
import time
from datetime import datetime
import requests
from django.utils import timezone
from .models import FileInfo, TaskError, created_workspaces
from .describe_images import ImageDescriber
from pathlib import Path
import json
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
from .sort_files import SortFiles

class FileScanner:
    def __init__(self, verbose = False):
        self.verbose = verbose
        self.files_to_add = []
        self.files_that_changed = []
        self.files_got_deleted = []

        self.image_describer = ImageDescriber(verbose=self.verbose)

        self.host_folder = os.environ.get("HOST_FOLDER")
        if not self.host_folder:
            self.host_folder = "/app/AnythingLLM"
        else:
            # Convert Windows path to Unix-style path for Docker
            self.host_folder = self.host_folder.replace("\\", "/")
        if self.verbose: print(f"Host folder: {self.host_folder}")


        self.image_description_activate = os.environ.get("IMAGE_DESCRIPTION_ACTIVATE", "false").lower() == "true"
        if not self.image_description_activate:
            self.image_description_activate = False

    def reset(self):
        self.files_to_add = []
        self.files_that_changed = []
        self.files_got_deleted = []
        
        self.image_description_activate = os.environ.get("IMAGE_DESCRIPTION_ACTIVATE", "false").lower() == "true"
        if not self.image_description_activate:
            self.image_description_activate = False

    def scan_files(self):
        if self.verbose: print(f"image_description_activate: {self.image_description_activate}")
        self._check_for_files_still_exist()
        if self.verbose: print(f"checking subfolder")
        self._check_subfolder(directory = self.host_folder, main_folder = self.host_folder, first_run = True)
        if self.verbose: print(f"removing duplicates")
        self._remove_duplicates()
        if self.verbose: 
            print(f"Files to add: {self.files_to_add}")
            print(f"Files that changed: {self.files_that_changed}")
            print(f"Files got deleted: {self.files_got_deleted}")
            print(
                f"Found {len(self.files_to_add)} files to add, {len(self.files_that_changed)} files that changed and {len(self.files_got_deleted)} files that got deleted. Updating AnythingLLM now"
            )
            if (
                len(self.files_got_deleted) == 0
                and len(self.files_to_add) == 0
                and len(self.files_that_changed) == 0
            ):
                print("Since no changes have been detected, nothing happens")
                TaskError.objects.create(success=True, error="Early stop cause no changes")
                print("Nothing to update")
        
        return self.files_to_add, self.files_that_changed, self.files_got_deleted

    def create_image_descriptions(self):
        if self.verbose: print(f"image_description_activate: {self.image_description_activate}")
        self.image_description_activate= True
        self._check_subfolder(directory = self.host_folder, main_folder = self.host_folder, first_run = True)
        self._remove_duplicates()

    def _check_for_files_still_exist(self):
        # ----------------------------
        # We go through each file in the DB and check if the file is still there
        # ----------------------------
        try:
            all_files = FileInfo.objects.all()
            for file in all_files:
                if not os.path.exists(file.absolute_path):
                    # if file doesnt exist anymore
                    self.files_got_deleted.append(
                        [file.absolute_path, file.main_folder, file.filename]
                    )
                    file.delete()

        except Exception as e:
            print(
                f"Error when checking for files from DB if they still exist in directory: {str(e)}"
            )
    def _check_subfolder(self, directory, main_folder, first_run):
        try:
            directory_path = Path(directory)
            for file_path in directory_path.iterdir():
                file_name = file_path.name
                image_extensions = ('.jpg', '.jpeg', '.png', '.gif', '.bmp', '.tiff', '.webp')

                if self.verbose: print(f"CHECKING: File '{file_name}' in directory '{directory}', first_run={first_run}")
                if file_path.is_dir():
                    self._check_subfolder(str(file_path), file_name, False)
                    pass
                elif file_name.lower().endswith(image_extensions) and self.image_description_activate:
                    if self.verbose: print(f"  Is an image, checking for description...")
                    description_file = file_path.with_suffix('.image_description')
                    if description_file.exists():
                        if self.verbose: print(f"  Description file exists, skipping.")
                        continue
                    description_path, worked = self.image_describer.image_to_description(str(file_path))
                    if worked:
                        self._file_in_db_or_updated(file_name, str(description_path), main_folder)
                    else:
                        print(f"Error creating image description for {file_name}: {description_path}")
                        continue
                else:
                    if self.verbose: print(f"  Calling _file_in_db_or_updated for '{file_name}' with path '{str(file_path)}'")
                    self._file_in_db_or_updated(file_name, str(file_path), main_folder)

        except Exception as e:
            print(f"Error parsing {directory}: {str(e)} ")
    def _file_in_db_or_updated(self, file_name, file_path, main_folder):
        file_info_obj = self._is_File_In_DB(file_name, file_path)
        if file_info_obj is None:  # Wenn die Datei NICHT in der DB ist
            self.files_to_add.append([file_path, main_folder.lower().replace(" ", "-"), file_name])
        else:  # Wenn die Datei in der DB ist
            if self._file_Changed(file_info_obj, file_path):
                self.files_that_changed.append([file_path, main_folder.lower().replace(" ", "-"), file_name])
                if self.verbose: print(f"File {file_name} has changed.")

    def _is_File_In_DB(self,file_name, file_path):
        if self.verbose: print(f"Checking if file {file_name} in is in the DB")
        try:
            # Normalisiere den Pfad mit pathlib
            normalized_path = Path(file_path).resolve()
            file_info_obj = FileInfo.objects.filter(
                filename=file_name,
                absolute_path=str(normalized_path)
            ).first()

            if file_info_obj is not None:  # Wenn ein Objekt gefunden wurde
                if self.verbose: print(f"File {file_name} is in the DB")
                return file_info_obj  # Gib das FileInfo Objekt zurück
            else:  # Wenn kein Objekt gefunden wurde (None)
                if self.verbose: print(f"File {file_name} is not in the DB")
                return None
        except Exception as e:
            print(f"Error checking file in DB: {str(e)}")
            return None

    def _deduplicate_list(self, input_list):
        seen_files = set()
        output_list = []
        for item in input_list:
            # Assuming item is a list e.g., [file_path, main_folder, file_name]
            # Convert to tuple to make it hashable for set storage
            item_tuple = tuple(item)
            if item_tuple not in seen_files:
                output_list.append(item) # Append the original item (list)
                seen_files.add(item_tuple)
        return output_list

    def _remove_duplicates(self):
        self.files_to_add = self._deduplicate_list(self.files_to_add)
        self.files_got_deleted = self._deduplicate_list(self.files_got_deleted)
        self.files_that_changed = self._deduplicate_list(self.files_that_changed)


    def _file_Changed(self, file_in_db, file_path):
        try:
            creation_time = os.path.getctime(file_path)
            modification_time = os.path.getmtime(file_path)

            found_file_size = os.path.getsize(file_path)
            # Ensure datetime objects are timezone-aware for comparison with Django model fields
            found_created_at = timezone.make_aware(
                 datetime.fromtimestamp(creation_time)
            ) if timezone.is_naive(datetime.fromtimestamp(creation_time)) else datetime.fromtimestamp(creation_time)
           
            found_modified_at = timezone.make_aware(
                datetime.fromtimestamp(modification_time)
            ) if timezone.is_naive(datetime.fromtimestamp(modification_time)) else datetime.fromtimestamp(modification_time)
            
            # Debugging output
            if self.verbose:
                print(f"Comparing file: {file_in_db.filename}")
                print(f"  DB Size: {file_in_db.file_size}, Found Size: {found_file_size}")
                print(f"  DB Created: {file_in_db.created_at}, Found Created: {found_created_at}")
                print(f"  DB Modified: {file_in_db.modified_at}, Found Modified: {found_modified_at}")

            return (
                file_in_db.file_size != found_file_size
                or file_in_db.created_at < found_created_at # Timestamps in DB should be older if file is newer
                or file_in_db.modified_at < found_modified_at # Timestamps in DB should be older if file is newer
            )
        except Exception as e:
            print(f"Error checking if file changed for {file_path}: {str(e)}")
            return False


class AnythingLLM_API_Client:
    def __init__(self, api_version = "v1", verbose = False):
        
        self.verbose = verbose

        # Get AnythingLLM API key and URL from environment variables
        self.AnythingLLM_api = os.environ.get("ANYTHING_LLM_API")
        if not self.AnythingLLM_api:
            raise ValueError("ANYTHING_LLM_API environment variable is not set")

        self.main_url = os.environ.get("ANYTHING_LLM_URL")
        if not self.main_url:
            raise ValueError("ANYTHING_LLM_URL environment variable is not set")

        self.headers_json = {  #Header for most request types
            "accept": "application/json",
            "Authorization": f"Bearer {self.AnythingLLM_api}",
            "Content-Type": "application/json",
        }
        self.headers_files = {  #Header for file upload requests
                "accept": "application/json",
                "Authorization": f"Bearer {self.AnythingLLM_api}",
            }
        
        self.api_version = api_version
        self.update_embeddings = []
        self._configure_api_version()

    def reset(self):
        self.update_embeddings = []

    def _configure_api_version(self):
        if self.api_version == "v1":
            # different urls parts for the API
            self.get_workspaces_url = "/api/v1/workspaces"
            self.get_documents_url = "/api/v1/documents"
            self.delete_documents_url = "/api/v1/system/remove-documents"
            self.post_document_add_url = "/api/v1/document/upload/" 
            self.create_folder_url = "/api/v1/document/create-folder"
            self.move_to_folder_url = "/api/v1/document/move-files"
            self.get_workspace_url = "/api/v1/workspace/" # Used for GET, POST (new), DELETE specific workspace
            self.update_embeddings_url = "/update-embeddings" # Appended to specific workspace URL
            self.new_workspace_url = "/api/v1/workspace/new" # This is the correct one for creating new workspaces
            self.delete_folder_url = "/api/v1/document/remove-folder"


    def update(self, files_to_add, files_that_changed, files_got_deleted ):
        try:
            self.add_files_to_anythingLLM(files_to_add)
            self.delete_files_from_anythingLLM(files_got_deleted)
            self.update_files_in_anythingLLM(files_that_changed)
            self.update_workspace_embeddings(self.update_embeddings)
            self.delete_unused_workspaces()
            self.delete_unused_folders() # As per original main()

            if self.verbose:
                print(f"Done Updating, task was successful. {len(FileInfo.objects.all())} documents in DB")

        
            TaskError.objects.create(success=True, error=None)
            print(f"Done Updating, task was successful. Found {len(files_to_add)} files to add, {len(files_that_changed)} files that changed and {len(files_got_deleted)} files that got deleted.")
        except Exception as e:
            print(f"Error in task execution: {e}")
            TaskError.objects.create(success=False, error=str(e))

    def add_files_to_anythingLLM(self, files_to_add):
        try:
            for file_info_list in files_to_add:
                file_path = file_info_list[0]
                # FIXED: Use a proper workspace name instead of a file path
                # Extract a simple name from the folder path
                folder_path = file_info_list[1]
                folder_name_slug = self._get_proper_workspace_name(folder_path)
                original_file_name = os.path.basename(file_path)

                if not os.path.exists(file_path):
                    if self.verbose: print(f"Skipping file {file_path} as it no longer exists")
                    continue

                with open(file_path, "rb") as f_binary:
                    files_payload = {
                        "file": (original_file_name, f_binary, "application/octet-stream")
                    }
                    
                    upload_url = self.main_url + self.post_document_add_url
                    
                    if self.verbose:
                        print(f"Uploading file {original_file_name} from {file_path} to AnythingLLM general upload endpoint: {upload_url}")
                        print(f"  Request Headers: {self.headers_files}")

                    response = requests.post(
                        upload_url,
                        headers=self.headers_files,
                        files=files_payload,
                        timeout=60, 
                        verify=False
                    )

                    if self.verbose:
                        print(f"Upload Response Status Code for {original_file_name}: {response.status_code}")
                        print(f"Upload Response Headers: {response.headers}")
                        try:
                            print(f"Upload Raw Response Text: {json.dumps(response.json(), indent=2)}")
                        except json.JSONDecodeError:
                            print(f"Upload Raw Response Text (non-JSON): {response.text}")

                    if response.status_code == 200:
                        try:
                            response_data = response.json()
                            if "documents" in response_data and isinstance(response_data["documents"], list) and response_data["documents"]:
                                document_meta = response_data["documents"][0]
                                location_of_uploaded_file = document_meta.get("location")

                                if location_of_uploaded_file:
                                    self._saveFile(file_path, folder_name_slug)
                                    # FIXED: Use the proper workspace name for embedding
                                    self.update_embeddings.append([folder_name_slug, location_of_uploaded_file])
                                    if self.verbose:
                                        print(f"Successfully uploaded {original_file_name} as '{location_of_uploaded_file}'. Queued for embedding in workspace '{folder_name_slug}'.")
                                else:
                                    print(f"Error: 'location' field missing in upload response for {original_file_name}.")
                                    print(f"  Response data: {json.dumps(response_data, indent=2)}")
                                    continue
                            else:
                                print(f"Error: 'documents' array in response for {original_file_name} is missing, not a list, or empty.")
                                print(f"  Response data: {json.dumps(response_data, indent=2)}")
                                continue
                        except json.JSONDecodeError:
                            print(f"Error decoding JSON from upload response for {original_file_name} (Status: {response.status_code}): {response.text}")
                            continue
                        except (KeyError, IndexError, TypeError) as e:
                            print(f"Error parsing upload response for {original_file_name}: {str(e)}")
                            print(f"  Response data: {json.dumps(response_data, indent=2) if 'response_data' in locals() and response_data else response.text}")

                            continue
                    else:
                        print(f"Error uploading file {original_file_name} from {file_path}:")
                        print(f"  Status Code: {response.status_code}")
                        print(f"  Reason: {response.reason}")
                        print(f"  Text: {response.text}")
                        continue
            if self.verbose: print(f"Completed add_files_to_anythingLLM. Files queued for embedding: {self.update_embeddings}")

        except Exception as e:
            print(f"General error in add_files_to_anythingLLM: {str(e)}")

    # ADDED: Helper method to convert file paths to proper workspace names
    def _get_proper_workspace_name(self, path_or_name):
        """Convert a file path or complex name to a proper workspace name"""
        # Remove path separators and use the last part of the path
        if '/' in path_or_name:
            # Extract the last part of the path
            name_part = path_or_name.split('/')[-1]
        else:
            name_part = path_or_name
            
        # If the name is empty (e.g., from a root path), use a default
        if not name_part or name_part.strip() == '':
            name_part = "documents"
            
        # Clean up the name - remove special characters and spaces
        clean_name = ''.join(c for c in name_part if c.isalnum() or c in ['-', '_']).lower()
        
        # If cleaning removed everything, use a default
        if not clean_name:
            clean_name = "documents"
            
        return clean_name

    def delete_files_from_anythingLLM(self, array_to_delete):
        if not array_to_delete:
            if self.verbose: print("No files marked for deletion from AnythingLLM.")
            return

        paths_to_delete_api = []
        try:
            if self.verbose: print(f"Fetching documents from AnythingLLM to identify for deletion...")
            response_docs = requests.get(
                self.main_url + self.get_documents_url, headers=self.headers_json, timeout=20, verify=False
            )
            response_docs.raise_for_status()
            all_docs_response = response_docs.json()
            
            if self.verbose: print(f"GET {self.get_documents_url} response: {json.dumps(all_docs_response, indent=2)}")

            folders_data = []
            if "localFiles" in all_docs_response and "items" in all_docs_response["localFiles"]:
                folders_data = all_docs_response["localFiles"]["items"]
            elif "items" in all_docs_response and isinstance(all_docs_response["items"], list):
                folders_data = all_docs_response["items"]
            else:
                if self.verbose: print(f"Warning: Could not find expected folder structure in GET {self.get_documents_url} response.")

            # FIXED: Use proper workspace names for deletion lookup
            delete_lookup = {(item[2], self._get_proper_workspace_name(item[1])) for item in array_to_delete}

            for folder_api_data in folders_data:
                folder_slug_from_api = folder_api_data.get("name")
                for doc_api_data in folder_api_data.get("items", []):
                    doc_original_title = doc_api_data.get("title")
                    doc_internal_name = doc_api_data.get("name")

                    if (doc_original_title, folder_slug_from_api) in delete_lookup:
                        path_for_api_delete = f"{folder_slug_from_api}/{doc_internal_name}"
                        paths_to_delete_api.append(path_for_api_delete)
                        if self.verbose: print(f"Marked for API deletion: '{path_for_api_delete}' (Original: {doc_original_title} in {folder_slug_from_api})")
            
            if paths_to_delete_api:
                if self.verbose: print(f"Attempting to delete from AnythingLLM: {paths_to_delete_api}")
                delete_response = requests.delete(
                    self.main_url + self.delete_documents_url,
                    headers=self.headers_json,
                    json={"names": paths_to_delete_api},
                    timeout=30,
                    verify=False
                )
                delete_response.raise_for_status()
                if self.verbose: print(f"Deletion from AnythingLLM successful. Response Status: {delete_response.status_code}")
            else:
                if self.verbose: print("No matching files found currently in AnythingLLM to delete based on the provided list.")

            # Delete from local database after successful API deletion (or if not found in API, assume it's gone)
            for file_to_remove_db in array_to_delete:
                original_filename_db = file_to_remove_db[2]
                absolute_path_db = file_to_remove_db[0]
                FileInfo.objects.filter(filename=original_filename_db, absolute_path=absolute_path_db).delete()
                if self.verbose: print(f"Deleted {original_filename_db} (path: {absolute_path_db}) from local DB.")

        except requests.exceptions.RequestException as e:
            print(f"Error communicating with AnythingLLM during deletion: {str(e)}")
        except Exception as e:
            print(f"Error in delete_files_from_anythingLLM: {str(e)}")


    def update_files_in_anythingLLM(self, files_that_changed):
        if not files_that_changed:
            if self.verbose: print("No files marked for update.")
            return
        try:
            if self.verbose: print(f"Updating {len(files_that_changed)} changed files...")
            # For changed files, the strategy is:
            # 1. Delete their old entries from the local DB.
            # 2. Delete the old versions from AnythingLLM API.
            # 3. Add the new versions to AnythingLLM API (which also saves to local DB and queues for embedding).

            # 1. Remove old entries from local DB first to avoid conflicts if _saveFile is called before API deletion.
            for file_info_list_update in files_that_changed:
                filename_update = file_info_list_update[2] 
                filepath_update = file_info_list_update[0]
                FileInfo.objects.filter(filename=filename_update, absolute_path=filepath_update).delete()
                if self.verbose: print(f"Removed old DB entry for changed file: {filename_update} at {filepath_update}")
            
            # 2. Delete from AnythingLLM API
            self.delete_files_from_anythingLLM(files_that_changed) 
            
            # 3. Add the new versions (this will also call _saveFile for new DB entry and queue for embedding)
            self.add_files_to_anythingLLM(files_that_changed)
            if self.verbose: print(f"Completed updating files. Re-added and queued {len(files_that_changed)} files for embedding.")
        except Exception as e:
            print(f"Error updating files: {str(e)}")

    def update_workspace_embeddings(self, list_of_new_embeddings):
        # list_of_new_embeddings contains items like: [workspace_slug, full_document_location_from_api]
        if not list_of_new_embeddings:
            if self.verbose: print("No new embeddings to update.")
            return
        try:
            # Group documents by workspace slug
            workspaces_to_process = {} 
            for embedding_info in list_of_new_embeddings:
                workspace_slug = embedding_info[0]
                document_location = embedding_info[1]

                if workspace_slug not in workspaces_to_process:
                    workspaces_to_process[workspace_slug] = []
                workspaces_to_process[workspace_slug].append(document_location)
            
            if self.verbose: print(f"Workspaces and documents to update embeddings for: {json.dumps(workspaces_to_process, indent=2)}")

            for ws_slug, doc_locations_list in workspaces_to_process.items():
                # FIXED: Ensure workspace slug is properly formatted and URL encoded
                formatted_ws_slug_for_url = ws_slug.lower().replace(" ", "-")
                
                # FIXED: Properly handle workspace checking with error handling for JSON parsing
                workspace_check_url = f"{self.main_url}{self.get_workspace_url}{formatted_ws_slug_for_url}"
                
                try:
                    if self.verbose: print(f"Checking/ensuring workspace exists: {ws_slug} (URL slug: {formatted_ws_slug_for_url}) at {workspace_check_url}")
                    response_ws_check = requests.get(
                        url=workspace_check_url,
                        headers=self.headers_json,
                        timeout=10,
                        verify=False
                    )
                    
                    # FIXED: Better error handling for JSON parsing
                    ws_data = None
                    if response_ws_check.status_code == 200:
                        try:
                            response_json = response_ws_check.json()
                            ws_data = response_json.get("workspace")
                        except json.JSONDecodeError as json_err:
                            print(f"Error parsing workspace check response: {json_err}")
                            print(f"Raw response: {response_ws_check.text[:200]}...")  # Print first 200 chars
                            # Continue with workspace creation since we couldn't verify it exists
                    
                    # A 404 means workspace doesn't exist. Other errors are more problematic.
                    if response_ws_check.status_code == 404 or not ws_data:
                        if self.verbose: print(f"Workspace '{ws_slug}' not found or invalid response, attempting to create.")
                        self._create_workspace_if_not_exists(ws_slug)
                    else:
                        if self.verbose: print(f"Workspace '{ws_slug}' exists.")

                except requests.exceptions.RequestException as e_ws:
                    print(f"Error checking workspace {ws_slug}: {e_ws}")
                    if hasattr(e_ws, 'response') and e_ws.response is not None:
                        print(f"  Response text: {e_ws.response.text}")
                    # Try to create the workspace anyway
                    self._create_workspace_if_not_exists(ws_slug)

                # Now, update embeddings for this workspace
                json_payload_embedding = {
                    "adds": doc_locations_list,
                    "deletes": []
                }
                embedding_update_url = f"{self.main_url}{self.get_workspace_url}{formatted_ws_slug_for_url}{self.update_embeddings_url}"
                
                if self.verbose:
                    print(f"Updating embeddings for workspace: {ws_slug} (URL slug: {formatted_ws_slug_for_url})")
                    print(f"  Target URL: {embedding_update_url}")
                    print(f"  Request Payload: {json.dumps(json_payload_embedding, indent=2)}")
                    print(f"  Request Headers: {self.headers_json}")

                try:
                    response_embedding = requests.post(
                        url=embedding_update_url,
                        headers=self.headers_json,
                        json=json_payload_embedding,
                        timeout=60, # Embedding can take time
                        verify=False
                    )
                    if self.verbose:
                        print(f"Embedding update response for workspace {ws_slug} - Status: {response_embedding.status_code}")
                        try:
                            print(f"  Response Body: {json.dumps(response_embedding.json(), indent=2)}")
                        except json.JSONDecodeError:
                            print(f"  Response Body (non-JSON): {response_embedding.text}")
                    
                    response_embedding.raise_for_status() 
                    # Check for JSONDecodeError on success if API is expected to return JSON
                    if response_embedding.text: # If there's a body, try to parse
                        try:
                            response_embedding.json()
                        except json.JSONDecodeError:
                            # Just log the error but don't fail if we can't parse JSON from a successful response
                            print(f"Warning: Could not parse JSON from successful embedding update response: {response_embedding.text[:200]}...")
                    
                    if self.verbose: print(f"Successfully triggered embedding update for workspace '{ws_slug}'.")

                except requests.exceptions.HTTPError as http_err_embed:
                    print(f"HTTP error updating embeddings for workspace {ws_slug}: {http_err_embed}")
                    print(f"  Failed Payload: {json.dumps(json_payload_embedding, indent=2)}")
                    if http_err_embed.response is not None:
                         print(f"  Response Text: {http_err_embed.response.text}")
                except json.JSONDecodeError as json_err_embed: # Catch if successful status but non-JSON response
                    print(f"Error decoding JSON response for embedding update for workspace {ws_slug} (Status: {response_embedding.status_code}): {json_err_embed}")
                    print(f"  Raw Response Text: {response_embedding.text}")
                except requests.exceptions.RequestException as e_embed:
                    print(f"Request error updating embeddings for workspace {ws_slug}: {e_embed}")
                except Exception as e_unexpected: 
                    print(f"Unexpected error during embedding update for workspace {ws_slug}: {e_unexpected}")

        except Exception as e:
            print(f"General error in update_workspace_embeddings: {str(e)}")

    def delete_unused_workspaces(self):
        try:
            all_created_workspaces_db = created_workspaces.objects.all()
            if not all_created_workspaces_db:
                if self.verbose: print("No workspaces in local DB to check for deletion.")
                return

            if self.verbose: print(f"Checking {len(all_created_workspaces_db)} workspaces from DB for potential deletion...")

            for ws_db_entry in all_created_workspaces_db:
                # ws_db_entry.name should be the workspace slug
                ws_slug_to_check = ws_db_entry.name.lower().replace(" ", "-") 
                workspace_api_url = f"{self.main_url}{self.get_workspace_url}{ws_slug_to_check}"
                
                if self.verbose: print(f"Checking workspace '{ws_slug_to_check}' for document count via API: {workspace_api_url}")
                try:
                    response_ws_get = requests.get(
                        url=workspace_api_url,
                        headers=self.headers_json,
                        timeout=10,
                        verify=False
                    )
                    
                    if response_ws_get.status_code == 200:
                        try:
                            workspace_api_data = response_ws_get.json().get("workspace")
                            if workspace_api_data and isinstance(workspace_api_data, dict) and not workspace_api_data.get("documents"):
                                if self.verbose: print(f"Workspace '{ws_slug_to_check}' has no documents. Attempting deletion from API.")
                                # Delete URL is the same as GET URL for a specific workspace
                                delete_response_ws = requests.delete(
                                    url=workspace_api_url,
                                    headers=self.headers_json,
                                    timeout=10,
                                    verify=False
                                )
                                delete_response_ws.raise_for_status()
                                ws_db_entry.delete() # Delete from local DB
                                if self.verbose: print(f"Successfully deleted empty workspace '{ws_slug_to_check}' from API and local DB.")
                            elif not workspace_api_data:
                                 if self.verbose: print(f"Workspace '{ws_slug_to_check}' API check returned 200 but no valid workspace data, removing from local DB.")
                                 ws_db_entry.delete()
                            else:
                                if self.verbose: print(f"Workspace '{ws_slug_to_check}' has {len(workspace_api_data.get('documents',[]))} documents, not deleting.")
                        except json.JSONDecodeError:
                            print(f"Error parsing JSON from workspace check response for '{ws_slug_to_check}': {response_ws_get.text[:200]}...")
                            # Don't delete from DB if we can't verify its status
                    elif response_ws_get.status_code == 404:
                        if self.verbose: print(f"Workspace '{ws_slug_to_check}' not found in API (404). Deleting from local DB.")
                        ws_db_entry.delete()
                    else: # Other HTTP errors
                        response_ws_get.raise_for_status()

                except requests.exceptions.RequestException as e_ws_del_check:
                    print(f"Request error checking/deleting workspace {ws_slug_to_check}: {str(e_ws_del_check)}")
                    if hasattr(e_ws_del_check, 'response') and e_ws_del_check.response is not None:
                        print(f"  Response text: {e_ws_del_check.response.text}")
                except Exception as e_ws_del_other:
                    print(f"Unexpected error with workspace {ws_slug_to_check} during deletion check: {str(e_ws_del_other)}")
        except Exception as e:
            print(f"Error during delete_unused_workspaces: {str(e)}")


    def delete_unused_folders(self):
        # This method assumes "folders" from get_documents_url are equivalent to workspaces
        # that might need cleanup if empty and not managed by created_workspaces DB.
        try:
            if self.verbose: print("Checking for unused remote folders (workspaces) via get_documents_url...")
            response_docs_folders = requests.get(self.main_url + self.get_documents_url, headers=self.headers_json, timeout=20, verify=False)
            response_docs_folders.raise_for_status()
            
            try:
                all_folders_data = response_docs_folders.json()
                folder_list_api = []
                if "localFiles" in all_folders_data and "items" in all_folders_data["localFiles"]:
                    folder_list_api = all_folders_data["localFiles"]["items"]
                elif "items" in all_folders_data and isinstance(all_folders_data["items"], list):
                    folder_list_api = all_folders_data["items"]
                else:
                    if self.verbose: print(f"Warning: Could not find expected folder list in GET {self.get_documents_url} response for unused folder deletion.")
            except json.JSONDecodeError:
                print(f"Error parsing JSON from get_documents response: {response_docs_folders.text[:200]}...")
                return  # Exit early if we can't parse the response

            for folder_data_api in folder_list_api:
                folder_api_name_slug = folder_data_api.get("name") # This should be the slug
                folder_api_items = folder_data_api.get("items",[])  # Documents within the folder

                if folder_api_name_slug == "custom-documents": 
                    if self.verbose: print(f"Skipping default folder: {folder_api_name_slug}")
                    continue

                if not folder_api_items: 
                    if self.verbose: print(f"Folder/Workspace '{folder_api_name_slug}' from API is empty. Attempting to delete if not in local 'created_workspaces' DB or if it should be cleaned.")
                    
                    # Check if this workspace is managed by our script (already handled by delete_unused_workspaces)
                    if created_workspaces.objects.filter(name=folder_api_name_slug).exists():
                        if self.verbose: print(f"  Folder/Workspace '{folder_api_name_slug}' is in local DB, delete_unused_workspaces should handle it.")
                        continue

                    # If not managed by our DB, and it's empty, we might want to delete it.
                    workspace_delete_url_folder = f"{self.main_url}{self.get_workspace_url}{folder_api_name_slug.lower().replace(' ', '-')}"
                    if self.verbose: print(f"  Attempting to delete as workspace: {workspace_delete_url_folder}")
                    try:
                        delete_response_folder = requests.delete(workspace_delete_url_folder, headers=self.headers_json, timeout=10, verify=False)
                        # A 404 here means it was already gone, which is fine.
                        if delete_response_folder.status_code == 200 or delete_response_folder.status_code == 204:
                             if self.verbose: print(f"  Successfully deleted empty folder/workspace from API: {folder_api_name_slug} (Status: {delete_response_folder.status_code})")
                        elif delete_response_folder.status_code == 404:
                             if self.verbose: print(f"  Folder/Workspace '{folder_api_name_slug}' was not found (404) during delete attempt, already gone.")
                        else:
                            delete_response_folder.raise_for_status() # Raise for other errors
                    except requests.exceptions.RequestException as e_folder_del:
                         print(f"  Error deleting folder/workspace {folder_api_name_slug} from API: {str(e_folder_del)}")
                         if hasattr(e_folder_del, 'response') and e_folder_del.response is not None:
                            print(f"    Response text: {e_folder_del.response.text}")
                else:
                    if self.verbose: print(f"Folder/Workspace '{folder_api_name_slug}' from API contains {len(folder_api_items)} items, not deleting.")
        except requests.exceptions.RequestException as e_get_folders:
            print(f"Error communicating with AnythingLLM for deleting unused folders (get_documents_url): {str(e_get_folders)}")
        except Exception as e_del_folders_other:
            print(f"Error in delete_unused_folders: {str(e_del_folders_other)}")


    def _create_workspace_if_not_exists(self, workspace_slug_to_create):
        # FIXED: Ensure workspace name is a simple name, not a file path
        # Clean up the workspace name to ensure it's valid
        workspace_name = self._get_proper_workspace_name(workspace_slug_to_create)
        
        # Check if already in DB to avoid duplicate creation attempts if called multiple times
        if created_workspaces.objects.filter(name=workspace_name).exists():
            if self.verbose: print(f"Workspace '{workspace_name}' creation already recorded in DB or was previously attempted.")

        try:
            if self.verbose: print(
                f"Attempting to create workspace with slug/name '{workspace_name}'."
            )
            # For /api/v1/workspace/new, the "name" in payload becomes the display name AND the slug (after formatting).
            new_workspace_json = {
                "name": workspace_name, # API will slugify this if needed
                "similarityThreshold": 0.25,
                "openAiTemp": 0.7,
                "openAiHistory": 20,
                "openAiPrompt": "Given the following conversation, relevant context, and a follow up question, reply with an answer to the current question the user is asking. Return only your response to the question given the above information following the users instructions as needed.",
                "queryRefusalResponse": "There is no relevant information in this workspace to answer your query.",
                "chatMode": "chat",
                "topN": 4,
            }

            if self.verbose: print(f"Creating new workspace with payload: {json.dumps(new_workspace_json, indent=2)}")
            
            response_create_ws = requests.post(
                url=self.main_url + self.new_workspace_url,
                headers=self.headers_json,
                json=new_workspace_json,
                timeout=20, 
                verify=False
            )
            
            if self.verbose:
                print(f"Create workspace response for '{workspace_name}' - Status: {response_create_ws.status_code}")
                try:
                    print(f"  Response Body: {json.dumps(response_create_ws.json(), indent=2)}")
                except json.JSONDecodeError:
                    print(f"  Response Body (non-JSON): {response_create_ws.text}")

            if response_create_ws.status_code == 200:
                try:
                    created_ws_api_data = response_create_ws.json().get("workspace")
                    actual_slug_from_api = workspace_name # Default to what we sent
                    if created_ws_api_data and created_ws_api_data.get("slug"):
                         actual_slug_from_api = created_ws_api_data.get("slug")
                         if self.verbose: print(f"Workspace '{workspace_name}' created successfully. API returned slug: '{actual_slug_from_api}'.")
                    else:
                         print(f"Warning: Workspace '{workspace_name}' creation response did not explicitly return a slug. Assuming slug is '{workspace_name}'.")
                    
                    created_workspaces.objects.get_or_create(name=actual_slug_from_api) # Save the slug used/returned by API
                except json.JSONDecodeError:
                    print(f"Error parsing JSON from workspace creation response: {response_create_ws.text[:200]}...")
                    # Still record in DB that we attempted to create it
                    created_workspaces.objects.get_or_create(name=workspace_name)
            
            # Handle case where workspace might already exist (e.g., API returns 400 or other specific code)
            elif response_create_ws.status_code == 400: # Common for "already exists"
                error_text_create = response_create_ws.text.lower()
                if "already exists" in error_text_create or "slug is already taken" in error_text_create or "workspace_name_already_exists" in error_text_create:
                    if self.verbose: print(f"Workspace with slug '{workspace_name}' likely already exists (API reported error). Recording in DB if not present.")
                    created_workspaces.objects.get_or_create(name=workspace_name)
                else: # Other 400 error
                    response_create_ws.raise_for_status() # Raise for other bad requests
            else: # Other non-200, non-400 errors
                response_create_ws.raise_for_status()


        except requests.exceptions.RequestException as e_create_ws_req:
            print(f"Request error creating workspace '{workspace_name}': {str(e_create_ws_req)}")
            if hasattr(e_create_ws_req, 'response') and e_create_ws_req.response is not None:
                print(f"  Response text: {e_create_ws_req.response.text}")
        except Exception as e_create_ws_other: 
            print(f"Unexpected error creating workspace '{workspace_name}': {str(e_create_ws_other)}")


    def _saveFile(self, file_path, main_folder_slug): 
        try:
            abs_file_path = str(Path(file_path).resolve()) 
            original_filename = os.path.basename(file_path)

            FileInfo.objects.filter(
                filename=original_filename,
                absolute_path=abs_file_path,
            ).delete()

            creation_time = os.path.getctime(file_path)
            modification_time = os.path.getmtime(file_path)
            
            dt_creation = datetime.fromtimestamp(creation_time)
            dt_modification = datetime.fromtimestamp(modification_time)

            found_created_at = timezone.make_aware(dt_creation) if timezone.is_naive(dt_creation) else dt_creation
            found_modified_at = timezone.make_aware(dt_modification) if timezone.is_naive(dt_modification) else dt_modification
            
            new_file_info_instance = FileInfo.objects.create(
                filename=original_filename,
                absolute_path=abs_file_path,
                main_folder=main_folder_slug, 
                file_size=os.path.getsize(file_path),
                created_at=found_created_at,
                modified_at=found_modified_at,
            )
            if self.verbose: print(f"Saved/Updated file info in DB: {original_filename} in folder {main_folder_slug} (Path: {abs_file_path})")
            return new_file_info_instance
        except Exception as e:
            print(f"Error saving file source in DB for {file_path}: {str(e)}")
            return None



def main():
    verbose_env = os.environ.get("VERBOSE", "false").lower()
    verbose = verbose_env == "true" 

    if verbose: print("Script starting with verbose mode enabled.")

    file_scanner = FileScanner(verbose=verbose)
    files_to_add, files_that_changed, files_got_deleted  = file_scanner.scan_files()
    
    anythingLLM_api_client = AnythingLLM_API_Client(verbose=verbose)
    anythingLLM_api_client.update(
        files_to_add=files_to_add, 
        files_that_changed=files_that_changed, 
        files_got_deleted=files_got_deleted
    )
    
    file_scanner.reset()
    anythingLLM_api_client.reset()
    
    if os.environ.get("SORT_FILES", "false").lower() == "true":
        if verbose: print("SORT_FILES is true, attempting to sort files...")
        sort_files_instance = SortFiles(verbose=verbose) 
        sort_files_instance.sort_files()
        sort_files_instance.reset()
    
    if os.environ.get("DELETE_UNUSED_FOLDERS", "false").lower() == "true":
        if verbose: print("DELETE_UNUSED_FOLDERS is true, running delete_unused_folders again or specifically...")
        # If it's meant to be distinct from the update() call.
        # For now, assuming update() covers it as per my modification.
        # If it needs to be separate: anythingLLM_api_client.delete_unused_folders() 
        pass


    result_message = f"Files added: {len(files_to_add)}, Files changed: {len(files_that_changed)}, Files deleted: {len(files_got_deleted)}. File system synchronization complete."
    if verbose: print(result_message)
    return result_message

if __name__ == "__main__":
    print("Starting main execution...")
    try:
        # Ensure Django is setup if running this script directly outside manage.py
        # import django
        # os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'your_project.settings') # Replace your_project.settings
        # django.setup()
        # from .models import FileInfo, TaskError, created_workspaces # Re-import after setup if needed
        
        main_result = main()
        print(f"Main execution finished. Result: {main_result}")
    except Exception as e:
        print(f"Critical error in __main__: {e}")
        # Attempt to log to Django TaskError if models are accessible
        try:
            TaskError.objects.create(success=False, error=f"Critical error in __main__: {str(e)}")
        except Exception as db_error:
            print(f"Could not log critical error to DB: {db_error}")

[MrMarans]

Main issue of this thread just got fixed. (Feature still ignores pinning tho)

The other problem with embedding you had seems to be because your files are directly in the folder you set in the docker-compose file.
The code you have send changes a lot of things (which i have not fully checked yet). What I saw however is the deletion of a "if in main folder, skip" part.
This seems to be the reason your files didnt get detected, as they get skipped by design.
My Program checks for subfolders within the path you set in the docker-compose file.
https://github.com/MrMarans/AnythingLLM_File-Manager?tab=readme-ov-file#folder-setup-

Meaning, if you set a path in the docker-compose file to, for example, "C:/Home", then files only get detected when they are in a subfolder, like "C:/Home/MyLoggs" or "C:/Home/MyContracts".
Files directly in "C:/Home" get ignored. This is because this way it reflects workspaces in anythingLLM with your file system within that main path.

This seems to match with your

I also went into the container and when I ls I can see all the files its supposed to emdeb:
/app/AnythingLLM$ ls
test.json
test.txt

You should only have folders as output when doing that, not files.
It would be great if you could test that with my old code.

I will edit that in the readme to make it more clear. Also Ill probably do a video of features and setup soon.

[BertrandakaBurt]

Hi,
So i tested your code and it does now create new workspaces based on subfolders and upload files but i constantly get timeout errors when embedding or sometimes it shows embedded and then the file disappear from the embedded workspace. Embedding a small file using your app usually works.

But, my infrastructure works like this I pull logs from my servers then I use log rotate to put it in a folder watched by your app so that it embeds to the ai. Does it not embed because it is really ressource intensive? If so would there be a way so that it infinitly queues new files so that embedding never fails but just takes more time instead of throwing an error and skipping files.

Thanks you for your help