Robust registry

README.md

@@ -25,28 +25,13 @@ Solo Server is a lightweight platform that enables users to manage and monitor A
 - **Cross-Platform Compatibility:** Deploy AI models effortlessly on your hardware
 - **Configurable Framework:** Auto-detect hardware (CPU, GPU, RAM) and sets configs
 
-## Supported Models
-Solo Server supports **multiple model sources**, including **Ollama & Hugging Face**.
-
-| **Model Name**         | **Source**                                                |
-|------------------------|----------------------------------------------------------|
-| **DeepSeek R1**        | `ollama://deepseek-r1`                                   |
-| **IBM Granite 3.1**    | `ollama://granite3.1-dense`                              |
-| **Granite Code 8B**    | `hf://ibm-granite/granite-8b-code-base-4k-GGUF`          |
-| **Granite Code 20B**   | `hf://ibm-granite/granite-20b-code-base-8k-GGUF`         |
-| **Granite Code 34B**   | `hf://ibm-granite/granite-34b-code-base-8k-GGUF`         |
-| **Mistral 7B**         | `hf://TheBloke/Mistral-7B-Instruct-v0.2-GGUF`            |
-| **Mistral 7B v3**      | `hf://MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF`       |
-| **Hermes 2 Pro**       | `hf://NousResearch/Hermes-2-Pro-Mistral-7B-GGUF`        |
-| **Cerebrum 1.0 7B**    | `hf://froggeric/Cerebrum-1.0-7b-GGUF`                    |
-| **Dragon Mistral 7B**  | `hf://llmware/dragon-mistral-7b-v0`                      |
 
 ## Table of Contents
 
 - [Features](#-features)
-- [Supported Models](#supported-models)
 - [Installation](#installation)
 - [Commands](#commands)
+- [Supported Models](#supported-models)
 - [Configuration](#configuration)
 - [Project Inspiration](#project-inspiration)
 
@@ -56,19 +41,61 @@ Solo Server supports **multiple model sources**, including **Ollama & Hugging Fa
 
 - **🐋 Docker:** Required for containerization 
   - [Install Docker](https://docs.docker.com/get-docker/)
-  - Ensure Docker daemon is running
-
 ### **🔹 Install via PyPI**
 ```sh
+# Make sure you have Python <= 3.12
+python --version  # Should be below 3.13
+
+# Create a new virtual environment
+python -m venv .venv
+
+# Activate the virtual environment
+source .venv/bin/activate  # On Unix/MacOS
+# OR
+.venv\Scripts\activate # On Windows
+```
+```
 pip install solo-server
 ```
-
 ### **🔹 Install with `uv` (Recommended)**
 ```sh
-curl -sSL https://getsolo.tech/install.sh | bash
+# Install uv
+# On Windows (PowerShell)
+iwr https://astral.sh/uv/install.ps1 -useb | iex
+
+# On Unix/MacOS
+curl -LsSf https://astral.sh/uv/install.sh | sh
+
+# Create virtual environment
+uv venv
+
+# Activate the virtual environment
+source .venv/bin/activate  # On Unix/MacOS
+# OR
+.venv\Scripts\activate     # On Windows
+```
 ```
-Creates an isolated environment using `uv` for performance and stability.  
+uv pip install solo-server
+```
+Creates an isolated environment using `uv` for performance and stability.
+
+### **🔹 Install in Dev Mode**
+```sh
+# Clone the repository
+git clone https://github.com/GetSoloTech/solo-server.git
+
+# Navigate to the directory
+cd solo-server
+
+# Create and activate virtual environment
+python -m venv .venv
+source .venv/bin/activate  # Unix/MacOS
+# OR
+.venv\Scripts\activate     # Windows
 
+# Install in editable mode
+pip install -e .
+```
 Run the **interactive setup** to configure Solo Server:
 ```sh
 solo start
@@ -214,6 +241,23 @@ solo stop
 
 ---
 
+## Supported Models
+Solo Server supports **multiple model sources**, including **Ollama & Hugging Face**.
+
+| **Model Name**         | **Source**                                                |
+|------------------------|----------------------------------------------------------|
+| **DeepSeek R1**        | `ollama://deepseek-r1`                                   |
+| **IBM Granite 3.1**    | `ollama://granite3.1-dense`                              |
+| **Granite Code 8B**    | `hf://ibm-granite/granite-8b-code-base-4k-GGUF`          |
+| **Granite Code 20B**   | `hf://ibm-granite/granite-20b-code-base-8k-GGUF`         |
+| **Granite Code 34B**   | `hf://ibm-granite/granite-34b-code-base-8k-GGUF`         |
+| **Mistral 7B**         | `hf://TheBloke/Mistral-7B-Instruct-v0.2-GGUF`            |
+| **Mistral 7B v3**      | `hf://MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF`       |
+| **Hermes 2 Pro**       | `hf://NousResearch/Hermes-2-Pro-Mistral-7B-GGUF`        |
+| **Cerebrum 1.0 7B**    | `hf://froggeric/Cerebrum-1.0-7b-GGUF`                    |
+| **Dragon Mistral 7B**  | `hf://llmware/dragon-mistral-7b-v0`                      |
+
+
 ## **⚙️ Configuration (`solo.conf`)**
 After setup, all settings are stored in:
 ```sh

solo_server/start.py

@@ -2,97 +2,109 @@
 import subprocess
 import shutil
 import time
-from .utils.hardware import detect_hardware, display_hardware_info
+import platform
+from solo_server.utils.hardware import detect_hardware, display_hardware_info
+from solo_server.utils.nvidia import check_nvidia_toolkit, install_nvidia_toolkit_linux, install_nvidia_toolkit_windows
 
-def check_nvidia_toolkit() -> bool:
+def start_docker_engine(os_name):
     """
-    Checks if Docker can actually run a GPU container using the NVIDIA runtime.
+    Attempts to start the Docker engine based on the OS.
     """
+    typer.echo("Starting the Docker engine...")
     try:
-        test_cmd = [
-            "docker", "run", "--rm", "--gpus", "all",
-            "nvidia/cuda:11.0.3-base-ubuntu20.04", "nvidia-smi"
-        ]
-        subprocess.run(test_cmd, check=True, capture_output=True, text=True)
-        return True
-    except subprocess.CalledProcessError:
-        return False
-
+        if os_name == "Windows":
+            try:
+                subprocess.run(["sc", "start", "docker"], check=True, capture_output=True)
+            except subprocess.CalledProcessError:
+                typer.echo("Docker service is not registered. Trying to start Docker Desktop...", err=True)
 
-def install_nvidia_toolkit_linux():
-    """
-    Installs the NVIDIA Container Toolkit on Linux (Debian & Ubuntu).
-    """
-    typer.echo("Configuring the repository")
-    try:
-        subprocess.run("curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg", shell=True, check=True)
-        subprocess.run("curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list", shell=True, check=True)
-        subprocess.run("sudo apt-get update", shell=True, check=True)
+                # Run PowerShell command to get Docker path
+                result = subprocess.run(
+                    ["powershell", "-Command", "(Get-Command docker | Select-Object -ExpandProperty Source)"],
+                    capture_output=True,
+                    text=True
+                )
 
-        typer.echo("Installing Nvidia Container Toolkit")
-        subprocess.run("sudo apt-get install -y nvidia-container-toolkit", shell=True, check=True)
-        subprocess.run("sudo nvidia-ctk runtime configure --runtime=docker", shell=True, check=True)
-        subprocess.run("sudo systemctl restart docker", shell=True, check=True)
+                docker_path = result.stdout.strip()
+                if "Docker" in docker_path:
+                    # Find the second occurrence of 'Docker'
+                    parts = docker_path.split("\\")
+                    docker_index = [i for i, part in enumerate(parts) if part.lower() == "docker"]
 
-        typer.echo("NVIDIA Container Toolkit installed successfully on Linux.")
-    except subprocess.CalledProcessError as e:
-        typer.echo(f"Failed to install NVIDIA Container Toolkit on Linux. Error: {e}", err=True)
+                    if len(docker_index) >= 2:
+                        docker_desktop_path = "\\".join(parts[:docker_index[1] + 1]) + "\\Docker Desktop.exe"
 
+                        typer.echo(f"Starting Docker Desktop from: {docker_desktop_path}")
+                        subprocess.run(["powershell", "-Command", f"Start-Process '{docker_desktop_path}' -Verb RunAs"], check=True)
+                    else:
+                        typer.echo("❌ Could not determine Docker Desktop path.", err=True)
+                else:
+                    typer.echo("❌ Docker is not installed or incorrectly configured.", err=True)
 
-def install_nvidia_toolkit_windows():
-    """
-    Provide a structured step-by-step guide for Windows users to configure
-    their system for NVIDIA GPU support, including driver & CUDA installation.
-    """
-    # Step-by-step instructions
-    typer.secho("\n========================================", fg=typer.colors.CYAN)
-    typer.secho(" Windows NVIDIA GPU Setup ", fg=typer.colors.CYAN, bold=True)
-    typer.secho("========================================\n", fg=typer.colors.CYAN)
-
-    typer.echo("Follow these steps to enable NVIDIA GPU support on Windows:\n")
-
-    steps = [
-        ("Step 1: Install or Update NVIDIA Drivers", "https://www.nvidia.com/Download/index.aspx"),
-        ("Step 2: Install the NVIDIA CUDA Toolkit", "https://developer.nvidia.com/cuda-downloads")
-    ]
-    for step_num, (step_title, link) in enumerate(steps, start=1):
-        typer.secho(f"{step_title}", fg=typer.colors.BRIGHT_GREEN)
-        typer.echo(f"   Link: {link}\n")
-
-    typer.echo("Once you've completed the above steps:")
-    typer.echo(" - Ensure Docker Desktop is installed and running.")
-    typer.echo(" - Enable 'Use the WSL 2 based engine' in Docker Desktop settings.\n")
-
-    typer.secho("⚠️  Please restart Solo Server after installing the required tools.", fg=typer.colors.YELLOW)
-    raise typer.Exit(1)
+        elif os_name == "Linux":
+            try:
+                # First try systemctl for system Docker service
+                subprocess.run(["sudo", "systemctl", "start", "docker"], check=True, capture_output=True)
+            except subprocess.CalledProcessError:
+                try:
+                    # If systemctl fails, try starting Docker Desktop
+                    subprocess.run(["systemctl", "--user", "start", "docker-desktop"], check=True, capture_output=True)
+                except subprocess.CalledProcessError:
+                    typer.echo("❌ Failed to start Docker. Please start manually", err=True)
+
+        elif os_name == "Darwin":  # macOS
+            subprocess.run(["open", "/Applications/Docker.app"], check=True, capture_output=True)
+        time.sleep(5) # Wait for Docker to start
+    except subprocess.CalledProcessError:
+        typer.echo("❌ Failed to start Docker. Please start Docker with admin privileges manually.", err=True)
+        return False
+    return True
 
 def start():
-
     """Setup solo-server environment."""
-
     display_hardware_info(typer)
+    cpu_model, cpu_cores, memory_gb, gpu_vendor, gpu_model, gpu_memory, compute_backend, os = detect_hardware()
+    use_gpu = False
     typer.echo("\n🚀 Setting up Solo Server...")
+
+    # Initialize container_exists flag
+    container_exists = False
 
     if not shutil.which("docker"):
         typer.echo(
-            "❌ Docker is not installed. Please install Docker first.\n"
-            "Link: https://docs.docker.com/get-docker/\n",
+            "❌ Docker is not installed. Please install Docker first.\n",
             err=True
         )
+        typer.secho(
+            "Install Here: https://docs.docker.com/get-docker/",
+            fg=typer.colors.GREEN
+        )
         raise typer.Exit(code=1)
 
     try:
         # Check if Docker daemon is running
-        subprocess.run(["docker", "info"], check=True, capture_output=True)
-        cpu_model, cpu_cores, memory_gb, gpu_vendor, gpu_model, gpu_memory, compute_backend, os = detect_hardware()
-        use_gpu = False
+        try:
+            subprocess.run(["docker", "info"], check=True, capture_output=True)
+        except subprocess.CalledProcessError:
+            typer.echo("Docker daemon is not running. Attempting to start Docker...", err=True)
+            if not start_docker_engine(os):
+                raise typer.Exit(code=1)
+            # Re-check if Docker is running
+            try:
+                subprocess.run(["docker", "info"], check=True, capture_output=True)
+            except subprocess.CalledProcessError:
+                typer.echo("❌ Docker commands are not working as expected.", err=True)
+                typer.echo("Try running the terminal with admin privileges.", err=True)
+                raise typer.Exit(code=1)
 
+        # Check for NVIDIA GPU
         if gpu_vendor == "NVIDIA":
-            if check_nvidia_toolkit():
-                typer.echo("✅ NVIDIA Docker Toolkit is already installed.\n")
+            if check_nvidia_toolkit(os):
+                typer.echo("✅ NVIDIA Toolkit is already installed.\n")
                 use_gpu = True
             else:
-                if typer.confirm("NVIDIA GPU detected but Toolkit is not installed. Do you want to install it?", default=False):
+                typer.echo("NVIDIA GPU detected but drivers are not installed.")
+                if typer.confirm("Would you like to install NVIDIA drivers?", default=False):
                     if os == "Linux":
                         install_nvidia_toolkit_linux()
                     elif os == "Windows":
@@ -133,9 +145,14 @@ def start():
 
             # Start Ollama container
             docker_run_cmd = ["docker", "run", "-d", "--name", "solo", "-v", "ollama:/root/.ollama", "-p", "11434:11434"]
-            if use_gpu:
+            if gpu_vendor == "NVIDIA" and use_gpu:
                 docker_run_cmd += ["--gpus", "all"]
                 docker_run_cmd.append("ollama/ollama")
+            elif gpu_vendor == "AMD":
+                docker_run_cmd += ["--device", "/dev/kfd", "--device", "/dev/dri"]
+                docker_run_cmd.append("ollama/ollama:rocm")
+            else:
+                docker_run_cmd.append("ollama/ollama")
 
             typer.echo("🚀 Starting Solo Server...")
             subprocess.run(docker_run_cmd, check=True)
@@ -167,6 +184,7 @@ def start():
         # Cleanup on failure
         if container_exists:
             subprocess.run(["docker", "stop", "solo"], check=False)
+        raise typer.Exit(code=1)
     except Exception as e:
         typer.echo(f"❌ Unexpected error: {e}", err=True)
 

solo_server/utils/nvidia.py

@@ -0,0 +1,78 @@
+import subprocess
+import typer
+import sys
+import platform
+from typing import Optional
+
+def check_nvidia_toolkit(os_name) -> bool:
+    """
+    Checks if NVIDIA toolkit is properly installed based on the operating system.
+    """
+    if os_name == "Linux":
+        try:
+            result = subprocess.run("docker info | grep -i nvidia", 
+                                 shell=True, 
+                                 check=True, 
+                                 capture_output=True, 
+                                 text=True)
+            return bool(result.stdout.strip())
+        except subprocess.CalledProcessError:
+            return False
+    elif os_name == "Windows":
+        try:
+            subprocess.run("nvidia-smi", 
+                         check=True, 
+                         capture_output=True, 
+                         text=True)
+            return True
+        except subprocess.CalledProcessError:
+            return False
+    return False
+
+
+def install_nvidia_toolkit_linux():
+    """
+    Installs the NVIDIA Container Toolkit on Linux (Debian & Ubuntu).
+    """
+    typer.echo("Configuring the repository")
+    try:
+        subprocess.run("curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg", shell=True, check=True)
+        subprocess.run("curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list", shell=True, check=True)
+        subprocess.run("sudo apt-get update", shell=True, check=True)
+
+        typer.echo("Installing Nvidia Container Toolkit")
+        subprocess.run("sudo apt-get install -y nvidia-container-toolkit", shell=True, check=True)
+        subprocess.run("sudo nvidia-ctk runtime configure --runtime=docker", shell=True, check=True)
+        subprocess.run("sudo systemctl restart docker", shell=True, check=True)
+
+        typer.echo("NVIDIA Container Toolkit installed successfully on Linux.")
+    except subprocess.CalledProcessError as e:
+        typer.echo(f"Failed to install NVIDIA Container Toolkit on Linux. Error: {e}", err=True)
+
+
+def install_nvidia_toolkit_windows():
+    """
+    Provide a structured step-by-step guide for Windows users to configure
+    their system for NVIDIA GPU support, including driver & CUDA installation.
+    """
+    # Step-by-step instructions
+    typer.secho("\n========================================", fg=typer.colors.CYAN)
+    typer.secho(" Windows NVIDIA GPU Setup ", fg=typer.colors.CYAN, bold=True)
+    typer.secho("========================================\n", fg=typer.colors.CYAN)
+
+    typer.echo("Follow these steps to enable NVIDIA GPU support on Windows:\n")
+
+    steps = [
+        ("Step 1: Install or Update NVIDIA Drivers", "https://www.nvidia.com/Download/index.aspx"),
+        ("Step 2: Install the NVIDIA CUDA Toolkit", "https://developer.nvidia.com/cuda-downloads")
+    ]
+    for step_num, (step_title, link) in enumerate(steps, start=1):
+        typer.secho(f"{step_title}", fg=typer.colors.BRIGHT_GREEN)
+        typer.echo(f"   Link: {link}\n")
+
+    typer.echo("Once you've completed the above steps:")
+    typer.echo(" - Ensure Docker Desktop is installed and running.")
+    typer.echo(" - Enable 'Use the WSL 2 based engine' in Docker Desktop settings.\n")
+
+    typer.secho("⚠️  Please restart Solo Server after installing the required tools.", fg=typer.colors.YELLOW)
+    raise typer.Exit(1)