DevOps - Seamless local vs. Docker

[bordumb]

Of course. The error nodename nor servname provided, or not known occurs because your application, when run locally, is configured to find the database using its Docker service name (e.g., postgres), which your Mac doesn't recognize.

The most elegant way to solve this is to use Hydra, a configuration tool you already have in your project, to manage separate settings for your local and docker environments. This allows you to switch between them seamlessly.

---

The Strategy: Environment-Specific Configuration

We will create a dedicated conf directory to hold different configuration files. Hydra will then compose the final configuration by merging a main file with an environment-specific file, which you can select from the command line.

Step 1: Create the Configuration Structure

First, create the following directory and file structure inside your agent-sim/src/ directory:

agent-sim/src/
└── conf/
    ├── config.yaml
    └── env/
        ├── docker.yaml
        └── local.yaml

Step 2: Define Your Environments

These files will contain the parts of your configuration that change between environments—specifically, the hostnames for your services.

1. Create agent-sim/src/conf/env/local.yaml:
This file uses localhost to connect to services running in Docker from your Mac.

# conf/env/local.yaml
# Settings for running on the host machine (your Mac)
services:
  database_host: localhost
  mlflow_host: localhost

2. Create agent-sim/src/conf/env/docker.yaml:
This file uses Docker's internal service names for container-to-container communication.

# conf/env/docker.yaml
# Settings for running inside a Docker container
services:
  database_host: postgres
  mlflow_host: mlflow

Step 3: Update Your .env File

Your .env file should now only contain secrets, not hostnames. Hydra will combine these variables with the hostnames from the config files.

# .env

# Database Credentials
DB_USER=admin
DB_PASSWORD=password
DB_NAME=agent_sim_db

# The full URLs are now constructed by Hydra
DATABASE_URL=postgresql+asyncpg://${DB_USER}:${DB_PASSWORD}@${services.database_host}:5432/${DB_NAME}
MLFLOW_TRACKING_URI=http://${services.mlflow_host}:5000

Step 4: Create the Main config.yaml

This file tells Hydra about the default setup and where to find the rest of your original configuration.

Place your main base_config.yml content inside this new file and add a defaults section at the top.

# agent-sim/src/conf/config.yaml

# --- ADD THIS SECTION AT THE TOP ---
defaults:
  - env: local # Default to the 'local' environment
  - _self_
# ------------------------------------

# --- PASTE YOUR EXISTING base_config.yml CONTENT BELOW ---
agent:
  # ... all your agent settings
environment:
  # ... all your environment settings
# etc.

Step 5: Integrate Hydra into Your Application

Now, modify your application's entrypoint, agent-sim/src/agent_sim/main.py, to use the @hydra.main() decorator. This replaces the manual OmegaConf loading.

# src/agent_sim/main.py
import hydra
from omegaconf import DictConfig, OmegaConf

# ... (other imports)

@hydra.main(config_path="../../conf", config_name="config", version_base=None)
def main(cfg: DictConfig) -> None:
    """
    CLI entrypoint for running a single, local simulation using Hydra.
    """
    print("--- ARLA Local Simulation Runner (Hydra) ---")
    print(f"Hydra Config:\n{OmegaConf.to_yaml(cfg)}")
    print("----------------------------------")

    try:
        # Convert the Hydra config to a plain dictionary for the simulation
        config_dict = OmegaConf.to_container(cfg, resolve=True)

        run_id = str(uuid.uuid4())
        task_id = "local_task"
        experiment_id = "local_experiment"
        
        sim_module = importlib.import_module(f"{cfg.package}.run")
        sim_module.start_simulation(
            run_id=run_id,
            task_id=task_id,
            experiment_id=experiment_id,
            config_overrides=config_dict,
        )
        print("\n✅ Simulation completed successfully.")
    except Exception as e:
        print(f"\n❌ An error occurred during the simulation: {e}")
        traceback.print_exc()
        exit(1)

if __name__ == "__main__":
    main()

Step 6: Update pyproject.toml

Finally, update your script entrypoint in pyproject.toml to point to the new main function.

# In pyproject.toml
[tool.poetry.scripts]
arla = "agent_sim.main:main" # Changed from agent_sim.main:app

After making this change, run poetry install one last time to update the script.

Your New Elegant Workflow ✨

You can now run your simulation locally without any modifications, and it will work correctly.

To Run Locally (default):
Hydra will automatically load the env/local.yaml configuration.

poetry run arla --steps 50

To Run with Docker-Specific Config:
If you ever need to run locally but use the Docker network settings, you can easily override the environment on the command line.

poetry run arla env=docker --steps 50