[Idea2] Remote version in your todo list

[draftman9]

Solution: Using SSHFS to Mount Local Directories on a Remote Server for Transparent File Access with DaemonMode.jl

---

Objective

Enable a remote server running DaemonMode.jl to seamlessly access files from your local machine (e.g., paths like ../../0res/data.dat) without manual synchronization. This ensures scripts can run transparently with local file paths while leveraging remote computational resources.

---

Step-by-Step Implementation

1. Install SSHFS on the Remote Server

SSHFS allows mounting a local directory on the remote server via SSH.

# Ubuntu/Debian
sudo apt-get install sshfs

# CentOS/RHEL
sudo yum install fuse-sshfs

---

2. Mount the Local Directory on the Remote Server

Mount your local project directory to a directory on the remote server.

# On the remote server:
mkdir -p ~/remote_mount  # Create a mount point
sshfs user@local_ip:/path/to/local/project ~/remote_mount

• Replace user@local_ip with your local username and IP.
• Replace /path/to/local/project with the local directory containing your scripts and data.

Example:

sshfs alice@192.168.1.100:/home/alice/my_project ~/remote_mount

---

3. Start DaemonMode.jl on the Remote Server

Run the DaemonMode server in the mounted directory to preserve local file paths.

cd ~/remote_mount
julia --startup-file=no -e 'using DaemonMode; serve()'

---

4. Run Scripts via SSH Tunnel

Forward the DaemonMode port to your local machine and execute scripts as if they were running locally.

# Step 1: Create an SSH tunnel (maps remote port 12345 to local port 12345)
ssh -N -L 12345:localhost:12345 user@remote_server_ip &

# Step 2: Run the script using the local-mounted path
julia --startup-file=no -e 'using DaemonMode; runargs(connect("localhost", 12345))' ~/remote_mount/script.jl

---

5. Script Compatibility

Your scripts can use local relative paths (e.g., ../../0res/data.dat) because the remote server’s mounted directory mirrors your local structure.

Example script (script.jl):

using CSV, DataFrames
df = CSV.read("../../0res/data.dat", DataFrame)  # Works as if running locally!
println(first(df, 3))

---

Automation Script

Save this as run_remote.sh for one-click execution:

#!/bin/bash
# Usage: ./run_remote.sh <REMOTE_IP> <LOCAL_PROJECT_DIR> <SCRIPT_PATH>

REMOTE_IP="$1"
LOCAL_PROJECT="$2"
SCRIPT_PATH="$3"
REMOTE_MOUNT="~/remote_mount"

# 1. Mount the local directory on the remote server
ssh "$REMOTE_IP" "mkdir -p $REMOTE_MOUNT; sshfs $USER@$(hostname -I | awk '{print $1}'):$LOCAL_PROJECT $REMOTE_MOUNT"

# 2. Start DaemonMode.jl server (if not running)
ssh "$REMOTE_IP" "cd $REMOTE_MOUNT && julia --startup-file=no -e 'using DaemonMode; serve()' &"

# 3. Create SSH tunnel
ssh -N -L 12345:localhost:12345 "$REMOTE_IP" &
TUNNEL_PID=$!

# 4. Execute the script
julia --startup-file=no -e "using DaemonMode; runargs(connect(\"localhost\", 12345))" "$SCRIPT_PATH"

# 5. Cleanup
kill $TUNNEL_PID
ssh "$REMOTE_IP" "fusermount -u $REMOTE_MOUNT"

Run it with:

chmod +x run_remote.sh
./run_remote.sh user@remote_server_ip /path/to/local/project script.jl

---

Key Advantages

1. Transparent File Access:

   • Scripts use local relative paths (e.g., ../../0res/data.dat) without modification.
   • No manual file synchronization needed.

2. Cross-Platform Compatibility:

   • Works on Linux, macOS, and Windows (via WSL).

3. Performance:

   • Avoids repeated file transfers; files are accessed on-demand over SSH.

---

Notes

1. Permissions: Ensure the remote user has read/write access to the mounted directory.
2. Network Stability: Use autossh for resilient SSH tunnels in unstable networks:

   autossh -M 0 -L 12345:localhost:12345 user@remote_server_ip

3. Large Files: For frequently accessed large files, consider caching them remotely.