Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
200 changes: 200 additions & 0 deletions Migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
# **LLMUnity Migration Guide: Upgrading to the New Architecture**

This guide helps you migrate to LLMUnity v3 from v2.

The LLM backend, [LlamaLib](https://github.com/undreamai/LlamaLib), has been completely rewritten and most of the LLM functionality in Unity has now been ported to the backend.<br>
At the same time, LlamaLib has been implemented as a clean, high-level library that can been used on its own by C++ and C# developers.<br>
Special care has been taken to provide better support for future llama.cpp versions, streamlining more the upgrade process.

## **Breaking Changes - Graphical Interface**
In terms of graphical interface in Unity, there have been almost no changes:
- The LLMCharacter class has been renamed to LLMAgent
- The chat templates have been removed from the LLM model management (LLM GameObjects)
- "Use extras" has been renamed to "Use cuBLAS" (LLM GameObjects)
- Grammars can be directly edited (LLMAgent GameObjects)

## **Breaking Changes - Scripting**

### **1\. LLM Server (LLM.cs)**

#### **API Changes**

**Setting Model:**

``` c#
// OLD
llm.SetModel("path/to/model.gguf");

// NEW
llm.model = "path/to/model.gguf";
```

**Setting Chat Template:**

Setting the chat template is not needed anymore, it is handled by LlamaLib and loads the LLM-supported template automatically.

``` c#
// OLD
llm.SetTemplate("chatml");

// NEW
// Template is auto-detected from model_
```

### **2\. Chat Agent (LLMCharacter → LLMAgent)**

LLMCharacter has been moved to a new class LLMAgent.<br>
**Note:** LLMCharacter still exists as a deprecated wrapper for backward compatibility, but will be removed in future versions.

#### **Property changes**

| **Old Property** | **New Property** | **Migration** |
| --- | --- | --- |
| playerName | userRole | Direct rename |
| AIName | assistantRole | Direct rename |
| prompt | systemPrompt | Direct rename |
| chat | chat | Now a property with getter/setter |
| saveCache | Removed | Cache management removed |


#### **API Changes**

**Clear History:**
``` c#
// OLD
character.ClearChat();

// NEW
await agent.ClearHistory();
```

**Setting / Changing the System Prompt:**
- Without clearing the history

``` c#
// OLD
character.SetPrompt("You are a helpful assistant", clearChat: false);

// NEW
agent.systemPrompt = "You are a helpful assistant";
```

- Clearing the history
``` c#
// OLD
character.SetPrompt("You are a helpful assistant", clearChat: true);

// NEW
agent.systemPrompt = "You are a helpful assistant";
await agent.ClearHistory();
```

**Managing History:**

``` c#
character.AddPlayerMessage("Hello");
character.AddAIMessage("Hi there");

// NEW
await agent.AddUserMessage("Hello");
await agent.AddAssistantMessage("Hi there");
```

**Saving/Loading:**

``` c#
// OLD
await character.Save("chat_history");
await character.Load("chat_history");

// NEW
await agent.SaveHistory();
await agent.LoadHistory();
```

The history path is directly the `character.save` property.

**Grammar:**

Setting the grammar directly:
``` c#
// OLD
llmClient.grammarString = "your grammar here";

// NEW
llmClient.grammar = "your grammar here";
```

Loading a grammar file:
``` c#
// OLD
await character.SetGrammar("path/to/grammar.gbnf");
await character.SetJSONGrammar("path/to/schema.json");

// NEW
llmClient.LoadGrammar("path/to/grammar.gbnf");
llmClient.LoadGrammar("path/to/schema.json");
```

### **Reasoning Mode (New Feature)**

``` c#
// NEW - Enable "thinking" mode of the LLM
llm.reasoning = true;
// Or
llm.SetReasoning(true);
```

## **Migration Steps**

### **Step 1: Update Class Inheritance**

``` c#
// Change LLMCharacter to LLMAgent
public class MyNPC : LLMAgent // was LLMCharacter
{
// Your code_
}
```

### **Step 2: Update Property Names**

``` c#
// In LLMAgent
agent.userRole = "Player"; // was character.playerName
agent.assistantRole = "AI"; // was character.AIName
agent.systemPrompt = "..."; // was character.prompt
```

### **Step 3: Update Method Calls**

``` c#
// History management
await agent.ClearHistory(); // was character.ClearChat()
await agent.AddUserMessage("Hi"); // was character.AddPlayerMessage("Hi")
await agent.AddAssistantMessage("Hello"); // was character.AddAIMessage("Hello")
```

### **Step 4: Update Grammar**

``` c#
agent.grammar = "your grammar here"; // was character.grammarString = "your grammar here"
// or
agent.LoadGrammar("grammar.gbnf"); // was await character.SetGrammar("path/to/grammar.gbnf"); or await character.SetJSONGrammar("path/to/schema.json");
```

### **Step 5: Update Save/Load**

``` c#
agent.save = "history"; // Set once
await agent.SaveHistory(); // was await character.Save("history");
await agent.LoadHistory(); // was await character.Load("history");
```

### **Step 6: Remove Chat Template**

``` c#
// OLD
// Remove:
// llm.SetTemplate("phi-3");
```
7 changes: 7 additions & 0 deletions Migration.md.meta

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

47 changes: 41 additions & 6 deletions Runtime/LLMBuilder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ namespace LLMUnity
public class LLMBuilder : AssetPostprocessor
{
static List<StringPair> movedPairs = new List<StringPair>();
public static string BuildTempDir = Path.Combine(Application.temporaryCachePath, "LLMUnityBuild");
public static string BuildTempDir = Path.Combine(Directory.GetParent(Application.dataPath).FullName, "LLMUnityBuild");
static string movedCache = Path.Combine(BuildTempDir, "moved.json");

[InitializeOnLoadMethod]
Expand All @@ -36,6 +36,28 @@ public static string PluginLibraryDir(string platform, bool relative = false)
return Path.Combine(PluginDir(platform, relative), LLMUnitySetup.libraryName);
}

public static void Retry(System.Action action, int retries = 10, int delayMs = 100)
{
for (int i = 0; i < retries; i++)
{
try
{
action();
return;
}
catch (IOException)
{
if (i == retries - 1) throw;
System.Threading.Thread.Sleep(delayMs);
}
catch (System.UnauthorizedAccessException)
{
if (i == retries - 1) throw;
System.Threading.Thread.Sleep(delayMs);
}
}
}

/// <summary>
/// Performs an action for a file or a directory recursively
/// </summary>
Expand All @@ -46,7 +68,7 @@ public static void HandleActionFileRecursive(string source, string target, Actio
{
if (File.Exists(source))
{
actionCallback(source, target);
Retry(() => actionCallback(source, target));
}
else if (Directory.Exists(source))
{
Expand Down Expand Up @@ -106,8 +128,8 @@ public static bool DeletePath(string path)
LLMUnitySetup.LogError($"Safeguard: {path} will not be deleted because it may not be safe");
return false;
}
if (File.Exists(path)) File.Delete(path);
else if (Directory.Exists(path)) Directory.Delete(path, true);
if (File.Exists(path)) Retry(() => File.Delete(path));
else if (Directory.Exists(path)) Retry(() => Directory.Delete(path, true));
return true;
}

Expand Down Expand Up @@ -300,8 +322,21 @@ public static void Reset()
bool refresh = false;
foreach (var pair in movedPairs)
{
if (pair.source == "") refresh |= DeletePath(pair.target);
else refresh |= MoveAction(pair.target, pair.source, false);
if (pair.source == "")
{
refresh |= DeletePath(pair.target);
}
else
{
if (File.Exists(pair.source) || Directory.Exists(pair.source))
{
refresh |= DeletePath(pair.target);
}
else
{
refresh |= MoveAction(pair.target, pair.source, false);
}
}
}
if (refresh) AssetDatabase.Refresh();
DeletePath(movedCache);
Expand Down