Skip to content

Fix BytecodeDeserializer to return BytecodeTypes, restore Strict.Bytecode and Strict compilation#38

Draft
Copilot wants to merge 53 commits intomasterfrom
copilot/experiment-image-processing-parallelization
Draft

Fix BytecodeDeserializer to return BytecodeTypes, restore Strict.Bytecode and Strict compilation#38
Copilot wants to merge 53 commits intomasterfrom
copilot/experiment-image-processing-parallelization

Conversation

Copy link
Contributor

Copilot AI commented Mar 14, 2026

The BytecodeDeserializer was left in a half-reworked state: TypeEntryData, Package, Instructions, PrecompiledMethods were inside a TODO comment block, Deserialize() referenced non-existent variables, and Runner.cs had incomplete statements. Nothing compiled.

BytecodeDeserializer

  • Restored TypeEntryData as private class, fixed Deserialize(Package) to read ZIP entries, extract metadata into BytecodeTypes, and read instructions
  • Added ReadTypeMetadataIntoBytecodeTypes — captures member/method string data into BytecodeTypes.TypeMembersAndMethods while still creating Language types needed for instruction deserialization
  • Uses BuildMethodInstructionKey format (TypeName|MethodName|ParamCount) consistently — matches what VirtualMachine expects for precompiled method lookup

BytecodeTypes

  • Fixed method.FullName.Namemethod.Name (FullName is already a string)
  • Updated Find to use BuildMethodInstructionKey format via shared GetTypeNameFromEntryName

Runner

  • Restored constructor initialization of package, mainType, currentFolder for both .strict source and .strictbinary input
  • Build() and Run() now branch on deserializedBytecodeTypes != null to handle bytecode-loaded vs source-loaded types
  • Removed dead commented-out code blocks

Usage after changes

// File-based: returns BytecodeTypes (new unified data structure)
var types = new BytecodeDeserializer(filePath).Deserialize(basePackage);
var instructions = types.Find("SimpleCalculator", "Run", 0);

// In-memory (tests): sets Package/Instructions/PrecompiledMethods properties
var deserializer = new BytecodeDeserializer(entryBytes, package);
vm.Execute(deserializer.Instructions!.Values.First());

Test results

  • Strict.Bytecode.Tests: 67/67 ✅
  • Strict.Compiler.Assembly.Tests: 158/158 ✅
  • Strict.Expressions.Tests: 546/546 ✅
  • Strict.Optimizers.Tests: 59/59 ✅

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 4 commits March 14, 2026 04:39
…single-thread

Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
…threshold

Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
…iable

Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
@BenjaminNitschke
Copy link
Collaborator

@copilot continue, we want to test gpu impact. Also the loop count alone is not a good indicator of overall complexity, a 1million iterations loop with 1 instruction each is less complex than a 10k iteration loop with 1k instructions each, which would be a perfect candidate for parallization and gpu usage

Copilot AI changed the title [WIP] Add parallel processing for image brightness adjustment Add MLIR parallel loop support and GPU execution plan for image processing Mar 14, 2026
Copilot AI requested a review from BenjaminNitschke March 14, 2026 04:52
Copilot AI and others added 2 commits March 14, 2026 04:59
…tions) and add GPU impact projection

Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 14, 2026

@copilot continue, we want to test gpu impact. Also the loop count alone is not a good indicator of overall complexity, a 1million iterations loop with 1 instruction each is less complex than a 10k it...

Implemented in 343fe60 and 42d88e7:

Complexity-based threshold: Changed from simple iteration count to iterations × body instruction count. ShouldEmitParallelLoop now calls CountLoopBodyInstructions() to scan between LoopBegin/LoopEnd and multiply by iteration count. So:

  • 10K iterations × 20 body instructions = 200K complexity → parallelizes ✓
  • 50K iterations × 1 body instruction = 50K complexity → stays sequential ✓
  • 1M iterations × 1 instruction = 1M → parallelizes ✓

GPU impact: Added projection test based on BlurPerformanceTests CUDA reference data (143x speedup). For 4K brightness adjustment: SingleThread ~1.8s, ParallelCpu ~730ms (2.5x), projected GPU ~92ms (20x, conservative estimate since real CUDA showed 143x for blur). The test validates projected GPU time beats CPU parallel.

Threshold constant is InstructionsToMlir.ComplexityThreshold = 100_000 (public, used by tests).

Copilot AI changed the title Add MLIR parallel loop support and GPU execution plan for image processing Add MLIR parallel loop support with complexity-based threshold and GPU execution plan Mar 14, 2026
Copilot AI and others added 2 commits March 14, 2026 05:39
…ia unrolled interior kernel

Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
…zes instead

Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
Copilot AI changed the title Add MLIR parallel loop support with complexity-based threshold and GPU execution plan Add 5×5 blur performance tests with optimized kernel and complexity-based parallelization Mar 14, 2026
…e Cuda.Tests how CPU, Single threaded vs Parallized vs GPU performs. What matters is implementation to Mlir generation
Copilot AI and others added 2 commits March 14, 2026 13:36
…2 of GpuSupportPlan)

Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
… with Phase 2 implementation status

Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
Copilot AI changed the title Add 5×5 blur performance tests with optimized kernel and complexity-based parallelization Implement MLIR GPU dialect emission for high-complexity loops (Phase 2) Mar 14, 2026
BenjaminNitschke and others added 30 commits March 16, 2026 05:25
…still need to be checked. Other projects also need to be updated, but not much work left ..
…e simplified, but we are far away from functional ..
…ow, seems like 80% of code there is not that useful and can still be simplified
… need some more cleanup, but lets first fix all projects and tests, then more refactoring
…rs in all projects

Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
…BinaryMethod simplified again, more work to be done, still plenty of TODOs
…es, basically all the constants we have anyway in Strict.Language
…, VirtualMachine getting slightly better, more work to be done
…red CompiledMethodInfo and CollectMethods

- Remove IsPlatformUsingStdLibAndHasPrintInstructionsInternal (callers will use BinaryExecutable.UsesConsolePrint)
- Remove HasPrintInstructionsInternal (callers will use BinaryExecutable.UsesConsolePrint)
- Add protected sealed CompiledMethodInfo class to deduplicate from 3 subclasses
- Add CollectMethods, BuildMethodInfo, BuildMethodSymbol, EnqueueInvokedMethods, HasNumericPrint shared methods
- Use new List<Instruction>(precompiled) instead of collection expression for ternary compatibility

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… private, remove HasPrintInstructions/IsPlatformUsingStdLib, move shared code to base class, remove dead Runner methods

Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
- Replace compiler.Compile(method) with CompileMethod(method) helper
- Replace compiler.CompileForPlatform calls with Compile(instructions, platform) helper
- Simplify complex tests using BinaryGenerator to produce full binaries
- Remove unused BuildMethodKey and GenerateMethodInstructions helpers
- Add three new private helpers: CompileMethod, Compile(instructions), Compile(binary)
- Update assertions for tests that used custom method names to use 'Run'
- Remove unused Strict.Bytecode.Serialization using

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace all CompileForPlatform calls with Compile helper methods
- Replace HasPrintInstructions tests with BinaryExecutable.UsesConsolePrint
- Simplify BinaryExecutable tests to use BinaryGenerator directly
- Remove BuildMethodKey and GenerateMethodInstructions helpers
- Remove unused Strict.Bytecode.Serialization import

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace all compiler.CompileForPlatform() calls with Compile() helper
- Replace compiler.HasPrintInstructions() with BinaryExecutable.UsesConsolePrint
- Simplify BinaryGenerator tests by removing manual precompiled dictionaries
- Remove unused BuildMethodKey and GenerateMethodInstructions helpers
- Remove unused Strict.Bytecode.Serialization import
- Update assertion strings to reflect 'Run' entry point naming

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…instead of CompileForPlatform, remove duplicate helpers, simplify

Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
Co-authored-by: BenjaminNitschke <1650127+BenjaminNitschke@users.noreply.github.com>
…, removed old serializer and deserializer tests that are unused and not really compatible anymore, new tests are in BinaryExecutableTests
…ed benchmark VirtualMachine, but it is fast in benchmark and horribly slow in NCrunch, hard to say ..
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants