Skip to content

Refactor and optimize the dataset package interfacing for Go idioms (reference: PyTorch) #102

@gitctrlx

Description

@gitctrlx

Background

The current dataset package in this framework could be streamlined for better usability and maintainability. PyTorch offers a robust and flexible dataset/dataloader design which is widely recognized in the ML community. However, Go's language features and developer habits are significantly different from Python.

Proposal

  • Analyze PyTorch's Dataset and DataLoader implementations as reference.
  • Redesign and refactor the dataset package to offer interfaces, conventions, and usage more idiomatic for Go users.
  • Provide concise examples demonstrating both basic and advanced usage.
  • Ensure loose coupling, testability, and extensibility.
  • Document design decisions and migration guidance from old APIs.

Goals

  • Deliver an ergonomic and Go-centric dataset interface while retaining the power and flexibility seen in PyTorch.
  • Reduce friction for Go ML developers transitioning from PyTorch or other Python-based frameworks.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions