Distributed ML Training Framework
Developed an MVP for coordinating machine-learning training across heterogeneous computers, combining PyTorch Distributed Data Parallel with node registration, telemetry, and job scheduling.
Related Projects
SYSTEM ARCHITECTURE
Heterogeneous consumer nodes coordinated through the cluster-management layer.
Problem
Consumer laptops have uneven CPU, RAM, and GPU capabilities, making it challenging to coordinate training jobs and monitor available compute resources across a local network.
Architecture
Node agents → gRPC-based cluster management and telemetry → SQLite state and job queue → resource-aware scheduler → PyTorch DDP training.
Challenges
- Implemented node registration and periodic hardware telemetry
- Connected job submission with a persistent queue
- Added basic resource-aware scheduling across registered nodes
- Tested local multi-process execution and simulated multi-node workflows
Benchmarks
MVP validation focused on cluster orchestration and telemetry. Multi-node behavior was partly simulated; do not present it as a production-scale distributed training benchmark.
Lessons Learned
- Cluster orchestration and model-gradient synchronization are separate responsibilities
- Telemetry is needed to make resource-aware scheduling decisions
- Networking, process startup, and heterogeneous hardware complicate distributed training