DVIDIA Training · Research alpha · Simulation only
Local training.
Published evidence.
A small CPU pipeline for training visual predictors from Skillspace footage, with an optional movement branch for aligned action labels.
A local, inspectable pipeline
The tool checks actual videos and connected source groups, freezes training/development/test splits, fits a small model on a CPU, and exports learned weights with held-out measurements and integrity hashes.
The base learner needs Python 3.12+, NumPy, and FFmpeg with ffprobe. It requires no GPU, CUDA, account, or API key. Training runs offline by default; installing software and downloading public data are separate steps.
Visual fitting and movement fitting have different inputs. Footage trains the visual predictor. The optional movement branch requires compatible, validated numerical action supervision and a task adapter.
Synthetic visual baseline
Ten explicitly synthetic moving-disc videos exercise intake, fitting, evaluation, and export. PCA is fitted on training footage only; a ridge next-frame predictor uses development groups to select regularization, then scores held-out footage.
- Synthetic clips
- 10
- Source duration
- 60 seconds
- Median pipeline time
- 0.5106 seconds
- Train / dev / test groups
- 7 / 2 / 1
| Predictor | RGB mean squared error |
|---|---|
| Learned next-frame predictor | 0.0003038793 |
| Last-frame baseline | 0.0017714138 |
| Training-mean frame baseline | 0.0013307309 |
Three pipeline repeats on one Apple M5 macOS host with 24 GiB RAM; the same held-out recording, containing 11 correlated transitions, was re-scored. Timing covers intake, fitting, and export. Software installation, imports, synthetic source creation, and the separate integrity inspection are excluded. This is a small software-throughput measurement, not a robotics success rate or a minimum hardware requirement.
A learned movement candidate
A separate native simulation pilot fits a six-joint delta head from aligned simulator state and authored joint targets. The training set contains 4,029 numerical labels across seven recordings; development uses two recordings, and one recording is held out.
| Movement predictor | Mean squared error |
|---|---|
| Learned joint-delta head | 5.7679 × 10−7 |
| Training-mean delta baseline | 7.8429 × 10−5 |
| Zero-delta baseline | 8.1727 × 10−5 |
The candidate retains authored task phases, gripper/contact logic, and recovery. Control uses privileged simulator observations; camera perception is not used to drive the arm. The labels were not inferred from human video.
One held-out scene, replayed
The candidate is installed into the bounded native placement adapter and tested in the exact scene of the held-out synthetic recording. The arm establishes contact, lifts the free box, and places it at the target. A control replay with identical joint targets and open jaws fails the placement test.
| Run | Recorded outcome |
|---|---|
| Learned candidate with contact/gripper supervisor | Pass |
| Identical joint targets, open jaws | Fail |
The replay displays saved native physics states. Playback, scrubbing, and view rotation do not run new training or a new physics experiment. A passing recorded scene does not establish generalization to another scene or a physical robot.
Run the tool locally
Install Python 3.12+ and FFmpeg first, then use these macOS/Linux commands. The local studio opens at 127.0.0.1:8270 on your own machine.
git clone https://github.com/Dvidia-Inference/dvidia-training.git
cd dvidia-training
python3.12 -m venv .venv
.venv/bin/python -m pip install .
.venv/bin/dvidia-training-studio --directory runs/studioTo reproduce the synthetic visual path, create a small example, train it, and inspect its export. Each run requires a fresh output directory.
.venv/bin/dvidia-train example --clips 10 --output examples/visual-demo
.venv/bin/dvidia-train run examples/visual-demo --output runs/visual-demo
.venv/bin/dvidia-train inspect runs/visual-demoThe optional native arm branch adds MuJoCo and the capsule tools through .venv/bin/python -m pip install ".[arm]". Its task contract and closed-loop qualification are separate from visual fitting.
Inspect the published artifacts
The source, example data, and pilot outputs are available for inspection and local reproduction. Integrity hashes check consistency; they do not authenticate capture history or independently establish measured truth.
- Example dataset ↗
- Explicitly synthetic visual examples and native simulation evidence, with frozen prepared inputs under
visual/preparedandnative/prepared. These are software and simulation fixtures, not recorded human skill demonstrations. - Pilot artifacts ↗
- Visual weights, the learned movement candidate, evaluation reports, and qualification/control evidence. A movement candidate remains bound to its declared adapter and scene qualification.
- Source repository ↗
- The MIT-licensed local training tool, installation guides, data contracts, and reproduction commands.
- Research publication ↗
- The pilot methods, observations, and measured limitations in the DVIDIA research notebook.
What remains open
The long-term aim is installable robot skills. This research alpha is a smaller, inspectable step toward that aim.
- Visual prediction error does not establish task success, calibrated geometry, or robot competence.
- No human-video action bridge, shoe organization policy, or demonstration sufficiency threshold has been established.
- The movement evidence covers one held-out synthetic recording and its exact simulation scene. Independent scene generalization remains unmeasured.
- Task phases, contact/gripper logic, and recovery remain authored. Hardware sensing, calibration, actuation, and physical qualification remain separate work.
Research alpha · no qualified hardware skill · no hosted training or uploads.