Merge pull request #80 from cybertronai/YAD/adopt-bytedmd

Adopt ByteDMD as primary metric (vendored from cybertronai/ByteDMD)

Author: yaroslavvb

CLAUDE.md

@@ -22,7 +22,8 @@ This is a research workspace for the **Sutro Group**, a study group exploring en
 - **Data Movement Complexity (DMC)**: Better proxy metric (Ding et al., arXiv:2312.14441). DMC = sum of sqrt(stack_distance) for all float accesses. Tracks alongside ARD in MemTracker. Baseline: ARD 4,104 / DMC 300,298.
 - **Cache Energy Model**: register 5pJ, L1 (64KB) 20pJ, L2 (256KB) 100pJ, HBM 640pJ per float access (Bill Dally numbers).
 - **CacheTracker**: Extended MemTracker with LRU cache simulation for realistic energy estimates.
-- **TrackedArray / Auto DMD**: `TrackedArray` wraps numpy arrays so every operation (ufuncs, indexing, slicing) auto-records reads and writes on an `LRUStackTracker`. Removes manual instrumentation errors. See `docs/research/tracked-numpy.md`.
+- **TrackedArray / Auto DMD**: `TrackedArray` wraps numpy arrays so every operation (ufuncs, indexing, slicing) auto-records reads and writes on an `LRUStackTracker`. Removes manual instrumentation errors. Legacy element-level metric. See `docs/research/tracked-numpy.md`.
+- **ByteDMD (primary metric, new)**: Pure Python tracer at `src/bytedmd/`. Wraps Python values; traces Python-level ops via dunders. Byte-granularity LRU stack with simultaneous pricing and eager liveness compaction. Reads cost `ceil(sqrt(depth))`; writes are free. Spec/reference: https://github.com/cybertronai/ByteDMD. Use `bytedmd(func, args)` for cost or `traced_eval(func, args)` for the trace. Submissions to the challenge should use Python ops (no numpy) to be fully tracked.
 
 ## Current Best Methods
 

docs/changelog.md

@@ -2,6 +2,17 @@
 
 All notable changes to this research workspace.
 
+## [0.27.0] - 2026-04-14
+
+### ByteDMD adopted as primary metric
+
+- Vendored [cybertronai/ByteDMD](https://github.com/cybertronai/ByteDMD) at `src/bytedmd/`
+- 13 tests (10 core + 3 gotchas) all pass
+- TrackedArray retained as legacy tracker for existing experiments (30 tests still pass)
+- New docs page: `docs/research/bytedmd.md`
+- Decision made by Yaroslav after meetings with Wesley Smith and Bill Dally feedback. Byte-granularity rewards smaller dtypes; pure Python eliminates numpy escape hatches.
+- Existing challenge submissions stay on legacy DMC. New submissions use ByteDMD.
+
 ## [0.26.0] - 2026-03-28
 
 Largest release: auto-instrumented DMD tracking, RL eval environment, agent infrastructure, Telegram SQLite sync, and documentation fixes. 84 commits from 4 contributors.

docs/google-docs/bigger-picture.md

@@ -115,6 +115,22 @@ Raw Thoughts
 
 ([[planning-sprint1.txt](https://drive.google.com/open?id=1Q2y9pdU4cT4Wn0MBE63XoSeg1xDbgwR-&usp=drive_fs)], [[planning-sprint2.txt](https://drive.google.com/open?id=1NmYwxYazoV5pXXEpV9D6I8J5iP9HB1fb&usp=drive_fs)], [[plannint-spring3.txt](https://drive.google.com/file/d/1xYjR7S4iSG0UeVRZzr-AS72FdsjQ4L-B/view?usp=sharing)], [[gemini](https://gemini.google.com/share/bef3c777a11b)])
 
+.,
+
+Yaroslav Sutro planning sprint #1
+
+Objective
+
+Solve the energy problem of AI. Similar mission to [[https://ml.energy/](https://ml.energy/)] .
+
+Method
+
+Blank slate rewrite. Draw on [[experience](https://docs.google.com/document/d/1PhArG_B1qrurqeXtqG41n_OorHw5J9kJcaZkhJOh7NU/edit?tab=t.0)] to prevent mistakes, (also, [[impractical research](https://www.youtube.com/watch?v=5CUTOOz-pYY&t=13s)])
+
+Raw Thoughts
+
+([[planning-sprint1.txt](https://drive.google.com/open?id=1Q2y9pdU4cT4Wn0MBE63XoSeg1xDbgwR-&usp=drive_fs)], [[planning-sprint2.txt](https://drive.google.com/open?id=1NmYwxYazoV5pXXEpV9D6I8J5iP9HB1fb&usp=drive_fs)], [[plannint-spring3.txt](https://drive.google.com/file/d/1xYjR7S4iSG0UeVRZzr-AS72FdsjQ4L-B/view?usp=sharing)], [[gemini](https://gemini.google.com/share/bef3c777a11b)])
+
 # Mistakes 
 
 1. Too simple of an objective
@@ -151,7 +167,7 @@ Math / kernels design.
 
 Hardware lottery (sarah [[hooker](https://hardwarelottery.github.io/)]), machine learning systems are [[stuck in a rut](https://dl.acm.org/doi/10.1145/3317550.3321441)] (Paul Barham, Michael Isard)
 
-To avoid \-- remove as many boundaries as possible. At some point, the problem becomes too big to make progress on, we are forced to partititon to make progress. If we are motivated by keeping the complexity manageable,  we can periodically re-adjust the partitioning. In contrast to the current paradigm which keeps problem partitioning fixed:
+To avoid \-- remove as many boundaries as possible. At some point, the problem becomes too big to make progress on, we are forced to partititon to make progress. If we are motivated by keeping the complexity manageable,  we can periodically re-adjust the partitioning. In contrast to the current paradigm which keeps problem partitioning fixed:
 
 - math people vs kernels people
 
@@ -206,3 +222,7 @@ Michael: [[sutro_challenge_3_sparce parity results.docx](https://docs.google.com
 2. The Metric [[Yaroslav Knowledge sprint #1](https://docs.google.com/document/d/105FkE_U5_cXA1o8sxMGrj1NuTxysfSBTZsLB7vift-M/edit?tab=t.0#heading=h.p686ct9trrt)]
 
 3. The Problem [[sutro group challenge #1: sparse parity](https://docs.google.com/document/d/16eeltCaTpiiM1t_m_5BSxRnqxoMoiJ-xn4cy0x-TFgc/edit?tab=t.0)]
+
+# Links 
+
+Deep Learning invention timeline: [[https://chatgpt.com/share/69cafda2-5cc0-83e8-99a9-575e876e2c7d](https://chatgpt.com/share/69cafda2-5cc0-83e8-99a9-575e876e2c7d)]

docs/google-docs/how-to-search.md

@@ -141,3 +141,15 @@ Click \"export chat history\", it'll save it to \"messages.html\"
 2.(optional) Use [[parse_telegram](https://drive.google.com/file/d/1NlGREDBJM7QXw_rmgSwsxdPiiCce3SFg/view?usp=sharing)] script to make the file smaller
 
 3. Drag the file to Gemini and ask your query
+
+1. Export archive. Click three verical dots[embedded image]
+
+Click \"View group info\"
+
+[embedded image]
+
+Click \"export chat history\", it'll save it to \"messages.html\"
+
+2.(optional) Use [[parse_telegram](https://drive.google.com/file/d/1NlGREDBJM7QXw_rmgSwsxdPiiCce3SFg/view?usp=sharing)] script to make the file smaller
+
+3. Drag the file to Gemini and ask your query

docs/google-docs/meeting-1-energy-intro.md

@@ -445,6 +445,40 @@ AI links (local to yaroslav)
 
 [4. Picture]
 
+AI links (local to yaroslav)
+
+[[https://gemini.google.com/app/baf86fdafcaac3af](https://gemini.google.com/app/baf86fdafcaac3af)]
+
+[[https://gemini.google.com/app/5c855e3a869d19cf](https://gemini.google.com/app/5c855e3a869d19cf)]
+
+[[https://chatgpt.com/c/6970003e-4114-8333-86a8-71a8a0e97b23](https://chatgpt.com/c/6970003e-4114-8333-86a8-71a8a0e97b23)]
+
+[Link to \"Slides\"]
+
+[Meeting notes summary]
+
+[Host: Yaroslav Bulatov]
+
+[Jackjack Ganbold \@JackJack]
+
+[Jonathan Belay (@Jonathan Belay )]
+
+[Anish Tondwalkar]
+
+[Seth Stafford]
+
+[Caleb Sirak]
+
+[Anushka Deshpande]
+
+[Daria Soboleva]
+
+[2. Companies & Named Entities]
+
+[3. Key Topics & Tidbits]
+
+[4. Picture]
+
 # Link to \"Slides\" 
 
 from [[energy-efficient learning](https://docs.google.com/document/d/1PhArG_B1qrurqeXtqG41n_OorHw5J9kJcaZkhJOh7NU/edit?tab=t.0#heading=h.cdeox1s0xsl1)]

docs/google-docs/meeting-2-forward-forward.md

@@ -1233,6 +1233,106 @@ Instead of a forward pass followed by a backward pass, the algorithm uses two fo
 - The Negative Pass (Sleep): The network processes \"negative\" or \"hallucinated\" data (e.g., a hybrid image of a 7 and a 1, or an image with a wrong label). The objective is to minimize \"Goodness\".\
 
 
+Key Technical Details:
+
+- Goodness Function: Typically defined as the sum of squared neural activities (ReLU outputs) in a layer.\
+  
+- Greedy Layer-wise Learning: Each layer has its own objective function and updates its weights immediately, independent of the layers above or below it. This removes the need for a global error signal.\
+  +1\
+  
+- Normalization: To prevent activity from simply exploding to maximize goodness, the input vector to each layer is length-normalized. This forces the layer to learn from the orientation (spatial pattern) of the activity, not the magnitude.\
+  
+- Inference: For classification, the label is usually embedded into the input (e.g., a one-hot code in the corner). At inference, the network runs multiple passes---one for each possible label---and selects the label that accumulates the highest \"goodness\" score.\
+  +1\
+  
+
+------------------------------------------------------------------------
+
+### 2. Summarized Questions and Concerns 
+
+During the discussion, the group likely identified several \"sticking points\" where the algorithm faces theoretical or practical hurdles:
+
+#### A. The \"Negative Data\" Problem 
+
+- Question: How do we generate valid \"negative data\" for complex tasks?
+- Concern: In the paper, Hinton uses spatial masks to stitch together MNIST digits. The group likely noted that generating high-quality negative data (that lies near the real data manifold but is clearly incorrect) is non-trivial for domains like NLP or Video. If the negative data is too easy (e.g., white noise), the network learns nothing; if it is too hard, the network fails to converge.
+
+#### B. The \"Greedy\" Limitation 
+
+- Question: Does greedy, layer-wise training limit the depth of learning?
+- Concern: Since Layer 1 optimizes its weights without feedback from Layer 10, it might discard information that seems useless locally but is critical for high-level abstraction. This contrasts with Backprop, which can credit-assign through depth.
+- Critique: This effectively turns a Deep Net into a stack of shallow nets. Early experiments show FF struggles to match Backprop on datasets much complex than MNIST (like CIFAR-10) without specific hacks.
+
+#### C. Inference Efficiency 
+
+- Question: Is this actually slower than Backprop?
+- Concern: The inference method (running the forward pass \$N\$ times, once for each class label) scales poorly. For a 1000-class ImageNet task, this would be prohibitively slow compared to a single Softmax pass in standard networks.
+
+#### D. Biological Plausibility vs. \"Hacks\" 
+
+- Question: Is Layer Normalization biologically plausible?
+- Concern: The algorithm relies heavily on normalizing activity vectors between layers to prevent goodness leakage. Some members may have argued that this is just as biologically suspect as Backprop, as it requires a global calculation over the layer.
+
+------------------------------------------------------------------------
+
+### 3. Follow-up Points & To-Dos 
+
+Based on the paper and the discussion, here are recommended next steps to deepen your understanding:
+
+Conceptual & Theoretical:
+
+- Read \"Mortal Computation\": Review Hinton's broader philosophy on \"Mortal Computation.\" It explains why he wants algorithms that run on analog hardware (where weights cannot be copied), which is the true use-case for FF.
+- Compare with Predictive Coding: FF is often compared to Predictive Coding (PC). Look into the differences: FF maximizes/minimizes \"Goodness\" (Energy), while PC typically minimizes \"Prediction Error.\"\
+  +1\
+  
+
+Experimental (Coding Projects):
+
+- Reproduce MNIST: Clone a simple PyTorch implementation.
+
+<!-- -->
+
+- Experiment: Visualize the \"Negative Data.\" Try changing the negative generation method (e.g., use random noise vs. permuted pixels) and observe how the accuracy collapses. This proves the algorithm relies entirely on the quality of the \"hallucinations.\"
+
+<!-- -->
+
+- The \"Sandwich\" Method: Address the inference speed concern by using FF to train the first 3 layers (as a feature extractor), freezing them, and then training a standard Linear Classifier (Softmax) on top using Backprop. This allows for \$O(1)\$ inference while still using FF for feature learning.
+- Permutation Invariance Test: Flatten MNIST images and shuffle the pixels. Train an FF network on this. It often performs surprisingly well compared to ConvNets, highlighting that FF relies on global correlations rather than local geometry.
+
+Advanced Reading:
+
+- The Recurrent Extension: If the \"Greedy\" critique (Point B) was a major concern, read the specific section of the paper on Recurrent Forward-Forward. Hinton proposes treating static images as a video sequence, allowing later layers to send feedback to earlier layers over time, which theoretically solves the greedy limitation.\
+
+This is a detailed summary of the reading group discussion regarding Geoffrey Hinton's \"The Forward-Forward Algorithm: Some Preliminary Investigations.\"
+
+This summary synthesizes the paper's core claims with the common critiques, questions, and follow-up topics that typically arise in technical deep dives on this work.
+
+------------------------------------------------------------------------
+
+### 1. Detailed Summary of the Paper 
+
+Title: The Forward-Forward Algorithm: Some Preliminary Investigations (Hinton, 2022)
+
+Core Motivation:
+
+The paper proposes an alternative to Backpropagation (BP). Hinton argues that BP is flawed as a model of biological learning and effectively blocks the development of energy-efficient analog hardware.
+
++1
+
+- Bio-Implausibility: The brain cannot \"freeze\" neural activity to propagate error derivatives backward, nor does it have the symmetric connectivity required to send errors back through the same synapses used for the forward pass.\
+  
+- Hardware Constraints: BP requires perfect knowledge of the forward pass to compute derivatives. This prevents the use of analog or \"black box\" hardware (which is variable and noisy) and prevents \"pipelining\" (processing a continuous stream of data) because the network must wait for the backward pass.\
+  
+
+The Mechanism (Forward-Forward):
+
+Instead of a forward pass followed by a backward pass, the algorithm uses two forward passes with opposing objectives.
+
+- The Positive Pass (Wake): The network processes \"real\" data (e.g., an image of a 7 combined with the label 7). The objective is to maximize \"Goodness\" (neural activity) in every layer.\
+  
+- The Negative Pass (Sleep): The network processes \"negative\" or \"hallucinated\" data (e.g., a hybrid image of a 7 and a 1, or an image with a wrong label). The objective is to minimize \"Goodness\".\
+  
+
 Key Technical Details:
 
 - Goodness Function: Typically defined as the sum of squared neural activities (ReLU outputs) in a layer.\