eta: the estimate clamps to exactly 2 hours and presents it as a real number

[LeyckerS]

Reported from a real run: the ETA showed ~4-5 minutes, jumped to 2 hours for a few seconds, came back to 2-3 minutes, then oscillated — 2.10, then 1.99, then 2.22 — before settling. On a second run it swung from 5 hours to 2 minutes and then to --.

Two separate causes, both in Engine.snapshot() (moon_engine.py:586-591).

1. The 2 hours is a clamp, not an estimate

eta = min(files_remaining * avg_file / (mbs * 1_048_576), 7200)

7200 seconds is exactly 2 hours. Whenever the computation blows up — which it does early in a run, when mbs is momentarily tiny — the result is clamped and the GUI renders it as "2h", indistinguishable from a genuine estimate. The operator sees a confident, precise, wrong number.

A clamped value means "I don't know". It should be displayed as unknown (--), the same as the eta = 0.0 branch already is, not as its ceiling.

2. The estimate is built on a badly skewed average early on

avg_file = total_downloaded / dl_done

total_downloaded sums bytes from every file including the ones still in flight, while dl_done counts only the ones that finished. Early in a run — say 40 files downloading, 2 finished — this divides 40 files' worth of bytes by 2, so avg_file is wildly overestimated and the ETA with it. That is the "5 hours" reading.

3. The oscillation

mbs comes from a 3-second window:

recent = [(t, b) for t, b in snap if t > now - 3.0]

Three seconds is short enough that normal chunk-arrival jitter moves the ETA visibly every refresh. 2.10 → 1.99 → 2.22 is that noise, not the estimate genuinely changing.

What to do

• Return None (rendered --) when the estimate is clamped or not yet meaningful, instead of the ceiling
• Compute avg_file from completed files only, or from bytes_total / (dl_done + partial progress of in-flight files) — either is defensible, say which you chose
• Smooth the rate used for the ETA over a longer window than the 3 seconds used for the live speed display, or apply an exponential moving average. The displayed speed should stay responsive; the ETA does not need to be

Acceptance criteria

• No run ever displays exactly "2h" as a result of the clamp
• The ETA does not visibly jitter every refresh once a run is in steady state
• The live speed readout stays as responsive as it is today

tests/test_snapshot.py is the place for the regression tests, and #57 is adding contract tests there already — coordinate if both are in flight.

[8nt0n]

Hey, i would love to work on this issue (I'll check out #57 so my updates to tests/test_snapshot.py don't collide with their contract tests).

My planned approach:

• Clamp Fix: Return None (rendering --) whenever the ETA calculation hits the 7200s ceiling or isnt meaningful yet.
• Average File Size: Calculate avg_file using completed files/bytes only (completed_bytes / dl_done) to prevent early-run skew (i believe this will likely result in a small underestimation early in the run since smaller files finish downloading first, is that okay?).
• Smoothing: Keep the responsive 3 second window for the live speed readout, but use a longer window (roughly 10-15s) for the ETA rate.

[LeyckerS]

@8nt0n — yours, assigned. The plan is right on all three points, and checking #57 for collisions before starting is exactly the correct instinct. Small note: the file you would be touching is tests/test_snapshot_speed.py; there is no tests/test_snapshot.py yet — that is what #57 will create, so the collision risk is even lower than you assumed.

To your direct question — yes, that is okay, but the skew is not the one you are expecting, and the current code is worse than you think.

Here is the whole expression, moon_engine.py:728:

total_downloaded = sum(b for _, b in snap)
files_remaining  = dl_tot - dl_done
if mbs > 0.1 and files_remaining > 0 and dl_done > 0:
    avg_file = total_downloaded / dl_done

snap is the byte-sample deque, so total_downloaded is every byte the run has pulled, including partial bytes of files still in flight. Dividing that by dl_done, a count of completed files only, mixes two populations: the numerator covers ~48 concurrent transfers, the denominator counts only the finished ones. avg_file therefore overestimates — and it does so most severely early in the run, when many files are open and few have closed. That is the opposite direction from the underestimate you were worried about introducing, and it is larger.

So completed_bytes / dl_done is not a trade that adds skew. It removes a bigger one and replaces it with a smaller one in the other direction. Go ahead.

One thing that may let you skip the averaging entirely. rec.file_bytes is set from Content-Length when the transfer starts (moon_download.py:567), not when it finishes, and the engine holds live records in self._tracked. So the sizes of in-flight files are known, not estimated. That makes a much more direct formula available:

remaining_bytes = Σ (file_bytes − done_bytes)  over tracked in-flight files
                + (files not yet started) × (average of the sizes you do know)

Only the last term is a guess, and it shrinks as the run proceeds. Worth trying before falling back to a single average — but if it turns out messier than it reads here, the averaging fix is a real improvement on its own and I will take it. Your call; you will see the shape of it once you are in there.

On the 10–15s ETA window against the 3s live-speed window: agreed, and keeping them separate is the right call. The 3s window is honest for a number that is supposed to twitch; an ETA that twitches is just noise.

Last thing, on returning None for the clamp: check what the front-ends do with it before you commit to that. web/app.js reads m.eta_s, and 0.0 is the current sentinel for "not meaningful" — so None may need a matching change there to render -- rather than something odd. If it does, that change is in scope; just call it out in the PR body so it does not read as unrelated.

[8nt0n]

Hey, thank you for the feedback - the lower collision risk is a blessing. I really like your approach for calculating remaining bytes and will definitely try to implement it that way first. its been a few days without any activity on this on my side so im just letting you know that im still actively working on this and i expect to have a PR ready in the next few days