Skip to content

Commit 6884c97

Browse files
authored
Merge pull request #175 from fiso64/spectre-rendering
Rewrite CLI progress rendering to use Spectre.Console
2 parents c730abb + 67d439f commit 6884c97

55 files changed

Lines changed: 4399 additions & 2342 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

TODO.md

Lines changed: 1 addition & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -17,33 +17,7 @@
1717

1818
- Skip retrieve full folder contents whenever it's already guaranteed to contain all files (e.g. when it was `cd`'d into).
1919

20-
- In interactive mode, show search results (for albums or individual files) immediately as soon as they arrive instead of waiting for the search to complete. Sort every time before showing the updated results. Show a loading indicator while the search is in progress. When the user has e.g. some result selected, updates should be handled cleanly:
21-
- If a new result arrives that will be sorted before the currently selected result, set the selected result index to the minimal index of the new results after updating.
22-
- If all new results are to be sorted AFTER the currently selected result, there is no need to change the currently selected index.
23-
E.g.: When the current selected index is 5 and a new result arrives: If after sorting the new result has index <= 5, set the selection to its index. If the new result has index > 5, keep the current selection index at 5.
24-
25-
### CLI Rendering
26-
Progress rendering is broken garbage. Try to use Terminal.Gui in a future PR instead.
27-
28-
- Leave --no-progress and json progress modes as they are
29-
- In normal progress mode, replace everything by a full TUI using terminal.gui
30-
- Old rendering logic should remain available with --legacy-rendering, but the code should be isolated.
31-
32-
Console rendering will be completely redone.
33-
34-
Design:
35-
- Main list of jobs. Each job entry has two text lines, one of which includes a progress bar (and/or percentage).
36-
- Each entry is clickable. When clicked, create a right side panel and show job details.
37-
- For song jobs, show useful info like peer username and speed, file properties including full filename (the name in the job list might be truncated), etc.
38-
- For album jobs, the details show album folder info and individual track progress (similar to how it is rendered now without --album-compact-progress)
39-
- For all jobs, details should be followed by a section showing logs produced by that specific job
40-
- For interactive album downloads when interaction is required, create and navigate to a new tab where the choices (results) are listed. For now, this can remain similar to how interactive mode is currently rendered: One result at a time, prev/next navigation, some shortcuts. Accepting the result closes the tab (if possible, switch to the next interactive tab on close if it exists).
41-
- Logs can also be printed to the main job list if possible (single line). Long logs (like errors) should still be printed as single lines, but clickable and show the full message in the details pane when clicked.
42-
- If this is not possible, we can also add a bottom log pane (though that would make the UI cluttered) or a dedicated log tab.
43-
- Cancellation prompt becomes an actual prompt box.
44-
- Esc, q should show a prompt asking to cancel all jobs (like `c` -> `all`).
45-
- Status bar which shows the `c cancel job` shortcut (`c` is clickable if possible) (Esc/q is self-explanatory) and overall progress.
46-
- Need to think how overall progress should be determined, as we don't always know things ahead of time. Maybe dynamic completed/total counts for every job type (extract, song, album). So it would start with 0/1 extract jobs -> 1/1 extract jobs and 0/Total song jobs and 0/Total album jobs based on what was extracted, or if the extract job produces more extract jobs, -> 1/Total extract jobs.
20+
- Logging is a scattered, inconsistent mess.
4721

4822
### YAML
4923
Maybe use yaml for settings instead of our custom format, and improve structure.
Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
#!/usr/bin/env python3
2+
"""Create a fake local music library for --mock-files-dir testing.
3+
4+
The generated files are not valid FLAC audio, but they have .flac extensions and
5+
real file sizes. Use with --mock-files-no-read-tags.
6+
"""
7+
8+
from __future__ import annotations
9+
10+
import argparse
11+
import csv
12+
import random
13+
import re
14+
from pathlib import Path
15+
16+
17+
ALBUMS: list[tuple[str, str]] = [
18+
("Radiohead", "In Rainbows"),
19+
("Daft Punk", "Discovery"),
20+
("Kendrick Lamar", "To Pimp a Butterfly"),
21+
("Fleetwood Mac", "Rumours"),
22+
("Nirvana", "Nevermind"),
23+
("The Beatles", "Abbey Road"),
24+
("Pink Floyd", "The Dark Side of the Moon"),
25+
("Stevie Wonder", "Songs in the Key of Life"),
26+
("Joni Mitchell", "Blue"),
27+
("Miles Davis", "Kind of Blue"),
28+
("The Strokes", "Is This It"),
29+
("Arcade Fire", "Funeral"),
30+
("Portishead", "Dummy"),
31+
("Massive Attack", "Mezzanine"),
32+
("Bjork", "Homogenic"),
33+
("Aphex Twin", "Selected Ambient Works 85-92"),
34+
("The Cure", "Disintegration"),
35+
("Prince", "Purple Rain"),
36+
("Kate Bush", "Hounds of Love"),
37+
("David Bowie", "Low"),
38+
("Lauryn Hill", "The Miseducation of Lauryn Hill"),
39+
("OutKast", "Aquemini"),
40+
("A Tribe Called Quest", "The Low End Theory"),
41+
("Public Enemy", "It Takes a Nation of Millions"),
42+
("Nas", "Illmatic"),
43+
("Wu-Tang Clan", "Enter the Wu-Tang"),
44+
("The Clash", "London Calling"),
45+
("Television", "Marquee Moon"),
46+
("Joy Division", "Unknown Pleasures"),
47+
("Talking Heads", "Remain in Light"),
48+
("Sonic Youth", "Daydream Nation"),
49+
("My Bloody Valentine", "Loveless"),
50+
("Neutral Milk Hotel", "In the Aeroplane Over the Sea"),
51+
("Elliott Smith", "Either Or"),
52+
("Sufjan Stevens", "Illinois"),
53+
("Bon Iver", "For Emma Forever Ago"),
54+
("Frank Ocean", "Blonde"),
55+
("FKA twigs", "LP1"),
56+
("LCD Soundsystem", "Sound of Silver"),
57+
("The Avalanches", "Since I Left You"),
58+
("Burial", "Untrue"),
59+
("Boards of Canada", "Music Has the Right to Children"),
60+
("Tame Impala", "Currents"),
61+
("Gorillaz", "Demon Days"),
62+
("The National", "Boxer"),
63+
("PJ Harvey", "Stories from the City Stories from the Sea"),
64+
("Mitski", "Be the Cowboy"),
65+
("Phoebe Bridgers", "Punisher"),
66+
("The War on Drugs", "Lost in the Dream"),
67+
("Wilco", "Yankee Hotel Foxtrot"),
68+
]
69+
70+
TITLE_WORDS = [
71+
"Midnight",
72+
"Signal",
73+
"Golden",
74+
"Static",
75+
"River",
76+
"Mirror",
77+
"City",
78+
"Garden",
79+
"Neon",
80+
"Summer",
81+
"Winter",
82+
"Velvet",
83+
"Satellite",
84+
"Memory",
85+
"Dream",
86+
"Ocean",
87+
"Street",
88+
"Horizon",
89+
"Fever",
90+
"Echo",
91+
"Light",
92+
"Shadow",
93+
"Palace",
94+
"Weather",
95+
"Morning",
96+
"Night",
97+
"Glass",
98+
"Fire",
99+
"Paper",
100+
"Silver",
101+
]
102+
103+
104+
def safe_path_part(value: str) -> str:
105+
value = re.sub(r'[<>:"/\\|?*]', "", value)
106+
value = re.sub(r"\s+", " ", value).strip()
107+
return value.rstrip(". ")
108+
109+
110+
def make_track_title(rng: random.Random, used: set[str]) -> str:
111+
while True:
112+
words = rng.sample(TITLE_WORDS, rng.randint(2, 4))
113+
title = " ".join(words)
114+
if title not in used:
115+
used.add(title)
116+
return title
117+
118+
119+
def create_fake_file(path: Path, size_bytes: int) -> None:
120+
path.parent.mkdir(parents=True, exist_ok=True)
121+
with path.open("wb") as handle:
122+
handle.truncate(size_bytes)
123+
124+
125+
def generate(root: Path, seed: int) -> None:
126+
rng = random.Random(seed)
127+
library_dir = root / "mock-library"
128+
csv_dir = root / "csv"
129+
csv_dir.mkdir(parents=True, exist_ok=True)
130+
131+
track_rows: list[dict[str, str]] = []
132+
album_rows: list[dict[str, str]] = []
133+
134+
for artist, album in ALBUMS:
135+
track_count = rng.randint(8, 12)
136+
album_rows.append({"artist": artist, "title": "", "album": album})
137+
used_titles: set[str] = set()
138+
139+
for track_number in range(1, track_count + 1):
140+
title = make_track_title(rng, used_titles)
141+
size_bytes = rng.randint(1, 5) * 1024 * 1024 + rng.randint(0, 1023)
142+
filename = f"{track_number:02d}. {artist} - {title}.flac"
143+
path = (
144+
library_dir
145+
/ safe_path_part(artist)
146+
/ safe_path_part(album)
147+
/ safe_path_part(filename)
148+
)
149+
create_fake_file(path, size_bytes)
150+
151+
track_rows.append({
152+
"artist": artist,
153+
"title": title,
154+
"album": album,
155+
})
156+
157+
selected_tracks = rng.sample(track_rows, 100)
158+
selected_albums = rng.sample(album_rows, 30)
159+
160+
list_lines: list[str] =[]
161+
sample_albums = rng.sample(album_rows, 25)
162+
sample_tracks = rng.sample(track_rows, 25)
163+
164+
for album in sample_albums:
165+
list_lines.append(f'a:"{album["artist"]} - {album["album"]}"')
166+
for track in sample_tracks:
167+
list_lines.append(f'"{track["artist"]} - {track["title"]}"')
168+
169+
rng.shuffle(list_lines)
170+
171+
tracks_csv = csv_dir / "tracks_to_download.csv"
172+
albums_csv = csv_dir / "albums_to_download.csv"
173+
list_txt = csv_dir / "list.txt"
174+
175+
with tracks_csv.open("w", newline="", encoding="utf-8") as handle:
176+
writer = csv.DictWriter(handle, fieldnames=["artist", "title", "album"])
177+
writer.writeheader()
178+
writer.writerows(selected_tracks)
179+
180+
with albums_csv.open("w", newline="", encoding="utf-8") as handle:
181+
writer = csv.DictWriter(handle, fieldnames=["artist", "title", "album"])
182+
writer.writeheader()
183+
writer.writerows(selected_albums)
184+
185+
with list_txt.open("w", encoding="utf-8") as handle:
186+
for line in list_lines:
187+
handle.write(line + "\n")
188+
189+
print(f"Created library: {library_dir}")
190+
print(f"Created tracks CSV: {tracks_csv}")
191+
print(f"Created albums CSV: {albums_csv}")
192+
print(f"Created list.txt: {list_txt}")
193+
print()
194+
print("Example commands:")
195+
print(f' sldl "{tracks_csv}" --mock-files-dir "{library_dir}" --mock-files-no-read-tags --mock-files-slow')
196+
print(f' sldl "{albums_csv}" --mock-files-dir "{library_dir}" --mock-files-no-read-tags --mock-files-slow')
197+
print(f' sldl "{list_txt}" --input-type list --mock-files-dir "{library_dir}" --mock-files-no-read-tags --mock-files-slow')
198+
199+
200+
def main() -> None:
201+
parser = argparse.ArgumentParser(description=__doc__)
202+
parser.add_argument(
203+
"-o",
204+
"--output",
205+
type=Path,
206+
required=True,
207+
help="Output directory for the generated library and CSV files.",
208+
)
209+
parser.add_argument(
210+
"--seed",
211+
type=int,
212+
default=55,
213+
help="Random seed for deterministic fixture generation.",
214+
)
215+
args = parser.parse_args()
216+
217+
generate(args.output, args.seed)
218+
219+
220+
if __name__ == "__main__":
221+
main()

slsk-batchdl.Cli.Tests/ConfigParsingTests.cs

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -338,6 +338,13 @@ public void Bool_InvertedFlag_MockFilesNoReadTags()
338338
Assert.IsFalse(eng.MockFilesReadTags);
339339
}
340340

341+
[TestMethod]
342+
public void Engine_MockFilesFailDownloads()
343+
{
344+
var (eng, _, _) = Bind("--mock-files-fail-downloads", "3");
345+
Assert.AreEqual(3, eng.MockFilesFailDownloads);
346+
}
347+
341348
[TestMethod]
342349
public void RemotePatch_ExplicitDefaultBool_IsRepresented()
343350
{

slsk-batchdl.Cli.Tests/EndToEndTests.cs

Lines changed: 27 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
using Sldl.Core.Services;
55
using Sldl.Core.Settings;
66
using Sldl.Cli;
7+
using Sldl.Server;
78

89
namespace Tests.EndToEnd;
910

@@ -111,8 +112,9 @@ public async Task InteractiveAlbumSelection_FromList_IsSerialized()
111112
var maxActivePickers = 0;
112113
var pickerCalls = 0;
113114

115+
var backend = new LocalCliBackend(app, rootSettings);
114116
var coordinator = new InteractiveCliCoordinator(
115-
app,
117+
backend,
116118
cliSettings,
117119
CancellationToken.None,
118120
async request =>
@@ -144,9 +146,13 @@ public async Task InteractiveAlbumSelection_FromList_IsSerialized()
144146
}
145147
});
146148

147-
coordinator.Start(new ExtractJob(rootSettings.Extraction.Input!, rootSettings.Extraction.InputType), rootSettings);
148-
149149
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(15));
150+
var workflowId = Guid.NewGuid();
151+
var submission = await coordinator.StartAsync(
152+
new SubmitExtractJobRequestDto(rootSettings.Extraction.Input!, rootSettings.Extraction.InputType.ToString(), Options: new SubmissionOptionsDto(workflowId)),
153+
cts.Token);
154+
_ = coordinator.RunUntilCompleteAsync(submission.WorkflowId, cts.Token)
155+
.ContinueWith(_ => app.CompleteEnqueue(), TaskScheduler.Default);
150156
await app.RunAsync(cts.Token);
151157

152158
Assert.IsFalse(cts.IsCancellationRequested, "RunAsync timed out");
@@ -218,8 +224,9 @@ public async Task InteractiveAlbumSelection_CancelledChosenAlbum_DoesNotReprompt
218224
albumJob.Cancel();
219225
};
220226

227+
var backend = new LocalCliBackend(app, rootSettings);
221228
var coordinator = new InteractiveCliCoordinator(
222-
app,
229+
backend,
223230
cliSettings,
224231
CancellationToken.None,
225232
request =>
@@ -235,9 +242,13 @@ public async Task InteractiveAlbumSelection_CancelledChosenAlbum_DoesNotReprompt
235242
request.FilterStr));
236243
});
237244

238-
coordinator.Start(new ExtractJob(rootSettings.Extraction.Input!, rootSettings.Extraction.InputType), rootSettings);
239-
240245
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(15));
246+
var workflowId = Guid.NewGuid();
247+
var submission = await coordinator.StartAsync(
248+
new SubmitExtractJobRequestDto(rootSettings.Extraction.Input!, rootSettings.Extraction.InputType.ToString(), Options: new SubmissionOptionsDto(workflowId)),
249+
cts.Token);
250+
_ = coordinator.RunUntilCompleteAsync(submission.WorkflowId, cts.Token)
251+
.ContinueWith(_ => app.CompleteEnqueue(), TaskScheduler.Default);
241252
await app.RunAsync(cts.Token);
242253

243254
Assert.IsFalse(cts.IsCancellationRequested, "RunAsync timed out");
@@ -289,17 +300,18 @@ public async Task InteractiveAlbumSelection_FailedChosenAlbum_RepromptsWithoutFa
289300
var app = new DownloadEngine(engineSettings, clientManager);
290301

291302
var pickerCalls = 0;
303+
var backend = new LocalCliBackend(app, rootSettings);
292304
var coordinator = new InteractiveCliCoordinator(
293-
app,
305+
backend,
294306
cliSettings,
295307
CancellationToken.None,
296308
request =>
297309
{
298310
pickerCalls++;
299-
311+
300312
Assert.IsTrue(request.Folders.Count >= 1, "Expected at least 1 folder candidate available to pick.");
301313
var folder = request.Folders.First();
302-
314+
303315
// We delete the file here so the download fails
304316
if (pickerCalls == 1 && File.Exists(doomedFilePath))
305317
{
@@ -314,8 +326,12 @@ public async Task InteractiveAlbumSelection_FailedChosenAlbum_RepromptsWithoutFa
314326
request.FilterStr));
315327
});
316328

317-
coordinator.Start(new ExtractJob(rootSettings.Extraction.Input!, rootSettings.Extraction.InputType), rootSettings);
318-
329+
var workflowId = Guid.NewGuid();
330+
var submission = await coordinator.StartAsync(
331+
new SubmitExtractJobRequestDto(rootSettings.Extraction.Input!, rootSettings.Extraction.InputType.ToString(), Options: new SubmissionOptionsDto(workflowId)),
332+
CancellationToken.None);
333+
_ = coordinator.RunUntilCompleteAsync(submission.WorkflowId, CancellationToken.None)
334+
.ContinueWith(_ => app.CompleteEnqueue(), TaskScheduler.Default);
319335
await app.RunAsync(CancellationToken.None);
320336

321337
Assert.AreEqual(2, pickerCalls, "A failed chosen album should reopen the picker with remaining candidates.");

0 commit comments

Comments
 (0)