Rewrite create-torrent script#193
Open
zpc0 wants to merge 193 commits into
Open
Conversation
deltarpm and zchunk are unwanted attack surface.
Member
|
You can drop the .py extension, it isn't needed for scripts (only modules need it). See also formatting issues flagged by CI. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #193 +/- ##
==========================================
+ Coverage 75.00% 79.52% +4.51%
==========================================
Files 51 60 +9
Lines 6053 7271 +1218
==========================================
+ Hits 4540 5782 +1242
+ Misses 1513 1489 -24 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Author
Thank you for review. Lint issues are now fixed, and I removed .py extension. |
zpc0
force-pushed
the
create_torrent
branch
2 times, most recently
from
ddc33f9 to
5b2c0df
Compare
- Python based - Can create v2 hybrid torrent on both Debian and Fedora - Avoid use of outdated mktorrent - python3-libtorrent(Debian) or rb_libtorrent-python3(Fedora) are required - To set custom WebSeed URL, place URL under "repository_upload_remote_host" config with "webseed" id.
Author
|
Update:
|
Signed-off-by: Rafał Wojdyła <omeg@invisiblethingslab.com>
* origin/pr/204: example-configs/archlinux: remove non-existing pkgs bin86,dev86 Pull request description: `init-cache` fails for archlinux example due to missing packages `bin86`,`dev86`. Fix.
* origin/pr/176: Use hardened DNF configuration in mock
* origin/pr/203: fix(arch): create chroot pacman.conf.d in generate-pacman fix(arch): create devtools makepkg.conf.d/ Pull request description: Create local `devtools` configuration directories before writing to them.
The Fedora 41 used as dom0 base has newer packages already. Few comments about patched packages: - libwnck3 - the old version had Fedora downstream patch that used cairo to render icons, which was buggy (https://gitlab.gnome.org/GNOME/libwnck/-/merge_requests/10#note_2176858, QSB-104). This patch was dropped upstream, so no need for own package anymore - pykickstart, lorax - there used to be patches to add package verification, but now it's handled natively in lorax (via repository file, instead of kickstart repo lines) QubesOS/qubes-issues#9402
Easiest method to apply a patch in Windows.
Package-list is a list of proper dicts already, no need to split them manually. This interface is already part of python-debian 1.0.0 too, so there is a chance it won't change drastically anymore.
Switch to Fedora 42 and Debian bookworm in VM. Keep bookworm too, as it's used by Whonix 17. QubesOS/qubes-issues#8841 QubesOS/qubes-issues#9807
Bootloader isn't installed there yet, but having the filesystem allows systemd to mount it (it really wants to, even if not booing in EFI mode...) instead of failing. And also, set 4k sector size to avoid compatibility issues. QubesOS/qubes-issues#4974 Fixes QubesOS/qubes-issues#8954
* origin/pr/241: ansible: fix windows_builder_dvm_name in RPC policy templates
orig.tar.gz has no devel suffix, so any other dist sharing the reprepro pool keeps pinning the previous one and reprepro rejects the new orig with conflicting checksums. Find every Debian/Ubuntu dist sharing the pool and drop the stale source so the orig.tar.gz is freed. Binary .debs stay published. QubesOS/qubes-issues#10555
Update pykickstart in R4.3+, to include a few fixes. QubesOS/qubes-issues#8993 QubesOS/qubes-issues#10516
* origin/pr/242: publish_deb: clean stale devel sources across the shared pool
Here are the commands:
mkdir -p tests/fixtures/devel-vm/pool/main/q/qubes-utils
rsync rsync://ftp.qubes-os.org/qubes-mirror/repo/deb/devel/vm/pool/main/q/qubes-utils/qubes-utils_4.3.16+deb12u1+devel{8,9}.{dsc,tar.xz} tests/fixtures/devel-vm/pool/main/q/qubes-utils/
* origin/pr/239: (24 commits)
tests: cover is_safe_dep path traversal and update non-dict cache
tests: cover cli_main option-parser branches
tests: cover container executor unknown-client and get_user/get_group
tests: cover list_deps skip and no-packages branches
tests: add chroot_{rpm,deb,archlinux} install-packages init-cache test
tests: add more list-deps integration tests
README: document list-deps --exclude option and per-dist caveats
list-deps: add --exclude option, default to dropping qubes-* packages
list-deps rpm: drop '|| true' on generate-spec
list-deps archlinux: use only build deps, drop runtime depends
Make python-black happy
ci: add benchmark script for builds with/without cache
chroot_deb/build_deb: bake qubes keyring into base.tgz and skip pbuilder update when install-packages is set
chroot_rpm: save pre-installed deps as root_cache_install to avoid overwriting minimal root_cache
config: support empty-dict override via -o key= or -o key={} in CLI options
tests: cover list-deps plugin and CLI
Make mypy happy
README: add list-deps CLI and options to install pkgs
list-deps: expose as a CLI command
list-deps: add RPM, DEB and Arch Linux dist plugins
...
Pull request description:
Related to:
- QubesOS/qubes-issues#9702
- QubesOS/qubes-issues#9703
This is used to generate links in metalink file.
Not for each commit over and over.
Will be useful in other tests
It takes a while if it's fully populated, so avoid the search.
Linux 7.1 includes negated entries in .gitignore, which tar --exclude-vcs-ignores doesn't support. Since it's literally fresh clone, the simplest option is to just don't use .gitignore, as there should be no extra files.
Check if .gitignore is correctly parsed, including re-including files.
* origin/pr/243: reproduce DB_NOTFOUND files from deb.qubes-os.org/devel publish_deb: tolerate removetrack failure on stale tracking entry
Add three options to CLI cleanup cache:
--installer-chroot-only-unused drop chroots for dists not in
the current builder.yml host set
--installer-templates-only-unused drop cached templates whose name is
no longer in cache.templates
--installer-templates-only-old keep only the newest RPM per
template name in the cache
`qb cleanup all` now runs the three prunes plus --installer-bootstrap
by default. Use --no-XXX to restore the previous behavior.
Fixes QubesOS/qubes-issues#10878
* origin/pr/248: cleanup: prune outdated installer chroot and templates cache
* digests: iso: drop weak hashes from digests file iso: make detached signature on digests file, instead of cleartext one
* origin/pr/193: installer: rewrite create-torrent Pull request description: This is Draft. Current create-torrent problems: - On Debian, creating v2 torrent isn't supported.(And mktorrent is outdated software) - Setting custom webseed hosts isn't supported. Creating Python script which uses libtorrent-rasterbar Python bindings will resolve both. - Both Debian(12+) and Fedora provides libtorrent-rasterbar python bindings package - Setting custom properties is easier This is draft because supporting custom webseed url is in progress. Any reviews are welcome!
Fix config usage, and create_torrent_qubes function call.
Member
|
Ouch, github got confused about the merge commit... Anyway, I resolved conflict and fixed some issues with the script. Now it appears to work. I also tried to include the Another thing I see, is a |
Author
|
It seems that Transmission have Could you try |
Member
|
Well, it's not about what I can try, we want a torrrent file that works with (at least) most popular clients... |
Author
Indeed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is Draft.
Current create-torrent problems:
Creating Python script which uses libtorrent-rasterbar Python bindings will resolve both.
This is draft because supporting custom webseed url is in progress.
Any reviews are welcome!