Adding loader for MULTIVOX - #696
Conversation
|
Hi @magdalenafuentes and @guillemcortes! @jesganaud and I (@iranroman) have been working on this data loader for MULTIVOX, which is a multimodal music dataset collected by @mezaga This implementation is complete. Please let us know if you need anything else from us for the workflows to be approved. Thanks! |
yujin-kimmm
left a comment
yujin-kimmm
left a comment
There was a problem hiding this comment.
Hi @jesganaud and @iranroman, thank you for making this PR!
I have left some comments after reviewing the code. Please check it and let us know whenever you need any help.
b2b8329 to
9f6a65c
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #696 +/- ##
==========================================
+ Coverage 97.06% 97.13% +0.06%
==========================================
Files 70 71 +1
Lines 7542 7825 +283
==========================================
+ Hits 7321 7601 +280
- Misses 221 224 +3 🚀 New features to boost your workflow:
|
|
Hello @yujin-kimmm we included the index on the right zenodo community four days ago. Please let us know if it looks good to you and how to move forward. |
Hi @iranroman ! Zenodo will be approved by Magdalena after she reviews it. After the index is approved, please push the edited code to this PR. As a note, we think it will be great if this loader includes a function that loads the video, so I tested some code for that. I will add the code once you finalize the multivox loader. Please let me know if you need any help. Thank you! |
This PR adds a new dataset loader for MULTIVOX, a multimodal, spatial audio–visual dataset of a-cappella vocal performances recorded in controlled conditions with both choir and vocal chamber ensembles. **Dataset Information:** - **Size**: 154 performances (≈3 hours total) - **Origin**: Recorded in two acoustically distinct spaces (auditorium and recording studio) - **Content**: A-cappella vocal performances with 6– and 16-singer configurations, including vocal warm-ups, Latin American songs, and arranged popular music - **Access**: Available at https://zenodo.org/records/17058101 - **License**: Creative Commons Attribution 4.0 International (CC-BY-4.0) **Annotations included:** - Synchronized 360° video - Far-field audio (ORTF stereo with separate left and right channel files, plus 360° camera audio) - Per-singer near-field recordings captured on personal devices - Singer metadata: facing directions, heights, genders, roles - Spatial arrangement metadata: circle layout configuration (6 or 16 positions) **Computed features:** - Angular positions for each singer (computed from circle layout and position index) - [x] Create a script in `scripts/`, `make_multivox_index.py`, which generates an index file. - [x] Run the script on the canonical version of the dataset and upload the index to [Zenodo Audio Data Loaders community](https://zenodo.org/communities/audio-data-loaders). - [x] Create a sample version of the index with the necessary information for testing. - [x] Create a module in mirdata, `mirdata/datasets/multivox.py` - [x] Create tests for your loader in `tests/datasets/test_multivox.py` - [x] Add your module to `docs/source/mirdata.rst` and `docs/source/table.rst` - [x] Run `black`, `flake8` and `mypy` (see [Running your tests locally](https://mirdata.readthedocs.io/en/stable/source/contributing.html?highlight=contributing#running-your-tests-locally)). - [x] Run `tests/test_full_dataset.py` on your dataset. (All tests passed: validation, load_tracks, index, and predetermined_splits) - [x] Check that codecov coverage does not decrease. (multivox module has 93% coverage; overall coverage will be verified in CI) To reduce friction, we will make commits on top of contributor's pull requests by default unless they use the `please-do-not-edit` flag. If you don't want this to happen don't forget to add the flag when you start your pull request.
9f6a65c to
2b342ea
Compare
|
@yujin-kimmm. @iranroman I've added the link to the approved index in this PR, addressed other comments and it is ready for review. As for the video loading, adding it in a future PR sounds like a good idea, I can also support on adding it if preferred. Please let me know how do you want to proceed. |
@jesganaud Thanks for editing the PR. I think I can just add the code on top of this commit since I was already working on it. I will let you guys know after adding it so you guys can also check if it looks good. |
|
Hello @yujin-kimmm Thanks! please push the code so that we can review it. |
|
Hi @jesganaud @iranroman , I just added the video function (including test). Please take a look and let me know what you guys think. Thank you! For the formatting that is not passing, I believe this is coming from the new version that the black just released last week. I am keeping it for now for us to easily check the multivox code first (It will push all the scripts that needs reformatting). |
|
Thanks all! should we approve the workflows to test everything included thus far? @yujin-kimmm @jesganaud |
|
Hi @yujin-kimmm, everything looks all right, just failing on the formatting check. Might I push the fix? |
|
@yujin-kimmm loader looks good, but we need to add moviepy to the ci environment and the try/except catch for users as in this loader as moviepy is an optional dependency for this loader. After that I'll merge. |
|
@magdalenafuentes ci updated with adding moviepy, try/except catch added. Now all test is passing. Ready to be merged! |
|
Great. @jesganaud @iranroman merging this loader, thanks for adding such a cool dataset! |
4 participants
This PR adds a new dataset loader for MULTIVOX, a multimodal, spatial audio–visual dataset of a-cappella vocal performances recorded in controlled conditions with both choir and vocal chamber ensembles.
Dataset Information:
Annotations included:
Computed features:
Angular positions for each singer (computed from circle layout and position index)
Create a script in
scripts/,make_multivox_index.py, which generates an index file.Run the script on the canonical version of the dataset and upload the index to Zenodo Audio Data Loaders community.
Create a sample version of the index with the necessary information for testing.
Create a module in mirdata,
mirdata/datasets/multivox.pyCreate tests for your loader in
tests/datasets/test_multivox.pyAdd your module to
docs/source/mirdata.rstanddocs/source/table.rstRun
black,flake8andmypy(see Running your tests locally).Run
tests/test_full_dataset.pyon your dataset. (All tests passed: validation, load_tracks, index, and predetermined_splits)Check that codecov coverage does not decrease. (multivox module has 93% coverage; overall coverage will be verified in CI)
To reduce friction, we will make commits on top of contributor's pull requests by default unless they use the
please-do-not-editflag. If you don't want this to happen don't forget to add the flag when you start your pull request.