Skip to content

New Tool: Extract frames with CV2#207

Merged
kostrykin merged 22 commits into
BMCV:masterfrom
rmassei:tool-cv2-extractframes
Mar 18, 2026
Merged

New Tool: Extract frames with CV2#207
kostrykin merged 22 commits into
BMCV:masterfrom
rmassei:tool-cv2-extractframes

Conversation

@rmassei

@rmassei rmassei commented Feb 25, 2026

Copy link
Copy Markdown
Contributor

This PR introduce a new imaging tool to extract frames from video files. At the present, it was tested with MP4 and AVI but it might work with any other video formats supported by OpenCV.

How does it works:
The tool accepts a video file as input source. Users can specify a time range (in seconds); every frame that falls within this interval is extracted and saved as an individual TIFF image.The tool also offers an optional grayscale conversion.

Screenshot from 2026-02-24 17-42-34

Maybe to check:

I am not 100% sure if the test was set up correctly (although it passes)

https://github.com/rmassei/galaxy-image-analysis/blob/adf8b716e3d00cdc0b6572bd9ddf84fdb339559e/tools/extract_frames/cv2_extract_frames.xml#L47

            <expand macro="tests/intensity_image_diff/element" name="frame_00001.tiff" value="frame_00001.tiff" ftype="tiff"/>

Does it make sense that the name and value fields have the same value?


Check-list for the contributor

Please make sure you have read the CONTRIBUTING.md document (last updated: 2025/12/22).

Mandatory checks:

  • License permits unrestricted use (educational + commercial).
  • I agree to license these and all my past contributions to the Galaxy Image Analysis codebase under the MIT license.

If this PR adds or updates a tool or tool collection:

  • This PR adds a new tool or tool collection.
  • This PR updates an existing tool or tool collection.
  • Tools added/updated by this PR comply with the Guidelines below (or explain why they do not).

Guidelines for the contributor

This section is cited from the Naming and Annotation Conventions for Tools in the Image Community in Galaxy.

Naming

Generally, the name of Galaxy tools in our community should be expressive and concise, while stating the purpose of the tool as precisely as possible. Consistency of the namings of Galaxy tools is important to ensure they can be found easily. To maintain consistency, we consider phrasing names as imperatives a good practice, such as "Analyze particles" or "Perform segmentation using watershed transformation". An acknowledged exception from this rule is the names of tool wrappers of major tool suites, where the name of a tool wrapper should be chosen identically to the module or function of the tool which is wrapped (e.g., "MaskImage" in CellProfiler).

Tool description

If a Galaxy tool is a thin tool wrapper (e.g, part of a major tool suite), then the name of the wrapped tool (and only the name of the wrapped tool, subsequent to the term "with" as in "with Bioformats") should be used as the description of the tool (further examples include "with CellProfiler", "with ImageJ2", "with ImageMagick", "with SpyBOAT", "with SuperDSM"). This ensures that the tool is found by typing the name of the wrapped tool into the "Search" field on the Galaxy interface. The tool description should be empty if a tool is either not part of a major tool suite, or the main functionality of the tool is implemented in the wrapper.

Annotations

We point out that there is a global list of precedential annotations with Bio.tools identifiers (Ison et al., 2019) in Galaxy (see mappings), which may outweigh the annotations made in the XML specification of a Galaxy tool (and thus the annotations of a tool reported within the web interface of Galaxy might be divergent). However, since the precedential annotations are subject to possible changes and to avoid redundant work, we do not aim to reflect those in our specifications (those which we make in the XML specifications of Galaxy tools).

@kostrykin

Copy link
Copy Markdown
Member

Very cool, thanks @rmassei!

I will try to look into it ASAP.

@kostrykin kostrykin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, thanks @rmassei!

A few comments inside.

If you have the time and patience, I'd appreciate if you could make this a tool suite for cv2 instead (with only a single tool included for now, but the possibility to extend in the future). I think this would make sense, given how large cv2 is. But again, I'm also find with merging this as a standalone tool (as is).

Comment thread tools/extract_frames/.shed.yml Outdated
Comment thread tools/extract_frames/.shed.yml Outdated
Comment thread tools/extract_frames/cv2_extract_frames.xml Outdated
Comment thread tools/extract_frames/cv2_extract_frames.xml Outdated
Comment thread tools/extract_frames/cv2_extract_frames.xml Outdated
Comment thread tools/extract_frames/cv2_extract_frames.xml Outdated
Comment thread tools/extract_frames/cv2_extract_frames.xml Outdated
Comment thread tools/extract_frames/cv2_extract_frames.xml Outdated
Comment thread tools/extract_frames/cv2_extract_frames.xml Outdated
Comment thread tools/extract_frames/cv2_extract_frames.xml Outdated
@kostrykin kostrykin self-assigned this Feb 27, 2026
Comment thread tools/extract_frames/.shed.yml Outdated
@rmassei rmassei requested a review from kostrykin March 9, 2026 14:11
@rmassei

rmassei commented Mar 9, 2026

Copy link
Copy Markdown
Contributor Author

@kostrykin what do you think about:

   **Codecs Support:** 
    The tool was tested with the following video codecs:
    - H264
    - H265 (HEVC)
    The tool may support other codecs, though this has not been tested yet. 
    If you have successfully converted files using alternative video formats or codecs, we would be grateful for your feedback at https://github.com/BMCV/galaxy-image-analysis.

for the help codec section?

Comment thread tools/cv2/.shed.yml Outdated
Comment thread tools/cv2/cv2_extract_frames.xml Outdated
Comment thread tools/cv2/cv2_extract_frames.xml Outdated
Comment thread tools/cv2/cv2_extract_frames.xml

@kostrykin kostrykin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more comments :)

could you maybe add a small test for end_frame<0?

Comment thread tools/cv2/cv2_extract_frames.py Outdated
Comment thread tools/cv2/cv2_extract_frames.py Outdated
Comment thread tools/cv2/cv2_extract_frames.py Outdated
Comment thread tools/cv2/cv2_extract_frames.xml Outdated
Co-authored-by: Leonid Kostrykin <void@evoid.de>

@kostrykin kostrykin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I think I had messed up a bit with my previous suggestions, this should hopefully fix it. Also, it would be very nice to have a test for end_time<0 and end_time=0 since these are different code branches.

Comment thread tools/cv2/cv2_extract_frames.py Outdated
Comment thread tools/cv2/cv2_extract_frames.xml Outdated
@kostrykin

Copy link
Copy Markdown
Member

rmassei#1 @rmassei what you think?

@rmassei

rmassei commented Mar 12, 2026

Copy link
Copy Markdown
Contributor Author

Yes, I just need to add tests for the different end_times!

Comment thread tools/cv2/cv2_extract_frames.xml Outdated
@rmassei rmassei requested a review from kostrykin March 16, 2026 21:03
<param name="start_time" value="0"/>
<param name="end_time" value="0"/>
<param name="convert_to_grey" value="True"/>
<output_collection name="frames" type="list" count="1"/>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool that this works!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep that's pretty neat!

@kostrykin

Copy link
Copy Markdown
Member

Does it make sense that the name and value fields have the same value?

It can make sense.

The thing is, as you can see from the definition of the macro,

<element name="@NAME@" value="@VALUE@" ftype="@FTYPE@" compare="image_diff" metric="@METRIC@" eps="@EPS@">

the attributes name and value are propagated to <element> which is basically documented here, although it is not explicitly mentioned in the docs: https://docs.galaxyproject.org/en/latest/dev/schema.html#id122

file | If specified, this value is the name of the output file stored in the target test-data directory which will be used to compare the results of executing the tool via the functional test framework.

and:

value | An alias for file.

What is not really clear from the docs is what the name attribute of <element> is. From my experience, I'm quite sure that this must match the name of the file that the dataset is discovered from.

Here is an example that might disambiguate this a bit:

  1. The tool writes files numbered like output/1.tiff, output/2.tiff, and so on:
    decimals = math.ceil(math.log10(1 + num_images))
    self.output_filepath_pattern = str(dir_path / f'%0{decimals}d.tiff')
  2. Within the tests, the names of the files are re-used to match the corresponding images from test-data:
    <output_collection name="output" type="list" count="3">
    <expand macro="tests/intensity_image_diff/element" name="1.tiff" value="rgb1_r.tiff" ftype="tiff"/>
    <expand macro="tests/intensity_image_diff/element" name="2.tiff" value="rgb1_g.tiff" ftype="tiff"/>
    <expand macro="tests/intensity_image_diff/element" name="3.tiff" value="rgb1_b.tiff" ftype="tiff"/>
    </output_collection>

I suppose this is because:

The test definition may omit any number of elements from a collection, but the ones that are specified will be checked against the actual resulting collection from the tool run and the order within the collection verified.

https://docs.galaxyproject.org/en/latest/dev/schema.html#tool-tests-test-output-collection

Comment thread tools/cv2/cv2_extract_frames.xml Outdated
<![CDATA[
mkdir ./output_frames &&

python "$__tool_directory__/cv2_extract_frames.py" output_frames

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use single-quotes

Comment thread tools/cv2/cv2_extract_frames.xml Outdated
mkdir ./output_frames &&

python "$__tool_directory__/cv2_extract_frames.py" output_frames
-v $video_path

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
-v $video_path
-v '$video_path'

Comment thread tools/cv2/cv2_extract_frames.xml
@rmassei rmassei requested a review from bgruening March 17, 2026 15:32
@kostrykin

Copy link
Copy Markdown
Member

ready to merge? @rmassei

@rmassei

rmassei commented Mar 18, 2026

Copy link
Copy Markdown
Contributor Author

Looks fine to me!

@kostrykin kostrykin merged commit 15c1352 into BMCV:master Mar 18, 2026
12 checks passed
@kostrykin

Copy link
Copy Markdown
Member

Thanks!

@rmassei

rmassei commented Mar 18, 2026

Copy link
Copy Markdown
Contributor Author

Thanks to you for all the help and, as usual, patience @kostrykin @bgruening

@kostrykin

Copy link
Copy Markdown
Member

@rmassei Can you please also make a PR for https://github.com/usegalaxy-eu/usegalaxy-eu-tools to add the tool to Galaxy EU?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants