Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,8 @@ There is no custom native-library loader. Since `src/OpenCvSharp/` requires net8

This repo has no `PULL_REQUEST_TEMPLATE.md`, so there's no fixed checklist to fill in — but that doesn't mean invent your own heading structure (`### Description`, `### Changes`, etc.). Look at a few recently merged PRs' bodies first and match their shape: a short plain-prose summary (no invented headings), optionally followed by a `## Test plan` checklist when the change needs one. Keep it terse — a large, sectioned write-up reads as noticeably more "AI-generated" than the plain style this repo's own history uses.

When an agent creates a pull request or issue, it must inspect the repository's existing labels and apply at least one relevant label whenever a semantically appropriate label exists. If no existing label applies, explicitly report that instead of silently leaving the item unlabeled. Whenever an agent creates a pull request in this repository, it must also assign the pull request to `@shimat`. Verify both labels and assignees after creation before reporting the item as complete.

## Issue backlog

`docs/issue-backlog.md` tracks actionable issues identified from closed/stale GitHub issues. Update the checkboxes as items are resolved.
112 changes: 112 additions & 0 deletions docs/docfx/articles/guides/feature-detection-and-matching.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Feature Detection and Matching

Local features describe visually distinctive points so that the same scene or object can be associated across images. A typical workflow detects keypoints, computes a descriptor for each point, matches descriptors, and rejects ambiguous matches.

This guide replaces the old per-algorithm Wiki examples for BRISK, ORB, FREAK, MSER, SIFT, SURF, and StarDetector. Those short examples used obsolete APIs and did not explain how detector, descriptor, and matcher choices must agree.

## Choose a feature algorithm

| Algorithm | Descriptor | Matcher norm | Typical reason to choose it |
|---|---|---|---|
| ORB | Binary | `Hamming` | Fast general-purpose baseline with no extra model files |
| BRISK | Binary | `Hamming` | Binary features with scale and rotation handling |
| AKAZE | Usually binary | `Hamming` | Nonlinear scale-space features |
| SIFT | Floating point | `L2` | More robust matching when computation cost is acceptable |

Some algorithms only detect keypoints, and some only compute descriptors. For example, MSER detects regions and FREAK computes a descriptor for supplied keypoints. Combine such components only when their keypoint and descriptor requirements are compatible.

Algorithms under `xfeatures2d`, including SURF and FREAK, require a runtime package that contains the corresponding OpenCV contrib module. Prefer a core algorithm such as ORB or SIFT unless a contrib algorithm is specifically required.

## Match two images with ORB

The following example uses two-nearest-neighbor matching and Lowe's ratio test to discard ambiguous matches:

```csharp
using OpenCvSharp;

using var image1 = Cv2.ImRead("object.jpg", ImreadModes.Grayscale);
using var image2 = Cv2.ImRead("scene.jpg", ImreadModes.Grayscale);
if (image1.Empty() || image2.Empty())
{
throw new FileNotFoundException("Could not decode one or both input images.");
}

using var orb = ORB.Create(nFeatures: 1_000);
using var descriptors1 = new Mat();
using var descriptors2 = new Mat();

orb.DetectAndCompute(image1, default, out KeyPoint[] keypoints1, descriptors1);
orb.DetectAndCompute(image2, default, out KeyPoint[] keypoints2, descriptors2);

if (descriptors1.Empty() || descriptors2.Empty())
{
throw new InvalidOperationException("No descriptors were found in one or both images.");
}

using var matcher = new BFMatcher(NormTypes.Hamming);
DMatch[][] nearestNeighbors = matcher.KnnMatch(
descriptors1,
descriptors2,
k: 2);

DMatch[] goodMatches = nearestNeighbors
.Where(matches => matches.Length == 2)
.Where(matches => matches[0].Distance < 0.75f * matches[1].Distance)
.Select(matches => matches[0])
.ToArray();

using var visualization = new Mat();
Cv2.DrawMatches(
image1,
keypoints1,
image2,
keypoints2,
goodMatches,
visualization,
flags: DrawMatchesFlags.NotDrawSinglePoints);

Cv2.ImWrite("matches.jpg", visualization);
```

The ratio `0.75` is a starting point, not a universal threshold. A lower value keeps fewer, less ambiguous matches. Evaluate it against representative data.

## Match the norm to the descriptor

The matcher compares descriptor rows. Binary descriptors such as ORB and BRISK represent bit patterns and require Hamming distance. Floating-point descriptors such as SIFT require L2 distance.

Using the wrong norm may produce poor results even though the code compiles:

```csharp
using var orbMatcher = new BFMatcher(NormTypes.Hamming);
using var siftMatcher = new BFMatcher(NormTypes.L2);
```

ORB with `WTA_K` equal to 3 or 4 requires `NormTypes.Hamming2` instead of `Hamming`.

## Matching is not geometric verification

Descriptor similarity alone does not prove that all accepted pairs belong to one object or scene transformation. For planar objects, estimate a homography with RANSAC from the matched keypoint coordinates and keep only inliers. For 3D scenes or calibrated cameras, choose a geometric model appropriate to the application.

Good production checks often include:

- A minimum number of accepted matches.
- A minimum number or ratio of geometric inliers.
- Limits on the estimated scale, orientation, or projected shape.
- Tests on images that contain no valid match.

## Reuse algorithm objects

Creating detectors and matchers for every frame adds unnecessary overhead. In a video or service pipeline, create them once, reuse them for sequential calls, and dispose them when the pipeline shuts down. Do not use one mutable native algorithm instance concurrently from multiple threads unless the underlying OpenCV API documents that use as safe.

## Related guides

- [Mat Basics](mat-basics.md)
- [Image Processing Pipeline](image-processing-pipeline.md)
- [Resource Management](resource-management.md)

## Related API

- [ORB](xref:OpenCvSharp.ORB)
- [SIFT](xref:OpenCvSharp.SIFT)
- [BFMatcher](xref:OpenCvSharp.BFMatcher)
- [Cv2.DrawMatches](xref:OpenCvSharp.Cv2.DrawMatches*)
77 changes: 77 additions & 0 deletions docs/docfx/articles/guides/file-storage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# File Storage

OpenCV `FileStorage` reads and writes matrices, scalars, sequences, and mappings in YAML, XML, or JSON. It is useful for calibration data and OpenCV-compatible model parameters. For ordinary application configuration, a .NET serializer may provide a more natural object model.

## Write values and a matrix

The file extension selects the format:

```csharp
using Mat transform = Mat.Eye(3, 3, MatType.CV_64FC1);

using var storage = new FileStorage("settings.yml", FileStorage.Modes.Write);
if (!storage.IsOpened())
{
throw new InvalidOperationException("Could not open settings.yml for writing.");
}

storage.Write("threshold", 128);
storage.Write("name", "example");
storage.Write("transform", transform);
```

Use `.xml`, `.yml` or `.yaml`, and `.json` for the corresponding formats. Append `.gz` to write or read a compressed XML or YAML file.

## Read values and a matrix

`FileStorage` indexers return `FileNode` objects that own native resources. Dispose each node after reading it:

```csharp
using var storage = new FileStorage("settings.yml", FileStorage.Modes.Read);
if (!storage.IsOpened())
{
throw new InvalidOperationException("Could not open settings.yml.");
}

using var thresholdNode = storage["threshold"]
?? throw new InvalidDataException("Missing threshold.");
using var nameNode = storage["name"]
?? throw new InvalidDataException("Missing name.");
using var transformNode = storage["transform"]
?? throw new InvalidDataException("Missing transform.");

int threshold = thresholdNode.ReadInt();
string name = nameNode.ReadString();
using Mat transform = transformNode.ReadMat();
```

The returned `Mat` owns its native data and must also be disposed.

## Work with an in-memory document

Combine `Modes.Memory` with the write or read mode:

```csharp
string yaml;
using (var writer = new FileStorage("yml", FileStorage.Modes.Write | FileStorage.Modes.Memory))
{
writer.Write("answer", 42);
yaml = writer.ReleaseAndGetString();
}

using var reader = new FileStorage(yaml, FileStorage.Modes.Read | FileStorage.Modes.Memory);
using var answerNode = reader["answer"]
?? throw new InvalidDataException("Missing answer.");
int answer = answerNode.ReadInt();
```

When writing in memory, the first constructor argument identifies the output format rather than a file path.

## Nested data

`FileStorage.Add` writes sequences and mappings using OpenCV's streaming syntax. `FileStorage.GetPath` reads a nested path while disposing intermediate `FileNode` instances. Prefer `GetPath` over a long indexer chain when reading nested data.

## Related API

- [FileStorage](xref:OpenCvSharp.FileStorage)
- [FileNode](xref:OpenCvSharp.FileNode)
146 changes: 146 additions & 0 deletions docs/docfx/articles/guides/histograms-and-contrast.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Histograms and Contrast

An image histogram counts how many pixels fall into each intensity range. Histograms are useful for diagnosing exposure, comparing image distributions, selecting thresholds, and improving contrast.

## Calculate a grayscale histogram

This example calculates 256 bins over the complete intensity range of an 8-bit grayscale image:

```csharp
using OpenCvSharp;

using var source = Cv2.ImRead("input.jpg", ImreadModes.Grayscale);
if (source.Empty())
{
throw new FileNotFoundException("Could not decode input.jpg.");
}

using var histogram = new Mat();
Cv2.CalcHist(
images: [source],
channels: [0],
mask: default,
hist: histogram,
dims: 1,
histSize: [256],
ranges: [new Rangef(0, 256)]);

float darkPixelCount = histogram.At<float>(32);
Console.WriteLine($"Pixels with intensity 32: {darkPixelCount}");
```

The result is a `CV_32FC1` matrix with one row per bin. The range end is exclusive, so `[0, 256)` covers every possible value in an 8-bit image.

## Plot the histogram

Normalize the bin counts to the plot height before drawing them:

```csharp
const int plotWidth = 512;
const int plotHeight = 300;

using var normalized = new Mat();
Cv2.Normalize(
histogram,
normalized,
alpha: 0,
beta: plotHeight - 1,
normType: NormTypes.MinMax);

using var plot = new Mat(
rows: plotHeight,
cols: plotWidth,
type: MatType.CV_8UC3,
s: Scalar.White);

int binWidth = plotWidth / 256;
for (int bin = 1; bin < 256; bin++)
{
int previousHeight = (int)Math.Round(normalized.At<float>(bin - 1));
int currentHeight = (int)Math.Round(normalized.At<float>(bin));

Cv2.Line(
plot,
new Point((bin - 1) * binWidth, plotHeight - 1 - previousHeight),
new Point(bin * binWidth, plotHeight - 1 - currentHeight),
Scalar.Black,
thickness: 2);
}

Cv2.ImWrite("histogram.png", plot);
```

Normalization here is only for display. Preserve the original histogram when its counts or proportions will be used in later calculations.

## Limit calculation to a region

`CalcHist` accepts an 8-bit single-channel mask. Nonzero mask pixels select the source pixels to include:

```csharp
using var mask = new Mat(source.Size(), MatType.CV_8UC1, Scalar.Black);
Cv2.Rectangle(
mask,
new Rect(50, 40, 200, 150),
Scalar.White,
thickness: -1);

using var regionHistogram = new Mat();
Cv2.CalcHist(
images: [source],
channels: [0],
mask: mask,
hist: regionHistogram,
dims: 1,
histSize: [256],
ranges: [new Rangef(0, 256)]);
```

The mask must have the same width and height as the source image.

## Improve grayscale contrast

Global histogram equalization redistributes intensities across the complete image:

```csharp
using var equalized = new Mat();
Cv2.EqualizeHist(source, equalized);
```

It works on an 8-bit single-channel source. It can over-amplify noise or produce unnatural results when illumination varies across the image.

Contrast Limited Adaptive Histogram Equalization (CLAHE) processes local tiles and limits amplification:

```csharp
using var clahe = Cv2.CreateCLAHE(
clipLimit: 2.0,
tileGridSize: new Size(8, 8));
using var improved = new Mat();

clahe.Apply(source, improved);
Cv2.ImWrite("improved.png", improved);
```

Treat `clipLimit` and `tileGridSize` as parameters to tune against representative images. Smaller tiles adapt to more local variation but can emphasize noise and tile boundaries.

## Color histograms

For a BGR image, channel index 0 is blue, 1 is green, and 2 is red. Separate BGR channel histograms describe the storage channels, but they are often poor measures of perceived color similarity. Convert to a color space such as HSV or Lab first when hue, saturation, or perceptual brightness is the quantity of interest.

## Common mistakes

- Match the histogram range to the source depth and the intended values; `[0, 256)` is specific to an 8-bit intensity channel.
- Do not compare raw bin counts from images or masks containing different numbers of pixels. Normalize the histograms first.
- Histogram similarity does not preserve spatial layout. Two visually different images can have the same histogram.
- Use a mask for a region of interest when the surrounding background would dominate the distribution.

## Related guides

- [Mat Basics](mat-basics.md)
- [Image Processing Pipeline](image-processing-pipeline.md)
- [Pixel Access](pixel-access.md)

## Related API

- [Cv2.CalcHist](xref:OpenCvSharp.Cv2.CalcHist*)
- [Cv2.EqualizeHist](xref:OpenCvSharp.Cv2.EqualizeHist*)
- [CLAHE](xref:OpenCvSharp.CLAHE)
Loading
Loading