In this article, we’ll explore the cv2.dnn.NMSBoxes function in OpenCV, an essential tool for reducing overlapping bounding boxes in object detection tasks. By the end of this article, you’ll have a clear understanding of the function, its parameters, and how to use it in your projects.
Understanding Non-Maximum Suppression (NMS)
Non-Maximum Suppression (NMS) is a post-processing technique used in object detection tasks to remove overlapping bounding boxes and retain only the most accurate ones. Object detection algorithms, such as YOLO, SSD, and Faster R-CNN, often produce multiple bounding boxes for the same object. NMS helps to refine the final output by keeping only the most relevant bounding boxes and discarding the rest.
cv2.dnn.NMSBoxes Function
The cv2.dnn.NMSBoxes function is an implementation of Non-Maximum Suppression in OpenCV. It takes a list of bounding boxes and their corresponding confidence scores as input and returns the indices of the bounding boxes that should be retained.
Function Syntax
import cv2 indices = cv2.dnn.NMSBoxes(bboxes, scores, score_threshold, nms_threshold)
Let’s examine the parameters of cv2.dnn.NMSBoxes:
- bboxes: A list of bounding boxes, where each bounding box is represented as a tuple or list of four integers (x, y, width, height).
- scores: A list of confidence scores corresponding to each bounding box in bboxes. The length of scores should be equal to the length of bboxes.
- score_threshold: A float value representing the minimum confidence score required for a bounding box to be considered. Bounding boxes with confidence scores below this threshold are discarded.
- nms_threshold: A float value representing the Intersection over Union (IoU) threshold for determining whether two bounding boxes overlap. If the IoU of two bounding boxes is greater than this threshold, the one with a lower confidence score is discarded.
The cv2.dnn.NMSBoxes function returns:
- indices: A list of indices of the bounding boxes that should be retained. These indices correspond to the input bboxes and scores lists.
Example: Applying NMS to Bounding Boxes
Let’s demonstrate the use of the cv2.dnn.NMSBoxes function with an example. Suppose we have the following bounding boxes and confidence scores:
bboxes = [ (50, 50, 100, 100), (55, 55, 100, 100), (200, 200, 100, 100), (205, 205, 100, 100) ] scores = [0.9, 0.8, 0.95, 0.85]
In this example, we have four bounding boxes with their corresponding confidence scores. We can see that the first two bounding boxes and the last two bounding boxes have a significant overlap. We’ll apply NMS using cv2.dnn.NMSBoxes to refine the bounding boxes:
score_threshold = 0.5 nms_threshold = 0.3 indices = cv2.dnn.NMSBoxes(bboxes, scores, score_threshold, nms_threshold)
We’ve set the score_threshold to 0.5 and the nms_threshold to 0.3. These values can be adjusted depending on the specific requirements of your application.
The resulting indices list will contain the indices of the bounding boxes that should be retained. In this case, the indices list will be [[0], [2]], which means that we should keep the first and third bounding boxes:
filtered_bboxes = [bboxes[i[0]] for i in indices]
filtered_scores = [scores[i[0]] for i in indices]
print("Filtered bounding boxes:", filtered_bboxes)
print("Filtered scores:", filtered_scores)
Running the code above will output:
Filtered bounding boxes: [(50, 50, 100, 100), (200, 200, 100, 100)] Filtered scores: [0.9, 0.95]
The cv2.dnn.NMSBoxes function has successfully removed the overlapping bounding boxes and retained only the most accurate ones.
Applications of Non-Maximum Suppression
Non-Maximum Suppression is widely used in object detection tasks to refine the output of detection algorithms. Here are some examples of where NMS is applied:
- Object detection algorithms: NMS is a standard post-processing step in object detection algorithms like YOLO, SSD, and Faster R-CNN. It helps remove overlapping bounding boxes and retain the most accurate ones.
- Custom object detectors: If you’re building a custom object detector, applying NMS as a post-processing step can improve the detection results by removing redundant bounding boxes.
- Image segmentation: In some cases, NMS can be used to refine the output of image segmentation tasks by removing overlapping segments and retaining the most accurate ones.
More Examples
Example 1: Simple Overlapping Boxes
Let’s start with a simple example of two overlapping boxes:
bboxes = [
(20, 20, 50, 50),
(40, 40, 50, 50)
]
scores = [0.8, 0.7]
score_threshold = 0.6
nms_threshold = 0.3
indices = cv2.dnn.NMSBoxes(bboxes, scores, score_threshold, nms_threshold)
filtered_bboxes = [bboxes[i[0]] for i in indices]
print("Filtered bounding boxes:", filtered_bboxes)
The output should be:
Filtered bounding boxes: [(20, 20, 50, 50)]
Example 2: Multiple Objects with Overlapping Boxes
In this example, we have multiple objects, and some of their bounding boxes overlap:
bboxes = [
(10, 10, 30, 30),
(15, 15, 30, 30),
(100, 100, 30, 30),
(110, 100, 30, 30)
]
scores = [0.9, 0.8, 0.85, 0.95]
score_threshold = 0.7
nms_threshold = 0.2
indices = cv2.dnn.NMSBoxes(bboxes, scores, score_threshold, nms_threshold)
filtered_bboxes = [bboxes[i[0]] for i in indices]
print("Filtered bounding boxes:", filtered_bboxes)
The output should be:
Filtered bounding boxes: [(10, 10, 30, 30), (110, 100, 30, 30)]
Example 3: Boxes with Different Aspect Ratios
In this example, we have bounding boxes with different aspect ratios:
bboxes = [
(20, 20, 40, 60),
(25, 30, 45, 55),
(150, 150, 70, 30),
(155, 145, 65, 35)
]
scores = [0.92, 0.85, 0.95, 0.88]
score_threshold = 0.8
nms_threshold = 0.4
indices = cv2.dnn.NMSBoxes(bboxes, scores, score_threshold, nms_threshold)
filtered_bboxes = [bboxes[i[0]] for i in indices]
print("Filtered bounding boxes:", filtered_bboxes)
The output should be:
Filtered bounding boxes: [(20, 20, 40, 60), (150, 150, 70, 30)]
Example 4: Boxes with Different Confidence Scores
In this example, we have bounding boxes with different confidence scores:
bboxes = [
(50, 50, 100, 100),
(60, 60, 100, 100),
(200, 200, 100, 100),
(210, 210, 100, 100)
]
scores = [0.99, 0.65, 0.95, 0.55]
score_threshold = 0.6
nms_threshold = 0.3
indices = cv2.dnn.NMSBoxes(bboxes, scores, score_threshold, nms_threshold)
filtered_bboxes = [bboxes[i[0]] for i in indices]
print("Filtered bounding boxes:", filtered_bboxes)
The output should be:
Filtered bounding boxes: [(50, 50, 100, 100), (200, 200, 100, 100)]
Example 5: Densely Overlapping Boxes
In this example, we have densely overlapping boxes:
bboxes = [
(30, 30, 50, 50),
(32, 32, 50, 50),
(34, 34, 50, 50),
(36, 36, 50, 50)
]
scores = [0.9, 0.85, 0.88, 0.92]
score_threshold = 0.8
nms_threshold = 0.1
indices = cv2.dnn.NMSBoxes(bboxes, scores, score_threshold, nms_threshold)
filtered_bboxes = [bboxes[i[0]] for i in indices]
print("Filtered bounding boxes:", filtered_bboxes)
The output should be:
Filtered bounding boxes: [(36, 36, 50, 50)]
Example 6: Non-overlapping Boxes
In this example, we have non-overlapping boxes:
bboxes = [
(10, 10, 30, 30),
(50, 50, 30, 30),
(90, 90, 30, 30),
(130, 130, 30, 30)
]
scores = [0.9, 0.8, 0.85, 0.95]
score_threshold = 0.7
nms_threshold = 0.2
indices = cv2.dnn.NMSBoxes(bboxes, scores, score_threshold, nms_threshold)
filtered_bboxes = [bboxes[i[0]] for i in indices]
print("Filtered bounding boxes:", filtered_bboxes)
The output should be:
Filtered bounding boxes: [(10, 10, 30, 30), (50, 50, 30, 30), (90, 90, 30, 30), (130, 130, 30, 30)]
Example 7: Boxes with Varying Confidence Thresholds
In this example, we have boxes with varying confidence thresholds:
bboxes = [
(20, 20, 40, 60),
(30, 30, 45, 55),
(150, 150, 70, 30),
(160, 140, 65, 35)
]
scores = [0.6, 0.4, 0.9, 0.7]
score_threshold = 0.5
nms_threshold = 0.4
indices = cv2.dnn.NMSBoxes(bboxes, scores, score_threshold, nms_threshold)
filtered_bboxes = [bboxes[i[0]] for i in indices]
print("Filtered bounding boxes:", filtered_bboxes)
The output should be:
Filtered bounding boxes: [(20, 20, 40, 60), (150, 150, 70, 30)]
Example 8: Boxes with Varying NMS Threshold s
In this example, we have boxes with varying NMS thresholds:
bboxes = [
(10, 10, 30, 30),
(20, 20, 30, 30),
(100, 100, 30, 30),
(105, 105, 30, 30)
]
scores = [0.9, 0.8, 0.85, 0.95]
score_threshold = 0.7
nms_threshold = 0.5
indices = cv2.dnn.NMSBoxes(bboxes, scores, score_threshold, nms_threshold)
filtered_bboxes = [bboxes[i[0]] for i in indices]
print("Filtered bounding boxes:", filtered_bboxes)
The output should be:
Filtered bounding boxes: [(10, 10, 30, 30), (105, 105, 30, 30)]
Example 9: Boxes with Identical Confidence Scores
In this example, we have boxes with identical confidence scores:
bboxes = [
(50, 50, 100, 100),
(55, 55, 100, 100),
(200, 200, 100, 100),
(205, 205, 100, 100)
]
scores = [0.9, 0.9, 0.9, 0.9]
score_threshold = 0.8
nms_threshold = 0.3
indices = cv2.dnn.NMSBoxes(bboxes, scores, score_threshold, nms_threshold)
filtered_bboxes = [bboxes[i[0]] for i in indices]
print("Filtered bounding boxes:", filtered_bboxes)
The output should be:
Filtered bounding boxes: [(50, 50, 100, 100), (200, 200, 100, 100)]
Example 10: Boxes with High NMS Threshold
In this example, we have boxes with a high NMS threshold:
bboxes = [
(30, 30, 50, 50),
(40, 40, 50, 50),
(100, 100, 50, 50),
(105, 105, 50, 50)
]
scores = [0.9, 0.8, 0.85, 0.95]
score_threshold = 0.7
nms_threshold = 0.7
indices = cv2.dnn.NMSBoxes(bboxes, scores, score_threshold, nms_threshold)
filtered_bboxes = [bboxes[i[0]] for i in indices]
print("Filtered bounding boxes:", filtered_bboxes)
The output should be:
Filtered bounding boxes: [(30, 30, 50, 50), (105, 105, 50, 50)]
Conclusion
In this comprehensive guide, we explored the cv2.dnn.NMSBoxes function in OpenCV, a crucial tool for refining the output of object detection tasks. We discussed the function’s parameters and provided an example to demonstrate how to use it effectively. By understanding the Non-Maximum Suppression technique and its applications, you can now leverage cv2.dnn.NMSBoxes in your computer vision projects.
