Duplicate Selector

The Duplicate Selector is an operational enhancement integrated into the experimental lab, designed to identify and manage duplicate identities or visual inputs effectively.

Overview

In datasets containing multiple similar photos, or when building high-precision biometric galleries, managing visual redundancy is critical. The Duplicate Selector provides an interface to resolve identity collisions securely.

Technologies Used

• ArcFace Embeddings: Uses the 512-D vector distance to group and identify highly similar faces.
• React Frontend: Interactive component allowing operators to review "collision" sets.

Workflow

1. Threshold Filtering: The system identifies a set of images where the Cosine Similarity falls within a near-identical band (e.g., > 0.95 similarity score).
2. Review Generation: Rather than discarding data automatically, the system flags the cluster as potential duplicates.
3. UI Selection: The Duplicate Image Selector component exposes these images side-by-side to the user.
4. Resolution: The user manually dictates whether to merge these entries under one unified identity, delete the duplicate, or keep both as distinct datasets (e.g., in the case of identical twins).

Use Cases

• Gallery Cleanup: Resolving multiple uploads of the same person.
• Accuracy Training: Removing redundant reference imagery to prevent embedding bias in the system's gallery.