CLIP Appearance Search

CLIP Appearance Search lets operators find objects and people across cameras by visual similarity rather than text labels. Upload a reference image or select a detection thumbnail, and Meridian VMS searches all cameras for visually similar appearances using CLIP neural network embeddings.

How It Works

The search pipeline uses a CLIP (Contrastive Language-Image Pretraining) model to create vector embeddings of detection thumbnails. These embeddings capture visual characteristics (color, shape, clothing, vehicle type) so that similar-looking objects can be found by visual similarity.

Embedding Pipeline

Detection Event
  --> Crop thumbnail (object bounding box)
  --> CLIP image encoder
  --> 512-dimensional embedding vector
  --> Store in vector index

Every detection event processed by the AI detection engine automatically generates a CLIP embedding from the object crop. These embeddings are stored alongside the event metadata and indexed for fast similarity search.

Search Process

Query Image
  --> CLIP image encoder
  --> 512-dimensional query vector
  --> Cosine similarity search against stored embeddings
  --> Ranked results by similarity score

The search computes cosine similarity between the query embedding and all stored embeddings, returning results ranked by visual similarity.

Search Interface

Access CLIP search from Operations > Appearance Search.

Search by Image Upload

1. Click Upload Image or drag and drop a reference photo.
2. The system encodes the image and searches across all cameras.
3. Results appear ranked by similarity with confidence scores.

Supported formats: JPEG, PNG. The image is cropped and resized automatically for optimal encoding.

Search by Detection Event

1. On the Events page, click any detection thumbnail.
2. Select Find Similar from the context menu.
3. The system uses that detection’s existing embedding as the query.

This is the fastest search method since no new encoding is needed.

Search by Text Description

CLIP also supports text-to-image search. Type a natural language description and the system finds visually matching detections:

• “person wearing red jacket”
• “white delivery van”
• “person carrying a backpack”

Text search uses CLIP’s text encoder to create a query vector in the same embedding space as image embeddings.

Search Filters

Narrow results using these filters:

Filter	Description
Cameras	Restrict search to specific cameras or locations
Date range	Limit to a specific time period
Object class	Only search within a class (person, car, truck, etc.)
Min similarity	Minimum cosine similarity threshold (default: 0.70)
Max results	Number of results to return (default: 50)

Results View

Search results display as a grid of thumbnails sorted by similarity score:

Column	Description
Thumbnail	Detection crop with bounding box
Similarity	Cosine similarity score (0.0 - 1.0)
Camera	Source camera name
Timestamp	When the detection occurred
Class	Detected object class
Zone	Detection zone (if applicable)

Result Actions

For each result:

• View Full Frame — See the complete camera frame at the time of detection
• Jump to Playback — Open recorded playback at the exact timestamp
• Find Similar — Use this result as a new search query (iterative refinement)
• Track Across Cameras — Show all appearances of this object across cameras, sorted by time

Cross-Camera Tracking

When investigating a specific person or vehicle, use the Track Across Cameras action to build a timeline of appearances:

1. Select a detection of the target.
2. Click Track Across Cameras.
3. The system searches all cameras for similar appearances within a configurable time window.
4. Results are displayed on a timeline showing the target’s movement across camera views.

This is useful for tracing a person’s path through a facility or tracking a vehicle across multiple sites.

Embedding Storage

CLIP embeddings are stored alongside detection events in the database. Each embedding is approximately 2 KB:

Scale	Storage Overhead
1,000 events/day	~2 MB/day
10,000 events/day	~20 MB/day
100,000 events/day	~200 MB/day

Embeddings are deleted when their associated detection events are removed by retention cleanup.

Performance

Operation	Typical Latency
Image encoding	50-100ms (GPU), 200-500ms (CPU)
Similarity search (100K embeddings)	20-50ms
Similarity search (1M embeddings)	100-300ms
Text encoding	10-30ms

The CLIP model requires approximately 400 MB of VRAM when running on GPU. On CPU, it uses approximately 600 MB of RAM.

Configuration

Configure CLIP search from Settings > AI Detection > Appearance Search:

Setting	Default	Description
Enable Embeddings	On (if GPU available)	Generate CLIP embeddings for new detections
Model	Default	CLIP model variant
Batch Size	16	Number of crops to encode per batch
Default Similarity	0.70	Default minimum similarity threshold for searches
Max Search Window	7 days	Maximum date range for a single search query

API Reference

Method	Endpoint	Description
POST	/api/clip/search	Search by uploaded image or text query
POST	/api/clip/search/event/{id}	Search using an existing event’s embedding
GET	/api/clip/track/{event_id}	Cross-camera tracking for a detection
GET	/api/clip/stats	Embedding index statistics