Skip to main content

How Visual Search Works

Visual Search finds images that look like your query image. Upload any image—even one not in your dataset—and Visual Layer returns the most visually similar media, ranked by similarity score.

​
Prerequisites

  • A dataset in READY status with at least one indexed embedding model.
  • A valid JWT token. See Authentication.
  • A dataset ID (visible in the browser URL when viewing a dataset: https://app.visual-layer.com/dataset/<dataset_id>/data).
Visual search requires the dataset to have been indexed with a visual embedding model. Datasets created through the standard upload workflow include this automatically.

​
How It Works

Visual search is a two-step process.
  1. Upload your query image to register it as a search anchor — the API returns an anchor_media_id.
  2. Pass that anchor_media_id to the Explore endpoint to retrieve ranked results.

​
Step 1: Upload a Query Image

Upload an image to use as the search reference. 
POST /api/v1/dataset/{dataset_id}/search-image-similarity
Authorization: Bearer <jwt>
Content-Type: multipart/form-data

​
Parameters

ParameterTypeRequiredDescription
dataset_idstring (UUID)YesThe dataset to search within.
entity_typestringYesIMAGES or OBJECTS.
thresholdintegerNoClustering threshold (0–4). Use 0 for the finest granularity.
filefileYesThe query image file (multipart form field).
bounding_boxstring (JSON)NoFocus search on a sub-region: {"x":0.1,"y":0.2,"width":0.5,"height":0.6} — values are fractions of image dimensions (0.0–1.0).

​
Example

curl -X POST \
  -H "Authorization: Bearer <jwt>" \
  -F "file=@/path/to/query_image.jpg" \
  "https://app.visual-layer.com/api/v1/dataset/<dataset_id>/search-image-similarity?threshold=0&entity_type=IMAGES"

​
Response

{
  "anchor_media_id": "f9d612d4-1234-11f1-bfca-fa39f6ed1f22",
  "anchor_type": "UPLOAD"
}
Save both anchor_media_id and anchor_type — you need them in Step 2.

​
Step 2: Retrieve Results

Pass the anchor values to the Explore endpoint to get ranked results. 
GET /api/v1/explore/{dataset_id}
Authorization: Bearer <jwt>

​
Parameters

ParameterTypeRequiredDescription
anchor_media_idstring (UUID)YesThe anchor_media_id returned in Step 1.
anchor_typestringYesUPLOAD when using a query image you uploaded; MEDIA when referencing an existing image in the dataset.
entity_typestringYesIMAGES or OBJECTS.
thresholdintegerNoClustering threshold (0–4). Must match the value used in Step 1.
page_numberintegerNoPage index for pagination (0-based). Results are paginated at 100 clusters per page.

​
Example

curl -H "Authorization: Bearer <jwt>" \
  "https://app.visual-layer.com/api/v1/explore/<dataset_id>?threshold=0&entity_type=IMAGES&page_number=0&anchor_media_id=f9d612d4-1234-11f1-bfca-fa39f6ed1f22&anchor_type=UPLOAD"

​
Response

{
  "clusters": [
    {
      "cluster_id": "39df7adc-16b7-406e-ac34-e4a24476bbf6",
      "type": "IMAGES",
      "n_images": 7,
      "n_objects": 0,
      "n_videos": 0,
      "similarity_threshold": "0",
      "relevance_score": 0.13,
      "relevance_score_type": "cosine_distance",
      "previews": [
        {
          "type": "IMAGE",
          "media_id": "300dad2c-1234-11f1-8483-5a879df30de4",
          "media_uri": "https://cdn.example.com/.../image.jpg",
          "media_thumb_uri": "https://cdn.example.com/.../thumb.webp",
          "caption": null,
          "file_name": "00046.jpg",
          "bounding_box": null,
          "relevance_score": 0.13,
          "relevance_score_type": "cosine_distance",
          "width": 786,
          "height": 492
        }
      ],
      "labels": null,
      "user_tags": null,
      "captions": null
    }
  ],
  "metadata": {
    "used_duckdb": true
  }
}

​
Understanding relevance_score

When relevance_score_type is cosine_distance, a lower score means more similar to your query image.
  • 0.0 — identical
  • ~0.1–0.3 — highly similar
  • ~0.5+ — loosely related

​
Search by Region (Bounding Box)

Focus the search on a specific area of the query image using bounding_box. Values are fractions of the image dimensions (0.0–1.0). 
curl -X POST \
  -H "Authorization: Bearer <jwt>" \
  -F "file=@/path/to/image.jpg" \
  "https://app.visual-layer.com/api/v1/dataset/<dataset_id>/search-image-similarity?threshold=0&entity_type=IMAGES&bounding_box=%7B%22x%22%3A0.1%2C%22y%22%3A0.2%2C%22width%22%3A0.5%2C%22height%22%3A0.6%7D"

​
Search by Existing Media ID

To find images similar to one already in your dataset, skip Step 1 and use anchor_type=MEDIA directly. The media_id is returned in the media_id field of any Explore endpoint response — for example, from a previous visual or semantic search result. 
curl -H "Authorization: Bearer <jwt>" \
  "https://app.visual-layer.com/api/v1/explore/<dataset_id>?threshold=0&entity_type=IMAGES&page_number=0&anchor_media_id=<media_id>&anchor_type=MEDIA"

​
Python Example

The following example runs a full visual search workflow. 
import requests
import time

VL_BASE_URL = "https://app.visual-layer.com"
JWT_TOKEN = "<your-jwt-token>"
DATASET_ID = "<your-dataset-id>"
QUERY_IMAGE_PATH = "/path/to/query_image.jpg"

headers = {"Authorization": f"Bearer {JWT_TOKEN}"}

# Step 1: Upload query image
with open(QUERY_IMAGE_PATH, "rb") as f:
    resp = requests.post(
        f"{VL_BASE_URL}/api/v1/dataset/{DATASET_ID}/search-image-similarity",
        headers=headers,
        params={"threshold": 0, "entity_type": "IMAGES"},
        files={"file": f},
    )
resp.raise_for_status()
anchor = resp.json()
anchor_media_id = anchor["anchor_media_id"]
anchor_type = anchor["anchor_type"]

# Step 2: Fetch results
resp = requests.get(
    f"{VL_BASE_URL}/api/v1/explore/{DATASET_ID}",
    headers=headers,
    params={
        "threshold": 0,
        "entity_type": "IMAGES",
        "page_number": 0,
        "anchor_media_id": anchor_media_id,
        "anchor_type": anchor_type,
    },
)
resp.raise_for_status()
results = resp.json()

clusters = results.get("clusters", [])
print(f"Found {len(clusters)} similar clusters")

for cluster in clusters:
    score = cluster.get("relevance_score")
    n = cluster.get("n_images")
    cid = cluster.get("cluster_id")
    print(f"  Cluster {cid[:8]}... — {n} images, similarity score: {score:.3f}")
    for preview in cluster.get("previews", [])[:3]:
        print(f"    {preview['file_name']} ({preview['relevance_score']:.3f})")

​
Response Codes

See Error Handling for the error response format and Python handling patterns.
HTTP CodeStatusDescription
200OKUpload successful, anchor returned.
202AcceptedRequest accepted for processing.
400Bad RequestMissing or invalid parameters. Check entity_type and that a file was provided.
401UnauthorizedInvalid or expired JWT token.
404Not FoundDataset not found or not accessible with your credentials.
409ConflictDataset status is not READY.
500Internal Server ErrorServer-side error.

Semantic Search

Search using natural language text queries.

Explore API

Browse and filter all dataset media.