Documentation

Complete guide to integrating and using RareSift AI-powered video search platform

Quick Start

Get started with RareSift in minutes. Upload your first video and start searching driving scenarios.

1

Create Account

Sign up and get access to the platform

2

Upload Videos

Drag and drop your driving footage

3

Start Searching

Use natural language to find scenarios

Getting Started

Platform Overview

RareSift uses OpenCLIP embeddings to enable semantic search of driving scenarios. Upload videos, and our AI automatically extracts frames and generates searchable embeddings.

  • Support for MP4, MOV, AVI video formats
  • Frame extraction at 1 frame per second
  • 1536-dimensional CLIP embeddings
  • Natural language and image-based search

Video Upload Process

  1. Drag and drop videos or click to browse
  2. Add metadata (optional): weather, time of day, location
  3. Click upload - processing starts automatically
  4. Monitor processing status in dashboard
  5. Search becomes available once processing completes

Search Features

Natural Language Search

Describe scenarios in plain English. Our AI understands context and finds matching frames.

"cars merging onto highway"
"intersection with traffic lights"
"pedestrian crossing street"

Visual Similarity Search

Upload a reference image to find visually similar scenes across your video collection.

  • Supports PNG, JPG, WEBP formats
  • Maximum file size: 10MB
  • AI-powered visual similarity matching
  • Configurable similarity thresholds

API Integration

REST API Endpoints

Integrate RareSift into your existing workflows with our comprehensive REST API.

POST /api/v1/videos/upload
POST /api/v1/search
GET /api/v1/videos/{id}
GET /api/v1/health

Authentication

Use API keys for secure access to all endpoints:

Authorization: Bearer YOUR_API_KEY

Advanced Features

Batch Processing

Process multiple videos simultaneously and export results in various formats.

  • ZIP exports with frames and metadata
  • JSON manifests with timestamps
  • CSV exports for analysis
  • Custom filtering and sorting

Performance Optimization

  • Redis caching for faster search results
  • PostgreSQL with pgvector for similarity search
  • Background processing for large uploads
  • GPU acceleration when available

Code Examples

Search Request (Python)

import requests

url = "https://api.raresift.com/v1/search"
headers = {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
}

payload = {
    "query": "cars merging onto highway",
    "limit": 10,
    "similarity_threshold": 0.7
}

response = requests.post(url, json=payload, headers=headers)
results = response.json()

for result in results["results"]:
    print(f"Frame {result['frame_id']} - Similarity: {result['similarity']:.2f}")
    print(f"Video: {result['video_filename']}")
    print(f"Timestamp: {result['timestamp']}s")
    print("---")

Video Upload (JavaScript)

const uploadVideo = async (file) => {
  const formData = new FormData();
  formData.append('file', file);
  formData.append('metadata', JSON.stringify({
    weather: 'sunny',
    time_of_day: 'day',
    location: 'highway'
  }));

  const response = await fetch('/api/v1/videos/upload', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_KEY}`
    },
    body: formData
  });

  const result = await response.json();
  console.log('Upload successful:', result.video_id);
  
  // Monitor processing status
  const checkStatus = setInterval(async () => {
    const status = await fetch(`/api/v1/videos/${result.video_id}`);
    const video = await status.json();
    
    if (video.processing_status === 'completed') {
      clearInterval(checkStatus);
      console.log('Processing complete!');
    }
  }, 5000);
};