WHAT IS IT?
Falcon-Perception is a PyTorch inference engine for multimodal vision-language models. It runs computer vision tasks - object detection, instance segmentation, OCR - driven by natural language queries. The goal: a minimal, readable yet genuinely performant implementation of an autoregressive Transformer that understands images and text together.
WHY IS IT INTERESTING?
- Paged KV cache: efficient GPU memory use with continuous batching, no custom attention implementation needed.
- CUDA graph capture: eliminates kernel launch overhead for faster inference.
- Text-driven vision: detection and segmentation from a plain natural language query, plus OCR in raw or layout-aware modes.
- MLX backend: native Apple Silicon support alongside CUDA, so it runs on Macs too.
- Production-ready: multi-GPU REST API server, preemption under memory pressure, background tokenization via a CPU thread pool.
- Readable code: built to be studied and modified, not an over-optimized black box.
USE CASES
- Extract structured text from scanned documents or screenshots with layout-aware OCR.
- Detect and segment objects in an image just by describing what you're looking for.
- Serve a vision-language model in production behind a multi-GPU REST API.
- Prototype quickly through the bundled Streamlit web interface.
- Benchmark visual perception models across multiple samples.