Diagnosing Aerial-View Detectors using Generative Models

“We use foundation image generators as diagnostic instruments: synthesize attribute-controlled aerial scenes, find where detectors fail, then fix them with a small, targeted dose of real data.”

Abstract

Overview — We leverage foundation generative models to synthesize diverse, attribute-controlled aerial-view image datasets, thereby enabling systematic diagnosis of the weaknesses of aerial-view vehicle detectors.

Recent advances in large-scale image generative models enable photorealistic scene synthesis with controllable attributes, yet their potential as diagnostic tools for trained vision systems remains unexplored in aerial and remote sensing. We introduce a synthetic diagnostic framework for aerial-view vehicle detection that combines text-guided generation, attribute-controlled editing, and automated attribute verification to build a controllable synthetic testbed. This enables fine-grained evaluation of pretrained detectors across scene types and environmental conditions that are hard to isolate in real datasets. Across three detection architectures and three real aerial datasets, synthetic scene-wise performance trends closely match real-world weaknesses. Guided by these diagnostics, targeted supplementation with small real datasets from the identified weak categories yields improvements of up to 11% AP50 while requiring substantially fewer samples than non-targeted augmentation. The framework is modular and can incorporate alternative generative or vision-language models as capabilities evolve.

Contributions

A synthetic, attribute-controlled diagnostic framework for analyzing aerial-view object detectors with foundation generative models.
Evidence that synthetic scene-wise evaluation reliably reflects real-world weaknesses across multiple detectors (YOLOv8, Faster R-CNN, ViTDet) and datasets (DOTAv2, LINZ, UGRC).
A targeted data-supplementation strategy guided by these diagnostics, reaching up to 11% AP50 gain with an order of magnitude less added data.

Method

Taxonomy

Our pipeline starts from an attribute taxonomy $\mathcal{T}$ spanning three environmental axes (scene type, season, weather) and three object-level axes (vehicle count, type, color).

Image generation and editing

We uniformly sample attribute tuples, an LLM composes nadir-view generation prompts, and a text-to-image model (Imagen 3) synthesizes the scenes. A multimodal LLM re-reads each image to verify attributes, and an embedding-based validation step reconciles predicted attributes with the taxonomy. A second image-editing stage rebalances under-represented attributes via image-and-text-to-image edits.

Method Diagram — Overview of the synthetic data generation pipeline. **Left:** Initial image generation using a *text-to-image* foundation model $f_\text{G}$ conditioned on prompts $p_i^{(\text{G})}$ derived from uniformly sampled attribute values in the taxonomy $\mathcal{T}$ . **Right:** Image editing via *image-and-text-to-image* generation to correct distributional biases and enrich underrepresented attribute values.

Annotations

Bounding boxes are produced by an ensemble of zero-shot VLMs (Gemini 2.5 Flash Lite, Moondream 2) and refined with a fast human-in-the-loop pass.

Diagnosis

Detectors are then evaluated per attribute value to expose scene-conditioned failure modes.

Attribute-defined Synthetic Diagnostic Dataset

Synthetic Dataset Showcase — A showcase of synthetic aerial view imagery generated by *Imagen 3*. Each image represents a combination of a different environment and a season. We use these data to identify inherent weaknesses in object detectors.

Attribute-based Image Editing Examples

Image with vehicles — Vehicles presence change: Vehicles → No vehicles

Image with vehicles removed — Vehicles presence change: Vehicles → No vehicles

Image with no vehicles — Vehicles presence change: No vehicles → Vehicles

Image with added vehicles — Vehicles presence change: No vehicles → Vehicles

Sunny winter image — Weather change: Sunny → Snowy

Snowy winter image — Weather change: Sunny → Snowy

Fall image — Season change: Fall → Summer

Summer image — Season change: Fall → Summer

Key Results

Synthetic diagnostics match real weaknesses

Model Diagnosis — Scene-wise $\text{AP}^{50}$ of three detectors trained on real aerial datasets (*DOTAv2*, *LINZ*, *UGRC*) and evaluated on the synthetic diagnostic set. Rows denote models; columns denote training datasets. Orange arrows highlight scene types below the overall AP (gray dashed line). Best viewed in color.

Across all nine detector–dataset combinations, urban scenes are flagged as problematic every time, industrial in seven, and desert in two — a consistent, model-agnostic failure signature.

Targeted supplementation works

Adding small, scene-specific real datasets (urban, industrial, desert) to the weak categories improves dataset-level AP50 in eight of nine cases, by up to 11.3%, despite the supplements being an order of magnitude smaller than the original training sets.

Datasets

Below we provide image count information about the synthetic diagnostic and the three supplementary real datasets we used in our paper.

Dataset Name	Type	Train Split	Test Split	Total
Imagen 3	Synthetic	–	5,453	5,453
Urban (Miami)	Real	2,284	–	2,284
Industrial (LA)	Real	2,000	–	2,000
Desert (Phoenix)	Real	2,000	–	2,000

Synthetic Diagnostic Aerial-view Dataset

Coming soon.

Real Supplementary Datasets

Coming soon.

BibTeX citation

@inproceedings{panev2026diagnosing,
  title={Diagnosing Aerial-View Object Detectors with Foundation Image Generative Models},
  author={Panev, Stanislav and Jeon, Minhyek and Khindkar, Vaishnavi and Deshpande, Ahish and de Melo, Celso M. and Hu, Shuowen and Chakraborty, Shayok and De la Torre, Fernando},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2026}
}

Acknowledgment

This work was funded by the DEVCOM Army Research Laboratory, USA.