mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-13 17:48:22 +06:00
de6d19ea92
* Add transforms for object detection * DETR models + Yolos * Scrappy additions * Maskformer image processor * Fix up; MaskFormer tests * Update owlvit processor * Add to docs * OwlViT tests * Update pad logic * Remove changes to transforms * Import fn directly * Update to include pad transformation * Remove uninstended changes * Add new owlvit post processing function * Tidy up * Fix copies * Fix some copies * Include device fix * Fix scipy imports * Update _pad_image * Update padding functionality * Fix bug * Properly handle ignore index * Fix up * Remove defaults to None in docstrings * Fix docstrings & docs * Fix sizes bug * Resolve conflicts in init * Cast to float after resizing * Tidy & add size if missing * Allow kwards when processing for owlvit * Update test values
66 lines
3.0 KiB
Plaintext
66 lines
3.0 KiB
Plaintext
<!--Copyright 2022 The HuggingFace Team. All rights reserved.
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
the License. You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
specific language governing permissions and limitations under the License.
|
|
-->
|
|
|
|
# Deformable DETR
|
|
|
|
## Overview
|
|
|
|
The Deformable DETR model was proposed in [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://arxiv.org/abs/2010.04159) by Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, Jifeng Dai.
|
|
Deformable DETR mitigates the slow convergence issues and limited feature spatial resolution of the original [DETR](detr) by leveraging a new deformable attention module which only attends to a small set of key sampling points around a reference.
|
|
|
|
The abstract from the paper is the following:
|
|
|
|
*DETR has been recently proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance. However, it suffers from slow convergence and limited feature spatial resolution, due to the limitation of Transformer attention modules in processing image feature maps. To mitigate these issues, we proposed Deformable DETR, whose attention modules only attend to a small set of key sampling points around a reference. Deformable DETR can achieve better performance than DETR (especially on small objects) with 10 times less training epochs. Extensive experiments on the COCO benchmark demonstrate the effectiveness of our approach.*
|
|
|
|
Tips:
|
|
|
|
- One can use [`DeformableDetrFeatureExtractor`] to prepare images (and optional targets) for the model.
|
|
- Training Deformable DETR is equivalent to training the original [DETR](detr) model. Demo notebooks can be found [here](https://github.com/NielsRogge/Transformers-Tutorials/tree/master/DETR).
|
|
|
|
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/deformable_detr_architecture.png"
|
|
alt="drawing" width="600"/>
|
|
|
|
<small> Deformable DETR architecture. Taken from the <a href="https://arxiv.org/abs/2010.04159">original paper</a>.</small>
|
|
|
|
This model was contributed by [nielsr](https://huggingface.co/nielsr). The original code can be found [here](https://github.com/fundamentalvision/Deformable-DETR).
|
|
|
|
## DeformableDetrImageProcessor
|
|
|
|
[[autodoc]] DeformableDetrImageProcessor
|
|
- preprocess
|
|
- pad_and_create_pixel_mask
|
|
- post_process_object_detection
|
|
|
|
## DeformableDetrFeatureExtractor
|
|
|
|
[[autodoc]] DeformableDetrFeatureExtractor
|
|
- __call__
|
|
- pad_and_create_pixel_mask
|
|
- post_process_object_detection
|
|
|
|
|
|
## DeformableDetrConfig
|
|
|
|
[[autodoc]] DeformableDetrConfig
|
|
|
|
|
|
## DeformableDetrModel
|
|
|
|
[[autodoc]] DeformableDetrModel
|
|
- forward
|
|
|
|
|
|
## DeformableDetrForObjectDetection
|
|
|
|
[[autodoc]] DeformableDetrForObjectDetection
|
|
- forward
|