Site-Specific Weed Management (SSWM) provides precise weed control and reduces the use of herbicides, which not only reduces the risk of environmental damage but also improves agricultural productivity. Accurate and efficient weed detection is the foundation for SSWM. However, complex field environments and small-target weeds in fields pose challenges for their detection. To address the above limitation, we developed WeedDETR, a real-time end-to-end detection model specifically designed to enhance the detection of small-target weeds in unmanned aerial vehicle (UAV) imagery. WeedDETR incorporates RepCBNet, a backbone network optimized through structural re-parameterization, to improve fine-grained feature extraction and accelerate inference. In addition, the designed feature complement fusion module (FCFM) was used for multi-scale feature fusion to alleviate the problem of small-target weed information being ignored in the deep network. During training, varifocal loss was used to focus on high-quality weed samples. We experimented on a new dataset, GZWeed, which contains weed imagery captured by an UAV. The experimental results demonstrated that WeedDETR achieves 73.9% and 91.8% AP0.5 (average precision at 0.5 intersection over union threshold) in the weed and Chinese cabbage [Brassica rapa subsp. chinensis (L.) Hanelt] categories, respectively, while achieving an inference speed of 76.28 FPS (frames per second). In comparison to YOLOv5-L, YOLOv6-M, and YOLOv8-L, WeedDETR demonstrated superior accuracy and speed, exhibiting 3.5%, 6.3%, and 3.6% higher AP0.5 for weed categories, while FPS was 14.9%, 12.9%, and 1.4% higher, respectively. The innovative architectural design of WeedDETR significantly enhances the detection accuracy of small-target weeds, enables efficient end-to-end weed detection. The proposed method establishes a solid technological foundation for UAV-based precision weeding systems in field conditions, advancing the development of deep learning-driven intelligent weed management.