The durability of civil infrastructures faces serious challenges in harsh environments, especially as the widespread use of glass fiber reinforced polymer (GFRP) materials in deserts and coastal areas makes their surfaces susceptible to erosion by wind and sand, which progressively weakens their structural integrity. Existing manual inspection methods are labor-intensive and inefficient, making it difficult to meet the demand for large-scale and real-time monitoring. Although the YOLO (You Only Look Once) series of algorithms show some potential in material defect detection, there is still a significant lack of ability to recognize fine-grained, multi-scale erosion features on GFRP surfaces under high-noise backgrounds, and the performance is limited, especially in small-scale damage detection. To address the above problems, this paper proposes a two-layer channel-optimized YOLO target detection model, DuoOpti-YOLO, for improving the detection accuracy of multi-scale damage on GFRP surfaces under wind-sand erosion. The model incorporates a hierarchical multi-convolutional bi-directional feature pyramid network (BiFPN-HMC) to enhance the multi-scale feature expression capability, and introduces a spatial-channel attentional mechanism (SA-C2f) to highlight the critical regions, suppress the background interference, and realize the coarse-to-fine progressive erosion recognition strategy. The experimental results show that DuoOpti-YOLO achieves 95.2% mean accuracy percentage (mAP) on the self-constructed GFRP wind-sand erosion dataset, which is better than the traditional YOLO model in terms of detection accuracy and inference efficiency, and demonstrates the potential of real-time monitoring of composite material damage in extreme environments and its application value.