Introduction

The advent of scientific and technological advancement has led to the proliferation of automated fruit-picking robots in the ___domain of intelligent agriculture, which helps to reduce production costs and minimise errors caused by manual labour and inexperienced workers. In harvesting systems, two principal components are typically required: a vision detection module and a mechanical harvesting module1. The vision module is used to accurately identify and locate the fruits and is essential to guide the robotic arm accurately and efficiently to increase productivity. Consequently, the ability to efficiently identify and accurately locate fruits is a pivotal aspect that determines the efficacy of automatic picking robot applications2.

The complexity of agricultural environments is such that the YOLO algorithm was selected for use in agricultural applications, primarily on account of its superior efficiency and adaptability to complex environments. In recent years, scholars have made significant advancements in the field of fruit detection in complex environments. In their study,Liu X et al.3 employed external characteristics of fruit colour and shape for the purpose of object detection. Tian Y et al.4 have developed an enhanced version of the You Only Look Once (YOLO) algorithm for the purpose of detecting fruits with varying growth cycles. B Xiao et al.5 sought to enhance the precision of fruit detection by augmenting the model complexity through the utilisation of Transformer. The advancement of deep learning technology has highlighted the necessity to optimise neural networks through a variety of techniques, which has become a pivotal focus in the field of target detection and target tracking6,7,8,9,10 research. Nevertheless, despite the sophistication of algorithms such as YOLOv5, the application of such technologies in agriculture encounters considerable obstacles. The intricate network structure results in suboptimal real-time efficiency, while the elevated computational power demands render deployment a challenging undertaking.

Scholars have directed increased attention towards enhancing target detection models through the use of an attention mechanism, with the objective of focusing on target features. Xu et al.11 enhanced the instance segmentation performance by incorporating channel attention into the segmentation network. Zhang et al.12 employed channel attention to optimise the crack. M. Jaderberg et al.13 put forth a Transformer-based channel attention mechanism for STNs, whereby the input data is spatially transformed through learning, thereby enhancing the network’s resilience to geometric transformations such as image distortion and rotation. Furthermore, the optimisation of the bounding box loss function is of paramount importance. The most commonly employed methods include IoU14, GioU15, Diou16, CIoU16, Eiou17 and SioU18. This is achieved primarily through the introduction of geometric constraints to the bounding box, thereby ensuring the accuracy of target detection.Nevertheless, despite the enhanced outcomes achieved by these techniques, there are still substantial challenges associated with their deployment in agricultural applications. The intricacy of the network structure results in suboptimal real-time performance and a high demand for computational resources.

Recently, numerous scholars have been engaged in research aimed at integrating the YOLO algorithm with lightweight detection modules, including the ensemble GhostNet19, MobileDets20, PP-PicoDet21, and EdgeNeXt22, to reduce the number of parameters. Notwithstanding the enhancement in detection velocity, these lightweight algorithms are deficient in their capacity for feature extraction. Plus this, Knowledge distillation is a frequently employed lightweighting method in target detection networks23. In a recent study, Lan et al.24 demonstrated that the application of this method to the detection of autopilot yielded superior results.

In an effort to salute these challenges, this paper puts forward a proposal for a lightweight object detection network for the purpose of detecting apples in complex environments.The BASELINE model used in this paper is the YOLOv5 algorithm. The principal contributions of this research are outlined below:

  1. 1.

    This study presents a novel approach to data integration, resulting in the construction of a high-quality dataset. The reliability of this dataset is then verified through a series of experiments. Notably, all experiments conducted in this paper are based on this dataset, thereby addressing the dearth of data in this field.

  2. 2.

    In order to achieve an optimal balance between processing speed and accuracy, In this study, we put forward a novel lightweight feature extraction module, designated Efficient Shuffle Slim (EGSS).

  3. 3.

    To enhance the capability of feature extraction module. This study presents an innovative plug-and-play lightweight hybrid attention mechanism that simultaneously integrates channel, spatial, local and global information.

  4. 4.

    In order to maximise the compression model, this paper proposes SlimPAN to eliminate redundant channels through channel compression.

  5. 5.

    In order to compensate for the lack of the traditional loss function, Shape-IOU is used in this paper by taking into account the effect of the bounding box itself.

  6. 6.

    The knowledge distillation approach guarantees that the network will perform the required object detection tasks with a minimal number of parameters and computations, thereby facilitating the rapid and efficient operation of the model.

  7. 7.

    In order to ensure the feature extraction capability of the model while reducing the complexity.A novel detection network, the Efficient Lightweight object Detector (ELD), is proposed. It integrates the EGSS module and combines with SlimPAN. The network in question requires only a minimal amount of computing resources.

This paper is structured as follows: Section II is dedicated to an in-depth examination of the high-quality dataset constructed in the course of this research project. Part III provides a comprehensive analytical account of the innovative lightweight feature extraction module, hybrid attention mechanism, and the novel object detection network theme. The fourth section presents the findings of the analysis of the experimental data collected throughout the research process. Parts V, VI and VII, respectively, present a discussion of the research process, an analysis of the limitations and a summary of the conclusions.

Related work

YOLOv5

The most prevalent target detection network is YOLO25,26,27,28, which is a single-stage network designed for expeditious and efficacious one-shot detection29. The YOLOv5 algorithm is a popular choice in the field of object detection due to its efficient performance. In this study, YOLOv5 is further developed in the base framework with the objective of reducing the complexity of the code and improving the performance of the network. This constitutes a benchmark for this paper, with the aim of improving the implementability of the work. As illustrated in Fig. 1, the YOLO network partitions the input image into a grid of S Ã- S cells. The grid cells predict Bgo bounding boxes, each of which contains a target. Each bounding box comprises five parameters: (x,y,h,w,c), which denote the centre coordinates, width, height and confidence level of the bounding box, respectively. Ultimately, a C-dimensional vector is generated for each bounding box, indicating the probability that the bounding box belongs to each category. In essence, the output tensor of YOLOv5 comprises predictive information for each grid cell, with a total output dimension of S Ã- S Ã- (B Ã- 5 + C). This encompasses the parameter and category predictions for the B bounding boxes of each grid cell.

Fig. 1
figure 1

YOLO model for apple detection from images.

Full size image

Knowledge distillation algorithm

Both knowledge distillation algorithms and neural network pruning30 are effective techniques for reducing the computational burden of deep learning models. However, neural network pruning is particularly well-suited to networks with larger models. Q Lan et al.24 reweighed each instance and each scale for distillation based on the teacher’s loss, thereby facilitating the selective transfer of the teacher’s knowledge to the students. Sangwoo Park et al.31 presents a Cosine Similarity-Based Knowledge Distillation (CSKD) for robust, lightweight object detectors.Jike et al.32 proposed the multi-constraint molecular generation (MCMG) approach that can satisfy multiple constraints by combining conditional transformer and reinforcement learning algorithms through knowledge distillation. Kim et al.33 enabled the model to exhibit high performance by knowledge distillation without charging feature spoofing caused by modal differences. These scholars have demonstrated proficiency in the application of knowledge distillation, which serves as a valuable reference point for this study.

Feature extraction module

The feature extraction module constitutes the core of the target detection network. A popular approach to enhancing its feature extraction capability is to add an attention mechanism to the feature extraction module. Zhao et al.34 integrated the convolutional block attention mechanism with ShuffleNetv2 as a feature extraction module, thereby enhancing the network’s capacity to discern object features.Wang et al.35 employed a self-attention mechanism for the extraction and integration of multi-scale features in both the feature extraction module and the feature fusion module. This approach demonstrated some improvement in accuracy but did not yield effective lightweighting.Guo et al.36 devised a lightweight feature extraction module to enhance the precision of safflower detection in agricultural settings through a synergistic integration of Ghostconv and CBAM attention mechanisms. Lin et al.37 devised a novel cross-attention mechanism based on the Transformer structure, which is deployed between image blocks derived from single-channel features to capture global information. They demonstrated that this module has the potential to serve as a general-purpose feature extraction network. However, it necessitates greater computational resources compared to convolution-based networks.J Fu et al.38 proposed a dual attention network to adaptively integrate local features with global dependencies. Its efficacy was validated on multiple datasets, yielding favourable results. However, the network’s intricate structure necessitates a considerable computational burden, which may prove challenging to accommodate within the feature extraction module. The research conducted by the aforementioned scholars serves as a valuable reference for this study. However, their proposed methods still require significant improvement in terms of lightweighting. Consequently, this paper proposes a feature extraction module that is more efficient and lightweight, and incorporates a lightweight hybrid attention mechanism. This results in a reduction in computation and makes it more suitable for deployment on devices with limited computing power.

Dataset construction

The intricate nature of the orchard environment presents a significant challenge for traditional object detection networks, particularly in terms of accurately extracting features from apple fruits within this context. In order to facilitate the network’s ability to discern a greater number of fruit features in authentic settings, this study has constructed a comprehensive dataset, designated “Appledatas,” which encompasses 3,151 images. The majority of these images were sourced from a contemporary orchard in Zibo, Shandong Province, while a modest proportion were gathered from a traditional orchard in Tai’an, Shandong Province. The rationale behind collecting data from disparate geographical regions is to mitigate the impact of regional variations in fruit morphology on the performance of the network in feature extraction. By encompassing data from both standardised and non-standardised orchards, the network is enabled to learn fruit features from a diverse range of environments, thereby enhancing its generalisation capabilities. The dataset encompasses the majority of object detection scenarios encountered throughout the project, predominantly comprising mainstream situations such as light influence, view angle influence, occlusion influence, and dense fruit influence. Illustrative examples can be observed in Fig. 2. The dataset is divided into three distinct sets: training, validation, and test. The ratio of these sets is 8:1:1, respectively. The data is labelled in YOLO format using Labelme. The publicly available datasets in the industry are composed of fruit images in a single background without any interference. This situation is far from the actual orchard environment and does not satisfy the industry’s needs. Consequently, this paper addresses this gap in the literature. The dataset enhances the target features, enabling the model to improve its ability to extract and generalise features, thereby avoiding overfitting and underfitting. The dataset has been validated through experimentation, demonstrating its efficacy as a reliable source of feature inputs for the network, which significantly improves the reliability and accuracy of the object detection network in practical applications. The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

Fig. 2
figure 2

Typical scenarios for appledatas data centres.

Full size image

Methods

Overall network architecture

Figure 3 illustrates the lightweight object detection network, designated as the ELD, which is the subject of this study. The objective of the network is to reduce the consumption of computational resources while maintaining high object detection accuracy, thereby enhancing its practicality. The specific network parameters are detailed in Table 1.

Table 1 Detailed configuration table of each module in the ELD network.
Full size table
Fig. 3
figure 3

Architecture of ELD network(The EGSS lightweight feature extraction module employs MCAttention, an attention mechanism, to construct a novel backbone network in an innovative way. Furthermore, it has been lightweighted by SlimPAN to achieve a compressed model. Subsequent incorporation of the Shape-IOU loss function and utilisation of knowledge refinement serve to enhance the accuracy and stability of target detection).

Full size image

Images captured in practical applications contain a considerable amount of superfluous background information. In order to eliminate the interference and maintain optimal network performance, a novel lightweight yet efficient backbone network is proposed. The module initially preserves the essential characteristics of the fruits with high efficiency through the Conv_bn_SiLU module. Subsequently, the underlying feature extraction is carried out by the EGSS module, including steps 1 and 2. The EGSS module prioritises the reuse of initial features as well as feature refinement and fusion through the MACttention concern mechanism. As the EGSS module is more capable of processing feature information at the end of the backbone, the traditional SPPF structure is cancelled at the end of the backbone.

The SlimPAN methodology employs channel compression and point-by-point convolution at the neck network to effectively conserve resources. This approach entails the initial reduction of the number of channels and their subsequent unification, which optimizes the utilization of network resources. Given the enhanced feature fusion capacity of the C3 configuration, the substructure does not result in significant resource wastage following channel compression, and thus it is retained for utilisation in this investigation. Furthermore, depth-separable convolution with a smaller parameter number is employed in lieu of the conventional convolution for downsampling. The convolution kernel size can be modified to regulate the sensory field size. Subsequently, the Shape-IOU loss function is employed to address the limitations of the traditional loss function, thereby enhancing the network’s resilience. Ultimately, a lightweight network with high accuracy and straightforward deployment is developed through knowledge distillation.

Lightweight feature extraction module

Images acquired during apple detection in orchard environments typically exhibit a relatively complex background and are frequently contaminated with a considerable amount of noise. This has a considerable detrimental effect on the feature extraction performed by the network, which in turn affects the stability of the real-time object detection by the network. In order to address these challenges and enhance the utilisation of useful feature information in complex scenes, we have designed a lightweight feature extraction module, EGSS(Efficient Ghost-Shuffle Slim moudle), which is available in two different structures. Figure 4a illustrates a structure with a step size of 1, whereby the original input is segmented in consideration of the richness of the original image features. One part of the image retains the original features, while the other part undergoes feature extraction to eliminate unnecessary noise. Subsequently, the features are processed through operations such as the attention mechanism. Subsequently, the original features are superimposed, thereby establishing direct information interaction and enabling direct supervision of the earlier layers. Finally, effective features are fully mixed through shuffling operations, thereby promoting the effective reuse of low-latitude feature information. The structural design ensures both lightness and effective use of low and high latitude features, thereby overcoming the limitations of the majority of existing models. Figure 4b illustrates the structure with a step size of 2, which performs a complete feature reuse and feature extraction of the original features to satisfy the requisite feature richness. Following the initial splicing step, additional feature processing mechanisms are incorporated to mitigate the network degradation caused by the complex uncertainty of the original features.

Fig. 4
figure 4

EGSS module.

Full size image

During the phase of feature extraction, the model size and model feature extraction capability are balanced by two different raw feature utilisation methods, which are combined with MCAttention. The function of the MCAttention attention mechanism is to process multiple feature information, such as channel information, with the aim of extract high-level feature information and capture dependencies between features. It is essential to that when employing cascade operations for feature map expansion, there is a potential for the loss of positional information. To address this, partial convolution operations can be utilized after cascading to enhance the perception of the information and further optimize the model’s capability in feature processing.

In the feature extraction module, depth-separable convolution was employed on multiple occasions. With respect to computational efficiency in comparison to the parameter count K Ã- K Ã- C1 Ã- C2 of the standard convolution, the parameter count (denoted as K Ã- K Ã- C1 + C1 Ã- C2) of the depth-separable convolution is significantly reduced, as illustrated in Fig. 5 for a comparison of the two convolutions. Despite the reduction in computational cost, the computational complexity of deep separable convolution during training is still higher than that of traditional convolution, particularly in deep networks. Furthermore, the lack of acceleration support for deep separable convolution on some hardware platforms may also reduce its efficiency. To address this issue, DWConv before MCAttention in the feature extraction module is only used for the teacher network in knowledge distillation. In general, the module guarantees lightweight operation while ensuring the effective utilisation of low-latitude feature information for feature extraction within the network.

Fig. 5
figure 5

Contrast between depthwise separable convolution and standard convolution.

Full size image

Mix channel attention

The prevailing channel attention mechanism is currently limited in its scope, focusing exclusively on the overall information content and failing to account for the spatial differences between channels. When the entire channel is subjected to feature compression without due consideration of the spatial information, some crucial data may be discarded, potentially leading to an inaccurate prediction of the categories. Furthermore, the pixel-level spatial attention mechanism is susceptible to feature redundancy due to the equal distribution of weights across specific local regions. It is therefore necessary to introduce spatial information in a way that avoids information redundancy. This can be achieved by first processing advanced chunking on one branch and then processing the other branch globally.

As illustrated in Fig. 6, which depicts the principle of MCAttention, the input for this module is processed in parts and subsequently merged. The initial stage transforms the input into a 1*C*K*K vector, enabling the extraction of local spatial information through local pooling. This is followed by a conversion to one-dimensional features, which forms the basis for the second stage, in which global information refinement is conducted directly. Subsequently, the two parts of information are merged to achieve hybrid attention. To circumvent the issue of accuracy deterioration resulting from channel reduction, a one-dimensional convolution is employed to capture the interaction data between each channel and its neighbouring channels. This is achieved through the utilisation of a hybrid channel attention mechanism39.

Fig. 6
figure 6

MCAttention module.

Full size image

The structure in question can be processed by weighting the two-part operation, where weight is (0,1). The experimental results demonstrate that the method of multiplying the two branches by 0.5 and subsequently adding them together yields superior outcomes compared to the direct addition of the branches. This is primarily due to the suppression of redundant negative information through the operation of weighted sum, as illustrated in Fig. 7. In this figure, MCAttention-0.5 represents the result of addition by a factor of 0.5, while MCAttention-none denotes the result of direct addition. The size of the weighting factor or the operation of direct summing may be selected based on the specific circumstances. This paper employs a weighting factor of 0.5. In conclusion, MCAttention is an effective method for partitioning and utilising the key information of each channel without causing computational redundancy.

Fig. 7
figure 7

Comparison of the experimental results of the two MCAttention branch superposition methods.

Full size image

Lightweight neck SlimPAN

Figure 8 illustrates the SlimPAN structure, where the green circles represent C3, Concat operations or DWSBS, Concat operations. The connection process between these circles represents the fusion process between different features in the neck network. Following the extraction of features from the backbone network, the input image generates three distinct feature maps of varying dimensions, which are then transferred to the neck network for feature fusion. Upon entering the SlimPAN, the data is initially unified through a 1Ã-1 convolution operation, which serves to eliminate redundant and unimportant channels while simultaneously reducing the cost of feature computation. The depth-separable convolution40 is employed for all convolutions, with the exception of the initial 1Ã-1 convolution. The depth-divisible convolution extends the perceptual range by employing 5Ã-5 convolutions, thereby enhancing the perceptual range and generalisation ability of the neural network. The results of the experiment are presented in Fig. 9. The objective of this design is to facilitate more efficient information fusion with a reduced number of parameters.

Fig. 8
figure 8

Lightweight feature fusion module.

Full size image
Fig. 9
figure 9

Comparison of experimental results with different receptive fields under three convolution kernels.

Full size image

Shape-IOU loss function

This paper introduces a new boundary regression loss function, called Shape-IoU41. As shown in Fig 10. This novel approach seeks to rectify the limitations of conventional loss functions by placing particular emphasis on the regression problem of the bounding box. The definition of this function is presented in Eqs. (1–5).

$$\begin{aligned} ww= & \frac{2\times (w^{gt})^{scale}}{(w^{gt})^{scale}+(h^{gt})^{scale}} \end{aligned}$$
(1)
$$\begin{aligned} hh= & \frac{2\times (h^{gt})^{scale}}{(w^{gt})^{scale}+(h^{gt})^{scale}} \end{aligned}$$
(2)
$$\begin{aligned} distance^{shape}= & hh\times (x_c-x_c^{gt})^2/c^2+ww\times (y_c-y_c^{gt})^2/c^2 \end{aligned}$$
(3)
$$\begin{aligned} \Omega ^{shape}= & \sum _{t=w,h} (1-e^{-\omega _{t}})^{\theta },\theta =4 \end{aligned}$$
(4)
$$\begin{aligned} & {\left\{ \begin{array}{ll}|\omega _{w}=hh\times \frac{|w-w^{gt}|}{max(w,w^{gt})}\\ |\omega _{h}=ww\times \frac{|h-h^{gt}|}{max(h,h^{gt})}\end{array}\right. } \end{aligned}$$
(5)

Where scale, ww and hh denote the scaling factor, horizontal and vertical weighting factors; depending on the size of the target and the size of the gtound truth (GT), respectively. Equation (6) represents the total Shape-IoU loss function.

$$\begin{aligned} L_{\text {Shape-IoU}}=1-\textrm{IoU}+\textrm{distance}^{shape}+0.5\times \Omega ^{shape} \end{aligned}$$
(6)
Fig. 10
figure 10

Principle of shape-iou loss function.

Full size image

Knowledge distillation

Figure 11 illustrates how knowledge distillation (KD) employs a more intricate teaching network to educate a more streamlined and lightweight learner network. This enables the smaller network to obtain a greater quantity of useful knowledge without imposing an additional burden on the network.

Fig. 11
figure 11

Knowledge distillation principle.

Full size image

The teacher network, as outlined in this paper, comprises a fine-tuned EGSS module and an extended ELD network, designated as ELD-Teacher. The fine-tuned EGSS module is referred to as EGSS-Teacher, which serves to enhance the feature. By incorporating depth-separable 3Ã-3 convolutions and expanding the depth and width of the network, the representation and abstraction capabilities of the teacher network can be enhanced. This allows for the extraction of more complex high-level features, which can be employed to educate the student network. It is unlikely that the teacher and student networks will exhibit significant structural differences. If the networks are too dissimilar, it will be challenging for the student network to acquire useful knowledge, even if the teacher network demonstrates robust learning capabilities and feature information. This process can be interpreted as the student’s learning capacity being constrained, rendering it impossible to learn new knowledge even if this limit is surpassed.

The pivotal stage in this process is the dynamic alteration of the softmax probability distribution through the modification of the temperature coefficient T, as illustrated in Eq. (7).

$$\begin{aligned} q_i=\frac{exp(\frac{z_i}{T})}{\Sigma _jexp(\frac{z_j}{T})} \end{aligned}$$
(7)

where T=1 corresponds to the original softmax function,T<1 steepens the probability distribution, and T>1 makes the probability distribution smoother. During the process of knowledge distillation, the output results of different classes are typically balanced by increasing the temperature (T>1). This operation allows the student model to consider the output results of different classes equally, resulting in more information being obtained during the training process.

Soft labels and soft predictions are generated when the teacher and student networks make predictions following a softmax warm-up. The discrepancy between the soft labels and the soft predictions is calculated in order to guarantee that the predictions of the student network are analogous to those of the teacher network. The calculation process is illustrated in Fig. 12.

Fig. 12
figure 12

Knowledge distillation loss calculation.

Full size image

Experimental results and analyses

Experimental platform and parameter settings

In our experiments, we utilize Microsoft Windows 10 Operating System, NVIDIA3060 graphics card, Python3.8, Pytorch, CUDA, and cudnn to build a deep learning frame work. The epoch uniform setting is set to 200, and the knowledge distillation operation epoch is set to 300. The optimizer uses SGD by default. The evaluation of network performance in the experiment is primarily based on the trained network’s perfor mance in the validation set. This includes the metrics such as mAP, P (Precision), R (Recall), model file size, relevant model parameters, GFLOPs, infer ence time, and GPU speed. GPU speed is calculated as 1000/(preprocessing time +inference time+post-processing time). The experimental results are quantified using the aforementioned references. Table 2 displays the hardware configuration and model parameters relevant to the experiment.

Table 2 Hardware configuration and model parameters.
Full size table

Evaluation standard

In this study, precision (P), recall (R), mean accuracy (mAP),Average accuracy for relatively small targets in the data(mAPsmall) and FPS are mainly used to evaluate the model performance. As shown in Eqs. 8 and 9. where TP and FN represent true positives and false negatives in each original sample, respectively. mAP denotes the mean accuracy across all categories, where i denotes a specific category and n denotes the total number of categories. In this study, the focus was limited to the identification of fruits, hence n = 1. FPS = 1000/(pre-process+NMS+inference).

$$\begin{aligned} & \begin{aligned}&P=\frac{\textrm{TP}}{\textrm{TP}+\textrm{FP}}\times 100\%\cdot \\&R=\frac{\textrm{TP}}{\textrm{TP}+\textrm{FN}}\times 100\%\cdot \leftarrow \end{aligned} \end{aligned}$$
(8)
$$\begin{aligned} AP= & \int _{0}^{1} PR\textrm{d}R\times 100\%^{\leftrightarrow }\qquad \qquad \qquad mAP=\frac{1}{n}\sum _{i}^{n} AP(i)\times 100\%^{\leftrightarrow } \end{aligned}$$
(9)

Comparison of attention mechanisms

In order to facilitate a comparative analysis of diverse attention mechanisms, this study incorporates a selection of representative approaches, including ECA42, SE43, CA44, CBAM45, SimAM46, and MCAttention, into the EGSS module for experimental comparison under the same dataset.

Figure 13 and Table 3 present a comparative analysis of the performance of various attention mechanisms, as measured by [email protected], [email protected]:0.95, precision, and recall. The designation “None” signifies the absence of an attention mechanism within the model. In contrast, the model utilising MCAttention exhibits the highest [email protected], [email protected]:0.95 and recall values, thereby indicating that the model employing MCAttention is more efficient in comparison to the models that do not utilise or employ alternative attention mechanisms. This suggests that the proposed attention method is an effective approach. In particular, the CA spatial attention mechanism exhibited a high degree of instability, indicating that the study is more applicable to channel attention. Furthermore, the incorporation of superfluous attentional confederates has the potential to negatively impact the outcomes.

Table 3 Comparison of addition attention mechanisms in EGSS modules.
Full size table
Fig.13
figure 13

Comparison of different attention modules in [email protected], [email protected]:0.95, precision and recall.

Full size image

In conjunction with Fig. 14, the impact of diverse attentions can be more effectively conveyed, thus providing a more comprehensive understanding of the subject matter. As illustrated in the heat map, CA, CBAM and a spatial attention model demonstrated the capacity to focus on specific target characteristics. Although SimAM may focus on more information regarding large fruits, it is unable to recognise occlusion and the target is not readily apparent. In contrast, SE and ECA, which represent traditional forms of attention, demonstrate superior results. However, they are unable to extract features from the entire fruit. In contrast, MCAttention is capable of not only focusing on the detailed parts of the image but also of performing comprehensive attention to the features of the large target. This is sufficient to demonstrate the superiority of this module.

Fig. 14
figure 14

Heat maps of different attention mechanisms.

Full size image

Comparison of loss functions

This paper employs the use of Shape-IoU and compares it with other commonly utilised IoUs. The P-value of Shape-IoU is significantly enhanced with the same detection speed, exhibiting a 0.26-point improvement over that of CIOU. Additionally, the map0.5 demonstrates a slight improvement, indicating that the model’s stability has been augmented. The results of the experiment are presented in Table 4. It should be noted that the target may require different specifications depending on the device and application scenario. In such cases, the scale factor can be adjusted to align with the actual requirements. In this study, the scale factor employed was 1.0.

Table 4 Loss function comparison experiment.
Full size table

Knowledge distillation series comparison experiments

The efficacy of the teacher network in knowledge distillation directly impacts the learning outcome of the student network. A comparative and analytical examination of the three teacher networks, namely YOLOV5s, YOLOV5x and EGSS-tercher, reveals that the utilisation of EGSS-teacher as the teacher network enables the student network to obtain a more comprehensive range of information. This indicates that the most efficient method of knowledge refinement is achieved when the teacher and student networks have similar structures, as evidenced by the experimental results presented in Table 5.

Table 5 Comparison of average precision after distillation of student networks by different teacher networks.
Full size table

It was established through empirical investigation that the optimal refining effect is attained by employing the L2 loss function and setting the refining temperature at T=20. This finding was derived from an experimental comparison that also demonstrated the pivotal role of the refining loss function in this process. These results are depicted in Figure 15. A comparison of the experimental results is presented in Table 6 for the reader’s convenience. It can be seen that the distillation loss function and temperature adjustment must be adjusted according to the specific circumstances in order to achieve the most effective outcome.

Table 6 Comparison experiment of distillation loss and distillation temperature.
Full size table
Fig. 15
figure 15

Comparison of experimental results on distillation loss and distillation temperature.

Full size image

Network improvement ablation experiments

Ablation experiments are a standard tool in the analysis of complex neural networks, allowing for the examination of the impact of specific substructures and training methods on the model. This is a crucial aspect of the structural design of neural networks. In order to verify the effectiveness of the EGSS feature extraction module and the MCAttention attention mechanism, ablation experiments were conducted, the results of which are presented in Table 7.

Table 7 Ablation experiments.
Full size table

Table 7 demonstrates that the incorporation of the EGSS module results in a 61% reduction in parameters and a 57% reduction in computation compared to the original model. The [email protected] value remains largely unaltered, while the object detection speed is significantly accelerated. The application of MCAttention on this basis results in an elevation of [email protected] to the standard of the original model, while the amount of computation remains unaltered. Subsequently, further model compression is achieved through lightweight feature fusion, resulting in a reduction of 94% in parameters and 89% in computation compared to the original model. This is accompanied by a slight decline of 1.9% in [email protected], which is deemed to be an acceptable trade-off. To further enhance the object detection efficiency, knowledge distillation was employed to elevate the model accuracy to 0.874, while the detection speed was increased to 1.77 times that of the original model. In summary, the integration of these modules has led to the enhancement of accuracy, a substantial reduction in both computational and parametric quantities of the new model, as well as notable improvements in inference time and frames per second. The network demonstrates an equilibrium between a streamlined configuration and network performance.

Experimental comparison of different target

This study compares the performance of the proposed ELD algorithm with that of established object detection algorithms to assess its effectiveness. The image resolution employed in the dataset is 640Ã-640, and all the comparative experimental algorithms are based on the PyTorch open-source framework. The experimental results are presented in Table 8, which includes the following algorithms: YOLOv547, YOLOv7-tiny48, YOLOv649, YOLOv850, Gold-YOLO51, YOLOv952, and YOLOv1053. These networks were evaluated on the basis of their relatively lightweight structure. Furthermore, networks that combine YOLO with lightweight models were evaluated, including Shufflenetv254, Ghostnetv255, Mobilenetv356, and FasterNet57. The latest lightweight networks include YOLOv5-DE58,GGS-PF-YOLOv559,YOLOv5-Lite60. DETR61 represents the inaugural object detection framework to utilise Transformer as a fundamental component of the object detection model, thereby providing a valuable reference point. The Transformer model features a sophisticated architectural design that incorporates a multi-head attention mechanism, positional encoding, residual connectivity, and other advanced components. This results in a large model size. The capture of long-distance dependencies in input sequences requires the model to have sufficient parameters and capacity to handle long sequences of information. Training on large-scale datasets is usually required to obtain better performance. The self-attention mechanism necessitates the computation of the correlation between all features, which results in a relatively large number of parameters. In contrast, convolutional neural networks typically possess the property of shared weights, and the number of parameters is relatively small.Challenges to meet the real-time requirements of edge devices.

Table 8 Comparison experiment of different target detection algorithms.
Full size table

In conjunction with Fig. 16, this provides a more detailed and sophisticated perspective on the performance of each network. The diameter of the circles in the figure is indicative of the number of parameters. The smaller the circle and the closer it is to the upper right corner, the superior the results. The ELD model is the smallest, fastest, and most accurate of the networks, indicating that this network achieves an optimal balance between these two aspects. In comparison to the benchmark model, YOLOv5s, the proposed model exhibits a mere 0.3% reduction in accuracy, yet is nearly twice as rapid, accompanied by a considerable decrease in the number of parameters and computational cost. The results demonstrate that the network exhibits a more comprehensive performance at a minimal cost in terms of accuracy.

Fig. 16
figure 16

Different object detection networks compared in terms of [email protected], FPS, and model size.

Full size image

Discussion

In practice, the fruit ripening period is distinguished by the presence of lush branches and leaves, as well as the viewing angle of image acquisition, which gives rise to complex scenarios characterised by overlapping and occluded targets, as well as dense targets. Figure 17 compares the detection performance of the improved model and the mainstream detection network in these scenarios. The YOLOv5, YOLOv7, and YOLOv8 models demonstrate a certain degree of leakage and insensitivity to the feature regions, in contrast to the improved model, which exhibits more accurate recognition and localization properties.

Fig. 17
figure 17

Heat map of different detection networks.

Full size image

Figure 18 illustrates the outcomes of the model proposed in this paper in comparison to those produced by the widely utilized object detection network. In cases (a) and (b), which are largely free of interference and relatively straightforward to detect, the networks demonstrate satisfactory performance. However, YOLOv7 exhibits a slight tendency to under-detect, while YOLOv5 displays a proclivity for re-detection. In Case (c), all three networks (YOLOv5, YOLOv7, and YOLOv8) demonstrate a tendency to miss detection of small target occlusion, with YOLOv5 exhibiting a particularly high rate of retries. The interference in Case (d) is more pronounced, posing a significant challenge to the network. All networks exhibit a degree of missed detection, with YOLOv5 displaying the most severe performance. The proposed model demonstrates the strongest interference ability.

Fig. 18
figure 18

Comparison of classical network detection results in typical real-world scenarios.

Full size image

Limitations of the study

The limitations of this questioned study can be summarised in these points:

  1. (1)

    The current dataset, which contains apple species as well as natural environment condition numbers can be further increased.

  2. (2)

    Although the model works better than conventional mainstream object detection algorithms in terms of accuracy and lightness, there is still a certain leakage rate under strong sunlight.

Conclusion

In the context of automated harvesting devices, the accuracy of target identification is of paramount importance. This paper addresses the limitations of existing methods and proposes a lightweight object detection model that maintains good object detection accuracy. This study considers real-world application scenarios in a comprehensive manner and proposes an EGSS feature extraction module and a hybrid attention mechanism for MCAttention in an innovative manner. The results demonstrate the efficacy of these methods in extracting target features for dense , even in the presence of strong interference. Both modules are designed as plug-and-play components that can be fully integrated into other object detection networks. Furthermore, the accuracy of the network is enhanced through the implementation of a knowledge refinement strategy. While the network and method values presented in this paper are specific to apple detection, they can be generalized to other fruit detection scenarios.

The final network achieves an accuracy of 87.4% with a mere 1.7 GFLOPs of computation, a parameter count that constitutes a mere 6.1% of the baseline model. It demonstrates the ability to reduce computational overhead, enhance object detection speed, and guarantee object detection accuracy. Subsequent to this, it will be deployed to edge devices to further validate its effectiveness and integrate a object detection and localization system for practical engineering applications.