Attention-enhanced and integrated deep learning approach for fishing vessel classification based on multiple features

The sustainable exploitation of marine living resources is the key to securing the global blue food supply and protecting marine ecosystems. Effective fisheries management is essential for achieving sustainability, and monitoring fishing vessels is a crucial component of fisheries management^1,2. In particular, monitoring the types of fishing vessels (trawling, gillnetting, etc.) is important because various types of fishing vessels have different effects on ocean environments and marine sources³. For example, bottom trawling is more likely to destroy seabed habitat, while gillnetting has a high probability of entangling sea turtles. However, traditional maritime navigation surveillance relies on radar and imagery, guided by operator expertise, leading to limited monitoring accuracy and significant human resource consumption⁴. The number of fishing vessels with unknown vessel types has increased with the rising number of registrations worldwide. In addition, the types of fishing vessels may be mislabeled for a variety of reasons, which is often associated with violations, such as smuggling, illegal, unreported, and unregulated (IUU) fishing^5,6. Therefore, accurately and effectively identifying fishing vessel types can assist in fisheries management².

Data from the automatic identification system (AIS) can be applied to identify types of fishing vessels^7,8. The AIS is a novel type of navigation-assisted telecommunication system that sends and receives real-time, uninterrupted information about vessels^9,10. The AIS receivers based on land and satellite generate a great number of AIS data, typically containing identification information, ___location coordinates, navigational statistics, vessel particulars, and timestamps^11,12, enabling global monitoring of vessel dynamics. Among them, the identification section includes the vessel’s name, maritime mobile service identification (MMSI number), and flag state. Location data represents the current longitude and latitude of the vessel, while navigational data encompasses status, direction, and speed. Vessel particulars consist of information regarding the length, width, and tonnage of a vessel. Timestamps record temporal data related to the vessel’s activities¹³. In recent years, a series of breakthroughs have been made based on AIS data in the issues such as ship positioning, marine traffic safety, monitoring marine pollution, fishing ground prediction, and fishing behavior classification¹⁴. Furthermore, real-time AIS data simplifies ship status identification and can extract navigational characteristics from historical records, making AIS data more valuable for maritime monitoring and anomaly detection¹⁵. However, certain vessels deliberately avoid detection and participate in illegal activities, like turning off transponders, altering position data, or transmitting false vessel type information¹⁶. This poses a challenge in accurately identifying vessel types in real-time or historical AIS data, obstructing further data investigation and analysis⁴. Hence, it is necessary to conduct scientific analysis of AIS data using statistical methods to accurately classify vessels¹⁷, which is crucial for combating illegal fishing activities at sea.

With the progression of computer technology and the evolution of big data processing techniques, advanced tools have been made available for AIS data mining. Leveraging artificial intelligence (AI) algorithms to extract behavioral traits from AIS data and categorize fishing vessels constitutes a pivotal research direction⁴. By extracting vessel features and combining machine learning technology¹⁸, various fishing vessel identification models have been proposed with various degrees of success. For example, Sheng et al.¹⁵ built a logistic regression model to classify fishing and cargo ships, which involves segmenting original trajectories into anchored-off, straight-sailing, and turning sub-trajectories, extracting specific features from each, and conducting thorough feature analysis and selection. Guan et al.¹⁹ explored a classifier using a light gradient boosting algorithm to classify three types of vessels, by calculating the statistical characteristics, such as heading, positional changes, speed, and displacement under various conditions in the Northern South China Sea. Then, Huang et al.⁴ introduced an algorithm based on random forest (RF) and extreme gradient boosting (XGBoost) approaches tailored for categorizing four types of vessels, achieving impressive classification performance using just three geometric features and one trajectory behavior feature of the vessels.

Because deep learning technology has accomplished great success in lots of domains²⁰, some researches have used it to address the problem of vessel classification. For instance, Nguyen et al.²¹ considered converting latitude and longitude coordinates into one-hot vectors to represent geometrical features and built a deep learning model by using recurrent neural network (RNN) with latent variable modeling, to solve the classification task of four vessel types. The predictor constructed by Duan et al.²² to identify four types of ships was a semi-supervised deep learning method based on a variable autoencoder, which extracted the kinematic and static trajectory information from historical AIS information and learned unlabeled data to achieve simultaneous discriminate learning and generative learning. Additionally, Llerena et al.²³ employed convolutional neural network (CNN) and long short-term memory network (LSTM) to conduct binary classification between fishing and non-fishing vessels by extracting features such as temporal changes in vessel activity, speed, distance, heading, and fine-tuning the model parameters within an imbalanced dataset. Subsequently, Gu et al.²⁴ explored a multimodal fast gated transformer model to classify single trawling fishing vessels. They combined AIS data and radar images, converted the data into trajectory point images, and conducted binary classification by extracting information from different dimensions in the time series and feature space. In a recent breakthrough, integrated algorithms of machine learning and deep learning have been gradually and extensively applied in fisheries and have reached greater performance than traditional approaches. Brian et al.²⁵ developed a method to process sequence data with the RNN framework, performed regional ship behavior prediction with the clustering algorithm of machine learning, and evaluated it using a geographical region as a test case, showing reliable performance. To date, Wang et al.²⁶ designed a multi-feature integrated learning recognition method, integrating computational technologies including RF, XGBoost, CNN, and bidirectional gated recurrent unit (BiGRU) to identify four types of vessels, namely cargo ships, fishing boats, passenger ships, and tankers from a large amount of space-based AIS data.

Despite major advances in recent years, there are still three major challenges in identifying fishing vessel types using AIS data. First, the majority of existing classification models typically have a number of categories ranging from 2 to 4, and the differences between fish carriers and different types of fishing vessels cannot be significantly distinguished. Second, although the static and dynamic features of vessels can be effectively used for classification, most methods mainly focus on a single feature and lack multiple features combining kinematic and static information. Finally, the integrated algorithms of deep learning could be further explored, and there is still room for improvement in the efficiency of classification algorithms. Therefore, it is essential to explore a more accurate method to address the problem of classifying multiple types of fishing vessels.

In light of the above problems, we proposed an integrated deep learning-based algorithm to effectively identify five types of fishing vessels. The main contribution of this paper consists of two points. On the one hand, a multiple perspective approach of fishing vessel feature construction was built to extract the geometrical, static and dynamic features from AIS data, which can discover the behavior patterns of different types of fishing vessels. On the other hand, we adopted a classification model called BiLSTM-CNN-Attention, integrating two-dimensional bidirectional long short-term memory (BiLSTM) and CNN with an attention module, significantly improves the precision of multiclassification tasks. In this paper, “Materials and Methods” provides a detailed account of data preparation, methods used for fishing vessel feature construction and extraction, and the development of the classification model. “Results” presents a specific analysis of relevant experiments, including the experimental setup, model parameter configuration, and analysis of experimental results. “Conclusion” summarizes the paper’s findings. In “Discussion”, the contributions of AIS to the monitoring and management of fishing vessels in the era of big data, and the assistance that AI will bring to the fishing industry are elaborated upon. To our knowledge, this study is the first research to develop the model of adding an attention mechanism to an integrated deep learning method combining the fusion of multiple features for fishing vessel classification using AIS data. We aimed to provide an accurate and effective approach to monitor the types of fishing vessels based on AIS data.

This study presented an ensemble deep learning framework, which consisted of four parts (Fig. 1). First, the sample sets were constructed by data preprocessing, the training dataset and independent test dataset were divided. Because the number of labeled fishing vessels collected varies, the number of samples obtained for every type is different. To solve the imbalance problem between the five types of fishing vessel samples, the synthetic minority oversampling technique (SMOTE) method, which is an effective approach to solve the problem of sample imbalance²⁷, was adopted. Second, the geometrical, static, and dynamic characterizations of the gillnetter, hook and liner, trawler, fish carrier, and stow net vessel were extracted and selected by the RF and XGBoost algorithm. Then, we designed an integrated deep learning-based approach with an attention mechanism to predict the behavior of the five types of fishing vessels. Finally, the prediction abilities of different classifiers were estimated through comparative experiments, and BiLSTM-CNN-Attention model showed the best performance in the multi-classification task.

The framework for the classification of five types of fishing vessels.

Full size image

Data preprocessing for benchmark datasets

The focus of this study is on the classification of gillnetter, hook and liner, trawler, fish carrier, and stow net vessel. The labeled AIS data was sourced from the National Data Center for Distant-water Fisheries of China. This platform is based in Shanghai Ocean University, which stores various types of national fishery data, including catch, effort, AIS, vessel types, etc. We selected the AIS data of Chinese offshore with known labels from 2018 to 2021 as the training dataset, and 2022 as the independent testing dataset. The data distribution is visualized in Fig. 2, where points of different colors represent various types of fishing vessels.

Original data distribution.

Full size image

There may be some problems with the AIS information obtained by satellites, such as data errors and losses by reason of human factors or satellite signal transmission, and the analysis in practical application is detrimental²⁸. Hence, it is required data preprocessing to address these negative impacts, which can provide reliable AIS data for the classification task of subsequent fishing vessels. In this study, mistake and incomplete data were deleted using the following procedure. First, only one duplicate AIS dataset was retained (identical fishing vessel, time and ___location, etc.). Second, we removed AIS data for fishing vessels with speeds greater than 12 knots. In general, the speeds of all kinds of fishing vessels were typically 0–12 knots²⁹, and speeds greater than 20 knots were considered abnormal values³⁰. However, the data analysis showed that very few of the AIS data used in this study had a speed greater than 12 knots, so the threshold for outliers was set at 12 knots. Next, all AIS data for each vessel were divided by day to further explore the daily trajectory of fishing vessels. Finally, AIS data without sufficient trajectory points were removed. If a fishing vessel does not have enough AIS data points or time information, it cannot accurately judge the activity behavior of the fishing vessels. According to the work of Duan et al.²², trajectories with a time length less than 6 h and AIS messages with fewer than 160 records were filtered out. After data preprocessing, the benchmark dataset included more than 6 million trajectory data of fishing vessels. Among them, the training dataset contained 3,832 gillnetters, 860 hook and liners, 4,513 trawlers, 1,363 fish carriers, and 1,917 stow net vessels, while the 1,000 fishing vessels were randomly adopted as the independent testing dataset, 200 of each type (Table 1). Due to the disparities in data quantities across different categories, this paper employed the SMOTE oversampling algorithm to balance the dataset. For categories with less data, it randomly selected a sample in the feature space, then calculated the closest several other samples based on their Euclidean distance. By connecting these samples and performing interpolation, new samples were generated to balance the data distribution.

Feature extraction and selection of fishing vessels

Extracting features is a critical procedure in the identification of fishing vessels and abnormal recognition³¹. Whether the chosen features could represent the discrepancies between various types of fishing vessels primarily determines the classification capability. Considering that the geometrical characteristics include the length, width, area, and ratio of a fishing vessel, the static characteristics refer the holistic characteristics of a fishing vessel during operation, such as position, speed and course, while the dynamic properties reflect the displacement over a certain period of time. This paper extracted and combined the geometric, static and dynamic characteristics of fishing vessels from AIS data to fully reflect the differences in fishing behavior among gillnetters, hook and liners, trawlers, fish carriers, and stow net vessels.

Geometrical features construction

Different types of fishing vessels vary in size due to different operating areas, sailing speeds, and target fish species. Therefore, geometrical features are one of the basic features that identify the sorts of fishing vessels. In this work, we constructed geometrical features that contain four characteristics of fishing vessels, i.e., \(Length\), \(Width\), \(Area\), and \(Ratio\)²⁶. Among them, \(Length\) represents the distance from bow to stern, \(Width\) represents the distance from port side to starboard, \(Area\) and \(Ratio\) can be calculated by formula (1).

The calculated four-dimensional features were labeled \({\text{F}}_{0}-{\text{F}}_{3}\) as the geometrical features and input for the classification model of fishing vessels.

Static features construction

It is well known that the sequential position, speed and course changes of various types of fishing vessels have different distinctions, but the changes in the same fishing vessels are similar³². To extract the differences in the static features of the five kinds of fishing vessels, statistical values were used for analysis. The most commonly used statistical methods, such as the arithmetic mean, median, upper quartile (UQ), and lower quartile (LQ) could reflect the overall trend of the data. The standard deviation (STD) and dispersion coefficient (COE) could be adopted to represent the tendency of divergence. The larger the STD and COE values of the sample data are, the greater the overall dispersion of the sample. Therefore, the mean, UQ, median, LQ, STD, and COE value of longitude, latitude, speed and heading of fishing vessels were chosen as static characteristics. Among them, the speed was divided into three segments, namely from 0 to 2 knots, from 2 to 8 knots, and from 8 to 12 knots, and the statistical value of each segment was calculated to further extract the various behavior information of fishing vessels in floating, fishing and sailing state.

As a result, by calculating six statistics for the longitude, latitude, heading, speed and three speed segments of fishing vessels in AIS data, a \(7\times 6\)-dimensional feature vector was formed, labeled \({\text{F}}_{4}-{\text{F}}_{45}\), which could on behalf of the distinctions in the overall characteristics of the fishing vessels.

Dynamic features construction

The statistical values of dynamic changes in each time period were calculated to deeply discover the distinctions of the latitudinal and longitudinal changes in the five fishing vessels when operating. To accurately characterize the trajectory changes of five types of fishing vessels when operating, the statistical values of the time change of each moment \(t\) relative to the previous moment \(t-1\) and the change of position \(\left({x}_{t},{y}_{t}\right)\) at each moment relative to the position \(\left({x}_{t-1},{y}_{t-1}\right)\) at the previous moment were calculated. Specifically, the value of time change \(\Delta time\), longitude change \(\Delta {lot}_{{x}_{t}-{x}_{t-1}}\), and latitude change \(\Delta {lat}_{{y}_{t}-{y}_{t-1}}\) were calculated. The UQ, median, LQ, STD, and COE of the changed values were calculated to represent the dynamic characteristics, which compositing a \(3\times 4\)-dimensional feature vector and were labeled \({\text{F}}_{46}-{\text{F}}_{57}\).

Having completed the feature extraction of five kinds of fishing vessels, a 58-dimensional feature vector was constructed, which included geometrical features, static features of latitude, longitude, speed and heading, and dynamic features of time and position changes (Table 2). This feature vector describes the comprehensive feature distinctions of different types of fishing vessels.

Feature selection based on the XGBoost and RF algorithm

Among the obtained 58 features that fused geometrical, static, and dynamic information, not all factors are conducive to the classification, and a few features may be collinear or even conflicting. Therefore, it is necessary to analyze the correlation between all features. We plotted a correlation heatmap of 59 elements (58 features and the type of vessels) as shown in Fig. 3. There is an obvious correlation between some features, such as STD (\({\text{F}}_{8}\)) and COE (\({\text{F}}_{9}\)) of longitude data, the median (\({\text{F}}_{12}\)) and STD (\({\text{F}}_{14}\)) of latitude data, the mean (\({\text{F}}_{16}\)) and median (\({\text{F}}_{18}\)) of speed data, and the UQ (\({\text{F}}_{41}\)) and median (\({\text{F}}_{42}\)) of heading data.

The correlation heatmap of different features. The closer the color is to the one corresponding to 0, the smaller the correlation between the two features. The lighter colors indicate a stronger negative correlation, while the darker colors indicate a stronger positive correlation.

Full size image

Tree-structure-based machine learning algorithms usually calculate scores to features to assess their importance, known as "built-in feature importance"³³. These scores quantify how important each feature is to the target variable, with higher scores indicating greater importance or relevance. Classification methods based on decision tree and gradient boosting algorithms such as RF and XGBoost, evaluate each feature’s contribution at various split points within decision trees to determine its significance. Nodes within these decision trees estimate feature importance by weighing the features and tracking their impact on boosting performance metrics. Essentially, significant features near the root node are given higher weight, suggesting its substantial contribution to the model’s performance. The importance of a feature increases with the number of decision trees selecting it. Feature importance scores are calculated by summing weighted contributions across all trees and averaging them. To reduce the impact of feature correlation on the precision of the classification model, we chose two powerful algorithms widely used and highly effective in the field of feature selection, RF and XGBoost. These two algorithms provide importance scores for each feature, aiding in the determination of which features are most important for the model’s predictive performance. Using the importance function of the RF and XGBoost algorithms, we ranked the importance of all the constructed fishing vessels features to evaluate each feature and make selections. RF utilized the gini function to assess node split quality and XGBoost used the gain metric to specify feature importance based on average gain from feature splits. Finally, we filtered 58 hybrid features to 48-dimensional features.

Ensemble learning model construction

The overall framework

The purpose of ensemble learning is to build a powerful predictor by combining the advantages of multiple base classifiers. In this study, a high-efficiency ensemble model of fishing vessel classification was constructed, and the overall framework process as shown in Fig. 4. First, the model combined geometric, static and dynamic features of fishing vessels from the AIS data, and selected features to form a 46-dimensional numerical vector as input. Second, we constructed two bidirectional LSTM layers to learn the global features and fed into the CNN model with ReLU activation function, which further extracted local features through two convolutional layers. Next, an attention layer with rectified linear unit (ReLU) activation function was added to focus on important features and speed up the calculations of model. Lastly, the fully connection layer adjusted the feature dimension and output the prediction results of five types of fishing vessels.

The structure and parameters of BiLSTM-CNN-Attention model. In the bidirectional network layer, \({h}_{i}\) (i = 1,2,3, …, n) represents the feature vectors output learned by the LSTM layer.

Full size image

BiLSTM-CNN ensemble learning model

Long Short-Term Memory (LSTM) is a specialized neural network³⁴ that excels in processing sequential data by considering long-term dependency, thereby gaining a deeper understanding of underlying patterns. Long-term dependency refers to situations where there exist correlations or dependencies between elements that are distant in time series data or sequential information. Analogous to the human brain’s memory system, LSTM excel at retaining information over extended time intervals by selecting crucial features to retain while discarding less significant ones, enhancing feature memorization in classification models. LSTM employs a mechanism called “gate units” to control the input, output, and retention of features, enhancing the network’s memory and learning capabilities³⁵. BiLSTM³⁶ combines two LSTM networks, each tackling forward and backward sequences, concurrently learning past and future feature information. This bidirectional learning approach comprehensively captures sequential data patterns and changes, thus enhancing the model’s precision and adaptability.

Similar to BiLSTM, CNN stands out as a prominent deep learning technique widely utilized across diverse domains³⁷. It comprises multiple layers, each offering unique functionality for classification models. The convolutional layer, resembling a feature extractor, observes small regions of sequential data to pinpoint features at different positions, aiding in crucial feature extraction and comprehension to support network learning and decision-making³⁸. Subsequently, the pooling layer acts as an information summarizer, focusing on overall features rather than intricate details. It assists in reducing data volume and computational complexity while preserving vital features, enhancing model training efficiency and inference. The activation layer functions akin to a switch for signal channels, triggering when the sum of weighted input data hits a specific threshold. This mechanism helps networks better identify and grasp complex data patterns, thereby facilitating improved decision-making and analysis. The flatten layer is typically used to modify the dimensions of the output vector. Finally, the fully connected layer, serving as a knowledge integrator, combines previously learned features and comprehending the relationships between features. This layer fosters a deep understanding and thorough analysis of overall data, enabling the network to more effectively discern data relationships and derive final classification outcomes. CNN not only autonomously learns features but also shares parameters to lessen computational burden, constituting a potent tool for tackling intricate classification challenges.

The BiLSTM-CNN integrated network combines two distinct analytical approaches to address classification problems collaboratively. Through BiLSTM, global features are transformed into dependent feature vector sequences comprising forward and backward global feature vectors coupled in a continuous manner. This fosters a comprehensive grasp of the overall sequential features. Utilizing CNN, the network learns the weight coefficients of local feature vectors in the dependent sequences, focusing on the crucial information. By integrating these methods, the BiLSTM-CNN network adeptly analyzes sequential data for accurate classification, thereby delving deeper into data regularities and patterns.

In this study, we used the BiLSTM-CNN model to make predictions for five different types of fishing vessels. This model built a nine-layer network (Table 3), including two bidirectional layers with ReLU activation functions, a reshaping layer, two convolutional layers with ReLU activation functions, a pooling layer, a flatten layer, an attention layer with ReLU activation function, and a fully connected layer with softmax activation function. Among them, the inclusion of non-linear activation functions serves to fulfill the activation layer’s role, assisting the network in grasping complex non-linear correlations within the data, thereby enhancing data fitting. The principal function of the flatten layer is to modify the feature dimensions. Moreover, by conducting hyperparameter tuning to explore all possible combinations of hyperparameters within a specified range, evaluating the performance of different parameter sets through cross-validation, and ultimately determining the best hyperparameter combination to ensure optimal model performance during testing. The parameters of the model proposed in this paper are shown in Table 4.

Attention mechanism

The role of the attention mechanism is to simulate the brain focusing on a specific area at a specific moment, selectively obtaining more valuable information and ignoring worthless information^39,40. It can enhance the impact of key information by distributing different weight values to the hidden layer units of a neural network. In this work, the attention mechanism was adopted to make the model learn sensible feature vectors and the vital information that dominating the prediction process to improve the accuracy of model judgment⁴¹. The attention mechanism processes the data according to the following equation.

where \(M\) is the hyperbolic tangent function, \(Y\) refers to the feature matrix caught by the integrated BiLSTM and CNN networks, \({w}_{\theta }^{T}\) represents the transpose of the weight matrix, \(\theta\) is the ReLU function, and \(A\) represents the last output result processed by the attention mechanism.

Classification performance evaluation

Different classification algorithm

Support vector machine (SVM)⁴², RF⁴³ and XGBoost⁴⁴ in machine learning, CNN⁴⁵, BiGRU and BiLSTM³⁴ in deep learning were used to classify and learn the feature selection results. We optimize the parameters for SVM, RF and XGBoost models by using the “sklearn” package version 1.1.3 and “xgboost” package version 1.6.2 in the python software version 3.9.13 environment. The parameters configured for these three machine learning algorithms are outlined in Table 5. To ensure a fair comparison, within the Python software version 3.9.13 environment, utilizing version 2.10.0 of the “tensorflow” package, the unit quantities and parameters of the CNN model are set identical to the CNN segment in the BiLSTM-CNN-Attention model, while the unit quantities and parameters of the BiGRU and BiLSTM models align with the BiLSTM segment in the BiLSTM-CNN-Attention model.

Different evaluation indicators

K-fold cross-validation (CV) and independent dataset tests are statistical analysis tools used to assess the effectiveness of machine learning or deep learning models. In this paper, we adopted the tenfold CV and the independent dataset test based on the benchmark datasets. On the training dataset, the dataset was randomly grouped into ten subsets of the equal size. Every validation used nine subsets for training, and the other subset was applied as the testing dataset. This procedure was repeated ten times till every subset had been tested.

For impartially evaluating the effectiveness of the raised model for fishing vessel classification on AIS data, four common evaluation metrics, including \(Accuracy\), \(Precision\), \(Recall\), and \(F1\_Score\), were adopted to evaluate the predictive ability of our approach. Among them, \(Accuracy\) is one of the most normal assessment indicators, which is the proportion of the number of samples rightly classified by the model to the whole. \(Precision\) refers to the ratio of the number of positive samples obtained by the classifier to the true positive samples, while \(Recall\) reflects the number of rightly judged positive samples as the proportion of the total positive number. \(F1\_Score\) is a weighted average of \(Precision\) and \(Recall\). For the four metrics, the values range from 0 to 1, and the higher the value is, the more accurate the classification results obtained by the model. The relevant calculations are as follows:

where \(TP\), \(TN\), \(FP\), and \(FN\) respectively represent the numbers of true positive, true negative, false positive, and false negative samples.

The existing classification models

At present, many methods have been developed to extract the features and classify the types of vessels. We selected three classification models based on machine learning or deep learning algorithms that use AIS data in recent years to compare with the model proposed in this paper. Table 6 summarizes the feature description and classification algorithms of these methods and our proposed approach.

Features of fishing vessels

Due to the various operation ways of different types of fishing vessels, there will be distinctions in the latitude and longitude when fishing. To investigate the preferences of various types of fishing vessel trajectories, we selected one vessel with over ten thousand trajectory points for each category of the five kinds of vessels as examples, and plotted the map of fishing trajectory distribution (Fig. 5), which displaying apparent distinctions in the trajectories of various fishing vessel kinds.

Fishing trajectory of a gillnetter (a), hook and liner (b), trawler (c), fish carrier (d), and stow net vessel (e), where different colors represent different speed ranges.

Full size image

Specifically, the gillnetter generally operates far from port during fishing, and the trajectory is relatively straight and tight (Fig. 5a). Compared with gillnetter, hook and liner is more maneuverable and flexible, so the trajectory of a hook and liner is spatially discontinuous (Fig. 5b). In addition, a trawler is often towed in one direction during a single operation, and the trajectory has a continuous directional change over a short period of time. Therefore, the trawler’s trajectory spans a wide range of longitude (Fig. 5c). In general, fish carrier can move at a relatively fast speed. As a result, the trajectory of a fish carrier is relatively scattered, and the distribution ranges of longitude and latitude are wide (Fig. 5d). For stow net vessel, the fishing areas is typically far from port (Fig. 5e). The operation of a stow net vessel is performed by releasing the net and returning to the point of origin, so there may be a closed loop in the trajectory of a single ship.

The frequency histograms of speed and heading were plotted to deeply discover the differences in characteristics between different kinds of fishing vessels (Fig. 6). The speed frequency histograms of all kinds of fishing vessels show an obvious peak value at 0 knots, this is because fishing vessels float at ocean for break to save fuel¹⁹. Additionally, gillnetters, hook and liners, and trawlers present a trimodal distribution, fish carriers and stow net vessels present a bimodal distribution (Fig. 6a). Generally, after lowering drift gillnets, the unpowered gillnetters move at a relatively low speed of 2.8 knots under the effect of waves and wind. The situation may sustain from a few minutes to most of the day, depending on weather, waves, and the amounts of fish caught. The hook and liners usually fish at a speed of approximately 2.2 knots. The trawlers mostly catch fish at a speed about 2 knots by dragging nets. For the fish carriers, catches are generally transported at a speed of 9 knots. When finding a group of fish, the stow net vessels surround them at a speed of approximately 7 knots. Compared to speed, the heading frequency histograms of the five kinds of fishing vessels are more similar to a unimodal distribution (Fig. 6b).

Frequency histograms of speed (a) and heading (b) for gillnetters, hook and liners, trawlers, fish carriers, and stow net vessels.

Full size image

The speed and heading of five kinds of fishing vessels vary considerably depending on the different operational methods, so the mean, UQ, median, LQ, STD and COE of speed and heading have large differences (Table 7). Hook and liners are primarily impacted by wind and waves with small motion amplitudes when fishing, which makes the values of the mean, UQ, median, LQ, and STD smaller but the COE larger than the statistical values of other fishing vessels. Stow net vessels constantly change direction to arrange nets when fishing, so the values of the mean, UQ, median, and LQ are larger, while STD and COE values are smaller.

According to the differences in the characteristics of different types of fishing vessels, this article constructed a 58-dimensional mixed feature and ranked the importance of features using the RF and XGBoost algorithms, as shown in Fig. 7. In the results of the feature importance ranking, we found that the COE of the speed segment from 2 to 8 knots (\({F}_{33}\)) in static features was the most important feature for the classification model, followed by the area of fishing vessels (\({F}_{2}\)) in geometrical features and the COE of speed (\({F}_{21})\) in static features. The width of fishing vessels in geometrical features (\({F}_{1}\)), median of \(\Delta {lat}_{{y}_{t}-{y}_{t-1}}\) (\({F}_{55}\)) in dynamic features, and the mean of latitude (\({F}_{10})\) in static features also cannot be Ignored. Fishing vessels in this speed segment from 2 to 8 knots are usually fishing, which indicates that there are remarkable differences in the features of various sorts of fishing vessels during operation. Moreover, the area and position of fishing vessels are also important characteristics.

Ranking of feature importance by using XGBoost (a) and RF (b) algorithm.

Full size image

To ascertain the number of removed features to obtain the best result of the classification model, RF and XGBoost algorithm were utilized to calculate the model’s accuracy for each feature deleted. The evaluation results of RF and XGBoost algorithms when deleting different numbers of features are presented in Fig. 8. The results from the XGBoost algorithm (Fig. 8a) reveal that the model achieves its best classification performance by excluding the last 12 features (\({F}_{45},{F}_{48},{F}_{38},{F}_{39},{F}_{44},{F}_{31},{F}_{32},{F}_{35},{F}_{27},{F}_{30},{F}_{25},{F}_{24}\)). The RF algorithm (Fig. 8b) indicates that removing the bottom 10 features (\({F}_{48},{F}_{39},{F}_{45},{F}_{38},{F}_{35},{F}_{27},{F}_{25},{F}_{24},{F}_{32},{F}_{44}\)) results in optimal classification model performance. However, eliminating 12 features with XGBoost yields better classification results than removing 10 features with RF, and employing fewer features can boost the model training speed. Thus, following the feature importance ranking from XGBoost, 12 features were deleted, and 46 features were selected for training the vessel classification model.

The score of selecting different numbers of features by using XGBoost (a) and RF (b) algorithm.

Full size image

Performance of BiLSTM-CNN-attention algorithm

To assess the effectiveness of the BiLSTM-CNN-Attention model, the tenfold CV was used on the training dataset and the results of each CV and overall average were recorded (Table 8). The BiLSTM-CNN-Attention model based on multiple features achieved excellent performance and showed great stability in 10 trainings, which the average of \(Accuracy\), \(Precision\) and \(Recall\) reached 98.33% and the average of \(F1\_Score\) reached 98.32%. It suggests that the extracted geometrical, static and dynamic features can effectively reflect the differences in behavior between the five types of fishing vessels, and the BiLSTM-CNN-Attention model can effectively select important features that have great impacts on the classification model to obtain more accurate classification results.

Performance evaluation on different algorithms

Because the parameters of individual models may differ during optimization, for a fair comparison, we conducted tenfold CV for each single models on the training dataset, and the comparison results are displayed as boxplots (Fig. 9). The proposed BiLSTM-CNN-Attention ensemble model based on multiple features provided the best performance on the training dataset compared to the other six base classifiers. All the average values of \(Accuracy\), \(Precision\), \(Recall\), and \(F1\_Score\) of the 10 training models were more than 98%, which illustrated that the model had strong predictive capability. The 10 training results of the BiLSTM-CNN-Attention model had a small difference in values, which proved that it had great stability. For the other three deep learning algorithms, CNN obtained better results in evaluation indexes but poor stability compared with BiLSTM and BiGRU. Although machine learning methods are relatively stable, the overall performance are lower than deep learning methods, especially SVM, which exhibited the lowest performance, indicating that deep learning algorithms have better learning abilities in multiclassification tasks with a large amount of data.

Performance comparisons of different classifiers on the training dataset by using the tenfold CV. (a) Accuracy; (b) Precision; (c) Recall; (d) F1 Score.

Full size image

Confusion matrixes of BiLSTM-CNN-Attention and six base classifiers on the testing set were plotted as heatmaps (Fig. 10). The abscissa of the heatmap represents the true type of fishing vessel, while the ordinate represents the type predicted by the model. So, the numbers in the diagonal squares of the confusion matrix represent the count of correct predictions made by the base classifiers, while the numbers in the other squares indicate the count of instances where one type was erroneously predicted as another type.

Confusion matrixes of different classifiers on the testing set. (a) SVM; (b) RF; (c) XGBoost; (d) CNN; (e) BiLSTM; (f) BiGRU; (g) BiLSTM-CNN-Attention.

Full size image

To confirm the robustness of the presented model, an independent test was performed on the testing dataset, and the results are shown in Table 9. The raised model achieved the best performance in terms of \(Accuracy\) (91.90%), \(Precision\) (92.22%), \(Recall\) (91.90%), and \(F1\_Score\) (91.36%). The performance of RF is second only to the BiLSTM-CNN-Attention model in \(Accuracy\) (90.20%), \(Precision\) (91.19%), \(Recall\) (90.20%), and \(F1\_Score\) (90.34%), while CNN, whose performance was second in the training set, performed poorly in the independent testing set. This is due to the instability of the CNN model and suggests that the stability of the model is one of the major factors to assess the quality of the model. In addition, the results of BiLSTM and BiGRU are similar and better than those of SVM and XGBoost.

Performance comparison with existing methods

To further validate the advantages of BiLSTM-CNN-Attention in classifying types of fishing vessels, we compared with the existing methods on the same independent testing dataset. A comparison is made across key metrics including \(Accuracy\), \(Precision\), \(Recall\), and \(F1\_Score\), with detailed results presented in Table 10. As illustrated in Table 10, the proposed BiLSTM-CNN-Attention model achieved the best performance across metrics in terms of \(Accuracy\), \(Precision\), \(Recall\), and \(F1\_Score\). In comparison to alternative methods, our approach extracts a more holistic set of features from the geometrical, static, and dynamic vessel trajectories, facilitating enhanced learning of pattern distinctions among various vessel types. Although our model extracts the highest dimension of feature vectors, it still exhibits the best performance on each metric, indicating the strong scalability of our model. In contrast to single or integrated machine learning algorithms, ensemble deep learning models exhibit superior learning capabilities and further improve classification accuracy.

In this study, we put forward an ensemble deep learning method called BiLSTM-CNN-Attention, which was a novel and high-precision five-classification approach for fishing vessels by using AIS data in China’s offshore waters from 2018 to 2022. The objective is to provide an effective alternative that combining some of the advanced deep learning algorithms and creating a general method to classify fishing vessels. The primary innovations of our model are as follows. First, the geometrical (length, width, area, ratio of a fishing vessel), static (longitude, latitude, speed, speed segments and heading) and dynamic (\(\Delta time\), \(\Delta {lot}_{{x}_{t}-{x}_{t-1}}\), and \(\Delta {lat}_{{y}_{t}-{y}_{t-1}}\)) features of fishing vessels were combined to better capture feature differences between gillnetters, hook and liners, trawlers, fish carriers, and stow net vessels. Second, the integration of two powerful deep learning models, BiLSTM and CNN with the addition of an attention mechanism, can speed up the operation speed and effectively improve the classification accuracy by extracting important features. The experiment results show that the proposed model is highly efficient and can be widely used in the classification of fishing vessels in the tenfold CV and the independent test based on the benchmark dataset. The results proved the applicability of the proposed approach in this study for the identification of five types of fishing vessels. We hope that our method will be effectively applied to the prediction of more types of fishing vessels and to more sea areas.

AI algorithms have already achieved great success in various fields, and it is necessary to enhance the ability to utilize AIS data for maritime surveillance with the help of AI. The deep mining and analysis of AIS big data using AI algorithms contribute to protect the ecological environment, improve production efficiency, understand the availability of fishery resources, and achieve sustainable development of marine living resources⁴⁶. Nonetheless, in the actual management of fisheries, the lack of quantity and reliability of relevant data often makes it difficult for fishery production to achieve the desired results. We need to complete sustainable fisheries management and assessment through the utilization of appropriate data in the context of extremely limited data⁴⁷. Consequently, it is very important to use the latest machine learning and deep learning algorithms to continuously improve the ability to obtain and analyze data to promote the intelligent development of fisheries.

In summary, the results indicated that the BiLSTM-CNN-Attention model achieved superior performance and better than other base classifiers on both the training dataset and the independent testing dataset. This model is a powerful tool to accurately classify the various types of fishing vessels and could be widely applied in a variety of fishing vessel classification tasks. The comparative experimental results of different algorithms in this paper demonstrate that, compared with machine learning algorithms, deep learning algorithms show superior performance in large-scale datasets, and integrated deep learning algorithms can obtain greater performance improvements²⁰. Specifically, SVM classifier by determining the separation hyperplane with the largest geometric interval⁴², so multi-classification and large-scale training samples are difficult to implement. Thus, the ability of SVM to classify fishing vessels in this paper is limited. Besides, both RF⁴³ and XGBoost⁴⁴ train samples in parallel using multiple decision trees and can focus on the influence of important features, so they can perform better than SVM on AIS big data. Among deep learning algorithms, CNN, BiLSTM, and BiGRU are the more advanced and commonly used predictive models. These algorithms can extrapolate new features from a finite set of features and show a strong ability to learn the essential features in classification task. However, the single base classifier cannot completely recognize the feature distinctions of various kinds of fishing vessels, resulting in poor robustness of models⁴⁸ in this research. Especially for the CNN model, which exhibited poorer stability on the independent testing set. This may be because CNN is generally more suitable for image recognition tasks, which pays more attention to the structural features and does not capture sequence dependencies as effectively as BiLSTM and BiGRU. The classification model proposed in this paper, which integrates BiLSTM and CNN algorithms with an attention mechanism, further enhances generalization and improves the accuracy of the fishing vessel classification. The reason why this model may outperform other single models could be attributed to two factors. On one hand, the ensemble algorithm combines the advantages of a single algorithm and reduces the risk of model overfitting. On the other hand, the attention mechanism can focus the limited attention of the model on the key features, thereby saving resources and quickly obtaining the information that is most effective for the classification model⁴⁹. In addition, it is worth noting that all models tended to confuse gillnetters as trawlers and confuse stow net vessels as trawlers in confusion matrixes (Fig. 10). There were two possible reasons for this misclassification. Judging from the behavior of the fishing vessels, it was suspected that these vessels were illegally catching fish. For the model, because the amount of trawler data was the largest, more features were obtained, and when the model cannot extract accurate trajectory features, it would be biased to predict the trawler.

The performance of machine learning classification algorithms is closely linked to the construction of feature vectors. One of the primary reasons for the ideal accuracy of a classification model is the presence of comprehensive, accurate, and moderately sized feature vectors as inputs—in essence, the key advantage of the first innovation proposed in this article. Moreover, while the ensemble deep learning model merely surpasses a single classification model by around 2% in accuracy, its additional advantages warrant consideration. Firstly, despite the greater complexity of ensemble models, they do not impose high demands on computational performance, and the training environment required is the same as that of a single classification model. Secondly, when handling vast datasets on the scale of millions, ensemble deep learning models can grasp intricate data features, capture latent patterns and correlations within datasets, making them particularly adept at managing high-dimensional and nonlinear data. Most importantly, ensemble models can compensate for the limitations of single models, their generalization capabilities typically exceed those of singular models, suggesting that they may perform more reliably and proficiently when faced with new data in diverse scenarios. The BiLSTM-CNN-Attention algorithm can extract critical information from multidimensional features, effectively distinguish the behavioral differences between different types of fishing vessels, and further improve the calculation rate and classification accuracy. It will provide more efficient classification algorithms for marine spatial planners, managers and the public.

As an emerging technology integrating remote sensing, modern communication, computer and electronic information technology, AIS is not affected by meteorological and sea conditions, has wide signal coverage, high communication reliability, and strong stability of tracking ships. The original purpose of AIS was to avoid collisions, know the position of vessels in real time, and assist in seek and save operations as well as the execution of fisheries regulations⁵⁰. In recent years, with the extensive application of AIS and the constant development of data processing methods by AI, AIS data have become an indispensable portion of fishery big data, which has received widespread concern from academic circles in relevant domestic and overseas areas⁵¹ and has been increasingly applied to various aspects of fisheries. However, while the popularization and use of AIS brings convenience to the fishery, there are also some limitations in the equipment itself and in the process of use⁵². When the operation area of fishing vessels is concentrated, a large number of signals appear in the same range, which may cause the communication road of the AIS to be blocked and unable to communicate in time, resulting in collision accidents. In the context of big data, AIS is constantly updated and improved, and its functions are more stable and mature. It is believed that in the near future, AIS will overcome difficulties, better serve the fishery, and be applied to the construction of fishery informatization on a larger scale.

Vessel monitoring contributes to strengthening ship management, fisheries regulation, marine ecological sustainability, and maritime safety protection, and the assignment of identifying the kinds of fishing vessels is becoming increasingly important⁵³. AIS data have proofed to be a beneficial source of big data in the field of ship behavior analysis. It is important to automatically identify the movement patterns of vessels in real time from AIS data, as it can reveal unusual or illegal vessel activities at appropriate times. Recently, many researchers have focused on the monitoring of fishing vessels by using AIS data, especially the classification of fishing vessels, to achieve effective fisheries management. For example, Guan et al.¹⁹ classified three kinds of vessels based on AIS data in the northern South China Sea to effectively manage multiple types of fishing vessels. The temporal and spatial distribution of vessels, the duration of fishing and other behavior patterns in various seasons were discussed. This study provides support for the progress of fishery resource and marine security governance in northern China. Additionally, to further realize global maritime ship monitoring, Yan et al.¹⁴ selected four main categories of maritime vessels as the objects of study, namely tanker, fishing, cargo, and passenger ships and proposed a ship identification model based on multi-classifier integrated learning by using global AIS data, which improved the availability of maritime vessel monitoring to a certain extent. The results of ship classification reveal patterns and differences in ship behavior in a specific area. Based on machine learning or deep learning technology, the developed classifiers can be used to identify the behavioral characteristics and types of ships. It will lay the basis for future scientific research on the operational behavior of different types of fishing vessels.

However, this study is based on AIS data of Chinese coastal fishing vessels, and the multi-dimensional features of the extracted fishing vessels include longitude and latitude information, and there are differences in fishing vessel operation locations, target species, technologies, and strategies among different countries or regions, which affect the performance of our model in other areas, thus limiting the generalization ability of the model. Furthermore, our analysis is confined to five types of vessels, whereas the actual types engaged in fishing activities at sea are more diverse, and an increase in vessel types might also impact the accuracy of identification. Our research primarily focuses on feature extraction and training of AIS data from fishing vessels within a specific longitude and latitude range to more accurately identify vessel types in the area, aiding in controlling and managing fishing vessel activities in specific marine regions. In the future, this model can be used to identify illegal or unauthorized vessel activities in marine protected areas or exclusive economic zones, assisting fisheries management authorities in monitoring vessel activities to ensure compliance with fishing regulations and sustainable resource utilization. In the next steps, we intend to collect AIS data from various types of fishing vessels globally or combine it with other data sources, such as satellite imagery. By utilizing deep learning algorithms for feature extraction from image data, we aim to mine more fishing vessel characteristics from comprehensive perspectives and train a multi-classification model, so as to identify broader range of marine areas and more varieties of fishing vessels. In the context of big data, more and more AI technologies have made breakthroughs and innovations, we consider trying to integrate other advanced AI algorithms, such as transformer and efficient neural network, to further enhance the generalization and robustness of the fishing vessel classification model. With the help of AI to provide guidance for fishery management and resource exploitation, smart fisheries will usher in broader development prospects.