Computational Intelligence In Optical Remote Sensing Image Processing.pdf

Accepted Manuscript Title: Computational intelligence in optical remote sensing image processing Authors: Yanfei Zhong, Ailong Ma, Yew soon Ong, Zexuan Zhu, Liangpei Zhang PII: DOI: Reference:

S1568-4946(17)30708-1 https://doi.org/10.1016/j.asoc.2017.11.045 ASOC 4589

To appear in:

Applied Soft Computing

Received date: Revised date: Accepted date:

11-4-2016 17-11-2017 29-11-2017

Please cite this article as: Yanfei Zhong, Ailong Ma, Yew soon Ong, Zexuan Zhu, Liangpei Zhang, Computational intelligence in optical remote sensing image processing, Applied Soft Computing Journal https://doi.org/10.1016/j.asoc.2017.11.045 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Computational intelligence in optical remote sensing image processing

State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote

SC R

1

IP T

Yanfei Zhong1, Ailong Ma1,*, Yew soon Ong2, Zexuan Zhu3, Liangpei Zhang1

Sensing, Wuhan University, P. R. China

School of Computer Engineering, Nanyang Technological University, Singapore

3

College of Computer Science and Software Engineering, Shenzhen University, P. R.

U

2

A

N

China

A

CC E

ED

PT

Graphical abstract

M

* Corresponding author E-mail: [email protected]

The general workflow of optical remote sensing image processing.

Highlights We review the computational intelligence in optical remote sensing image processing.



Feature representation and selection based on computational intelligence are reviewed



Classification using in evolutional computation and neural networks are reviewed.



Change detection based on computational intelligence are reviewed.



The core potentials of computational intelligence for optical remote sensing image

SC R

processing are discussed.

IP T



Abstract—With the ongoing development of Earth observation techniques, huge

U

amounts of remote sensing images with a high spectral-spatial-temporal resolution are

A

N

now available, and have been successfully applied in a variety of fields. In the process,

M

they bring about great challenges, such as high-dimensional datasets (the high spatial resolution and hyperspectral features), complex data structures (nonlinear and

ED

overlapping distributions), and the nonlinear optimization problem (high computational

PT

complexity). Computational intelligence techniques, which are inspired by biological systems, can provide possible solutions to the above-mentioned problems. In this paper,

CC E

we provide an overview of the application of computational intelligence technologies in optical remote sensing image processing, including: 1) feature representation and

A

selection; 2) classification and clustering; and 3) change detection. Subsequently, the core potentials of computational intelligence for optical remote sensing image processing are delineated and discussed. Index—optical remote sensing image processing, computational intelligence, evolutionary algorithm

Keywords: optical remote sensing; image processing; computational intelligence; evolutionary algorithm; classification

IP T

1. Introduction

SC R

Remote sensing is an important Earth observation technique that is able to acquire remote sensing images and obtain object information without making physical contact,

U

through the use of sensors on satellites or aircraft [1, 2]. Optical remote sensing imaging,

N

in particular, is a major branch of remote sensing, whose products have been widely

A

used in many real-world applications, such as global land-cover mapping [3],

M

vegetation monitoring [4], water quality monitoring [5], urban climate and

ED

environmental studies [6], detection of forest fires [7], mineral exploration [8], oil spill detection [9], and precision agriculture [10].

PT

Optical remote sensing images can be characterized by three resolutions, namely: 1)

CC E

the spatial resolution [11] (e.g. QuickBird, IKONOS, and the Chinese Gaofen-1 system); 2) the spectral resolution [11] (e.g. Hyperion, AVIRIS, HYDICE, and ROSIS); and 3) the temporal resolution [11] (e.g. MODIS). The availability of remote sensing images

A

with high spatial-spectral-temporal resolutions provides great potential for the development of remote sensing data processing, including: 1) the feature preprocessing (e.g. feature representation and hyperspectral band selection); and 2) the specific applications (e.g. supervised and unsupervised classification, and change

detection). Unfortunately, the conventional remote sensing image processing methods struggle to handle the new challenges brought by the problem complexity, including both the data complexity and the model complexity. 1) “Data complexity” means that the spatial-spectral-temporal resolutions of the remote

IP T

sensing images have become higher. Both the data volumes and dimensionality have significantly increased, and the data distribution in the feature space has become more

SC R

complex and sparse. The commonly used Gaussian distribution cannot model such remote sensing data very well. Thus, the traditional approaches cannot work well on

U

certain tasks in remote sensing image processing (e.g. classification [12] and change

N

detection [13]). Because some of the traditional approaches (e.g. the maximum

A

likelihood classifier (MLC) and support vector machine (SVM)) transform such

M

problems into a classification or regression problem and resolve them by using known

ED

training samples to predict the corresponding attributes (e.g. the class label or change label), they can only achieve a satisfactory performance under certain assumptions (a

PT

normal or other distribution) or conditions (a small number of training samples).

CC E

However, the data complexity makes it difficult for these assumptions or conditions to be satisfied. Therefore, the data complexity raises new challenges for remote sensing image processing.

A

2) “Model complexity”. In order to deal with the data complexity, different models have been designed with both a powerful optimization capability and the ability to handle multi-objective problems. On the one hand, certain tasks of remote sensing image processing can be transformed into a continuous optimization problem (e.g. clustering

[14]) or a knapsack problem (e.g. hyperspectral band selection [15], endmember extraction [16], and change detection [17]), which is a representative NP-hard problem. Although the traditional optimization methods such as the mountain climbing based methods are time-efficient in remote sensing image processing, they are sensitive to the

IP T

initial values, and it is easy for them to get stuck in local optima. On the other hand, in certain remote sensing image processing tasks (e.g. clustering [18] or hyperspectral

SC R

unmixing [19, 20]), as a result of the uncertainty of the data structure, differently designed objective functions are often conflicting and achieve different performances

U

on different remote sensing images, which can dramatically influence the generalization

N

capability of the traditional approaches. Therefore, the newly designed model, which

A

needs to take multiple objective functions into consideration, becomes complex.

M

Accordingly, different objective functions are often combined into a single objective

ED

function with the help of a regularization parameter. However, the determination of the regularization parameter is not an easy task. Therefore, the model complexity raises

PT

other new challenges for remote sensing image processing.

CC E

The remaining parts of this paper are organized as follows. Section II describes the potential of computational intelligence in optical remote sensing image processing. Section III reviews the workflow of remote sensing image processing and the

A

applications of computational intelligence in the different fields of optical remote sensing image processing, including: 1) feature representation and hyperspectral band selection; 2) classification and clustering; and 3) change detection. Section IV provides the conclusion and specifies some future research directions.

2. The potential of computational intelligence in optical remote sensing image processing Computational intelligence has become one of the most effective tools for handling

IP T

the complexities and uncertainties in remote sensing image analysis. Computational intelligence techniques inspired by the evolutionary mechanism of biological systems,

SC R

the neural mechanism in the brain, or human reasoning, include artificial neural networks (ANNs), evolutionary algorithms (EAs), and fuzzy logic. In this paper, our

U

focus is on ANNs and EAs, because of their merits and wide use in resolving the

N

problems brought by the data complexity and model complexity in remote sensing

Robustness to the complexity of remote sensing data

M

2.1

A

image processing.

ED

Computational intelligence techniques can help us to discover new knowledge (e.g. band subsets, cluster centers, change detection maps, etc.) from the original data. This

PT

knowledge can then be used to refine the original data. Finally, the refined data can be

CC E

used to further acquire knowledge. In this process, as shown in Fig. 1, the learned knowledge can be fitted with the original data step by step, without any assumption of the data distribution, making the EA- and ANN-based methods robust to the

A

characteristics of remote sensing data. As a result, these methods are characterized by their ability to self-learn from the training samples, without any assumption about the data, and they are able to cope with the data complexity in remote sensing image processing. For example, in the artificial immune system (AIS) based classifiers for

remote sensing imagery [12], the memory cells (representing the class centers) develop automatically, based on the antigens (representing the training samples), without any constraints. Computational intelligence techniques have been used in remote sensing image classification [12, 21-26] and change detection [13, 27-33], due to their

Powerful global search capability

SC R

2.2

IP T

robustness to the complexity of remote sensing data.

EAs and swarm intelligence methods originate from the evolutionary mechanism of natural species (genetic algorithms (GAs), evolutionary strategies (ESs), genetic

U

programming (GP)) and other social behaviors of natural species (particle swarm

N

optimization (PSO), ant colony optimization (ACO)), or some other biological-based

M

A

system (AIS). The general framework of the EA-based methods is shown in Fig. 2 and includes: 1) population initialization; 2) stochastic operators; and 3) final solutions. The

ED

process of individual encoding in the population initialization is used to transform the

PT

original problem (e.g. band selection or clustering) into a population optimization problem, in which the individuals can represent the different kinds of solutions (e.g.

CC E

band subsets or cluster centers). In the objective space shown in Fig. 2(d), we can see that the population initialization can generate many solutions with different objective

A

function values, which greatly increases the available search space, giving the EAbased methods a powerful global search capability. In the stochastic operator process, different operators can be applied in different EAs, such as GAs, differential evolution (DE), etc. By using the operators shown in Fig. 2(b), the solutions in the objective space shown in Fig. 2(e) are pushed to the minimum. The stochastic operators can further

enhance the global search capability, as well as guarantee the convergence. Finally, the best solution in Fig. 2(f) can be acquired, i.e. the minimum in the objective space. Because of the unique combination of stochastic operators and the fact that they can maintain their working memory in the form of a population of candidate solutions, EAs

IP T

guarantee a powerful global search capability [34]. As a result, EAs have been widely used in remote sensing image processing, including combinatorial optimization

SC R

problems such as hyperspectral band selection [15, 33, 35-46], endmember extraction [16, 47], sub-pixel mapping [48], and change detection [17], and continuous

Capability of handling multi-objective optimization

N

2.3

U

optimization problems such as clustering [18, 49-55].

M

A

problems

The goal of multi-objective optimization is to find a set of solutions which can take

ED

all the involved objective functions into consideration and achieve a tradeoff between

PT

them [56]. Traditionally, multi-objective optimization problems are usually transformed into single-objective optimization problems using the weighted sum of all

CC E

the objective functions as a target function. However, the balance between the different objective functions must be controlled manually, which can introduce bias into the

A

solutions. Because of their population-based property, EAs have been widely used for resolving multi-objective optimization problems, and are referred to as “multi-objective evolutionary algorithms” (MOEAs) [56]. One important concept in MOEAs is “Pareto dominance”, which is a binary relation between two solutions. One solution is Pareto dominant with respect to another solution if, for all objectives, this solution improves

on the other solution. If Solution 1 is better than Solution 2 for one objective, and worse for another objective, these two solutions are non-dominated with respect to each other. The framework of evolutionary multi-objective optimization algorithms also consists of: 1) population initialization; 2) stochastic operators; and 3) final solutions, as shown

IP T

in Fig. 3. Compared with the single-objective optimization process shown in Fig. 2, the final solutions in the multi-objective optimization process shown in Fig. 3(f) consist of

SC R

several possible solutions, instead of the single solution in Fig. 2(f), due to the fact that

two candidate solutions need to be compared using two or more conflicting objective

U

functions. Thus, if these objective functions conflict with each other, many non-

N

dominated solutions can be generated. Therefore, in the feature space shown in Fig.

A

3(d)–(f), the true solutions of the multi-objective optimization algorithms lie in one

ED

problem shown in Fig. 2(f).

M

specific manifold [57], instead of one single optima in the single-objective optimization

In the population initialization process, many different candidate solutions can be

PT

generated, spreading across the whole feature space shown in Fig. 3(d) and generating

CC E

different fronts in the objective space shown in Fig. 3(g), which gives the process a powerful global search capability. Meanwhile, the stochastic operators, which increase the global search capability and guarantee convergence, can push the candidate

A

solutions further towards the solution manifold. We can also see in the objective space shown in Fig. 3(h) that the stochastic operators can push the candidate solutions towards a better Pareto front. Finally, the non-dominated solutions can be acquired that lie along this manifold, as shown in Fig. 3(f), and the true Pareto front, as shown in Fig. 3(i).

We can see that in both the feature space and the objective space, the evolutionary multi-objective optimization methods can take several objective functions into consideration simultaneously, giving them intrinsic superiority in resolving multiobjective optimization problems. As a result of their rapid development [58, 59], the

IP T

potential of these methods in resolving the model complexity of remote sensing image processing is attracting more and more attention, in applications such as multi-objective

SC R

hyperspectral band selection [45], multi-objective clustering [60-63], multi-objective

U

change detection [64, 65], and multi-objective hyperspectral unmixing [19, 20].

N

In order to quantitatively show the applications of computational intelligence in

A

remote sensing image processing, we counted the number of papers on the different

M

applications over the last two decades. Firstly, two novel EAs—DE and PSO—were

ED

selected as representative methods. We then searched for the keywords of “remote sensing”, “differential evolution”, and “particle swarm optimization” in the published

PT

papers. Secondly, back-propagation neural networks (BPNNs) and deep learning were

CC E

used as representative methods. We then searched for the keywords of “remote sensing”, “back-propagation neural network”, and “deep learning” in the published papers. As can be seen in Fig. 4, the number of published papers on the application of

A

computational intelligence in remote sensing image processing has increased significantly in recent years.

3. The application of computational intelligence in optical remote sensing image processing This paper reviews the different applications of computational intelligence in optical

IP T

remote sensing imaging processing. The review is presented according to the basic workflow of remote sensing image processing shown in Fig. 5. The feature pre-

SC R

processing, which can produce more efficient and robust features for the corresponding

tasks, is the first step in remote sensing image processing. However, feature pre-

U

processing is a non-trivial task for the following reasons. Firstly, the advanced

N

applications of remote sensing (e.g. accurate object recognition or scene classification)

A

need to distinguish the critical fragment of a spectral curve or embed the semantic

M

information of remote sensing data. Secondly, hyperspectral remote sensing images

ED

may have hundreds of features (bands). However, the bands are correlated, resulting in redundant information for the remote sensing image processing task. Therefore,

PT

hyperspectral band selection is used to select an informative (but not redundant) band

CC E

subset. Based on the above analysis, feature representation and band selection (which we focus on in Section 3.1) are the two main tasks of feature pre-processing for remote sensing images. After the feature pre-processing, the features can be used in the other

A

fields of remote sensing image processing, such as classification, which involves separating the remote sensing image into a number of homogeneous regions. Classification can be categorized into supervised classification and unsupervised classification (clustering). Supervised classification is more commonly used because it

can acquire a more satisfactory result by the use of training samples. However, for remote sensing images, it is often time-consuming and expensive to acquire sufficient training samples for supervised classification. Hence, unsupervised classification is attracting more and more attention. In Section 3.2, we review the applications of

IP T

computational intelligence in remote sensing image classification. Change detection is an advanced technique which is used to extract land-cover change information from the

SC R

same area by analyzing images taken at different times. Change detection from remote

sensing images has made significant progress and now plays an important role in many

U

fields, such as disaster evaluation and urban growth analysis [66]. Based on whether or

N

not training samples are used in the change detection process, change detection

Feature pre-processing of remote sensing images

ED

3.1

M

are described in Section 3.3.

A

techniques can be categorized into unsupervised and supervised approaches [67], which

PT

3.1.1 Feature representation of remote sensing images

CC E

Sensors onboard satellites can record the reflectance of objects on the Earth’s surface at a specific range of the wavelength, which constitutes the basic feature of remote

A

sensing images, namely, the spectral feature (see Fig. 6(a)). In this way, each pixel is characterized by a feature vector. However, remote sensing images with a higher spatial resolution contain a lot of detailed information about the objects on the ground, meaning that the neighboring pixels tend to be labeled as the same class. Thus, it is insufficient to only use the spectral feature, and other feature descriptors such as the

gray-level co-occurrence matrix (GLCM) are often used to enhance the classification or change detection performance [68] (see Fig. 6(b)). What is more, with the ongoing development of remote sensing technology, the demand for advanced applications has become increasingly pressing, resulting in the development of two intelligent feature

IP T

types: DNA encoding features [69-71] and learned features [72]. 1) Artificial DNA encoding. Artificial DNA encoding is inspired by the fact that most

SC R

of the DNA sequences of different species are highly similar, and only tiny differences can effectively distinguish them. Similarly, in remote sensing images, many objects

U

have similar spectral characteristics in most of the spectral curve, whereas only a small

N

range of the spectral curve is different. The traditional classification methods do not

A

function well in distinguishing such objects. The artificial DNA encoding technique

M

was thus proposed to transform the original spectral feature into DNA-like symbols (i.e.

ED

A, G, C, and T) (see Fig. 6(c)), which can represent the spectral curve more efficiently and can capture the subtle feature variation in the spectral curve. Moreover, the

PT

matching between different patterns (pixels) is more convenient and has less

CC E

computational burden, thanks to the existence of only four symbols. Artificial DNA computing has been successfully applied to hyperspectral remote sensing image classification (artificial DNA computing based spectral matching (ADSM [69])),

A

clustering (unsupervised spectral matching based on artificial DNA computing (UADSM [70])), and semi-supervised classification (the semi-supervised subspacebased DNA encoding and matching classifier (SSDNA [71])). In [69], the hyperspectral remote sensing image was first transformed into a DNA cube composed of the four

symbols. Subsequently, the DNA cube was used to generate the spectral library and classify the hyperspectral remote sensing image. In [70], similarity and heterogeneity tests were proposed for the DNA cube to generate new cluster centers, which were used to cluster the hyperspectral remote sensing image. In [71], in order to resolve the lack

IP T

of training samples problem, the unlabeled samples were fully utilized to enhance the training samples, and an artificial DNA computing based semi-supervised classification

SC R

method was proposed.

2) Learned features. Recently, with the development of deep neural networks (i.e.

U

deep learning), unsupervised feature learning has become a widely used technique, in

N

which the different levels of features can be learned one by one (see Fig. 6(d)). These

A

features include the traditional low-level features (e.g. corners, edges, etc.). In addition,

M

low-level features such as semantic information (e.g. the social attributes) can also be

ED

embedded in these learned features [73, 74]. One main application of these learned features in remote sensing image processing is scene classification, where the objective

PT

is to determine the social attributes rather than the physical attributes of regions [72].

CC E

The auto-encoder was the first deep learning method introduced to remote sensing image scene classification [75]. The method proposed in [75] combines a sample selection strategy based on saliency with the auto-encoder to extract features from the

A

raw data, and can achieve a better performance than the traditional remote sensing image scene classification methods. In [76], the feature selection problem in a deep belief network (DBN) was transformed into a feature reconstruction problem, in which the features learned by the DBN are removed if their reconstruction errors exceed the

threshold. In [77], a sparse strategy was applied to the auto-encoder to ensure both population sparsity and lifetime sparsity, and the approach was named the “enforcing lifetime and population sparsity” algorithm. In addition, convolutional neural networks (CNNs) have also been widely used as deep learning methods for remote sensing image

IP T

scene classification. In [78], multiple CNN models were combined based on boosting theory, achieving a better performance than the auto-encoder methods. Furthermore,

SC R

deep learning has shown great potential in other fields of remote sensing image

processing [79], such as auto-encoder based [80], DBN based [81], and multiscale CNN

U

based [82] hyperspectral classification, change detection [83], and object detection (e.g.

N

airplane detection [84], vehicle detection [85], [86], road network extraction [87], and

M

A

ship detection [88]).

ED

3.1.2 Hyperspectral band selection Due to the high computational burden of hyperspectral remote sensing images and

PT

their sensitivity to the initialization, the traditional hyperspectral band selection

CC E

methods based on exhaustive search or heuristic search (e.g. sequential forward selection (SFS) [89], sequential backward selection (SBS) [90], the sequential forward floating selection (SFFS) technique, and the sequential backward floating selection

A

(SBFS) technique [91]) do not perform well. Computational intelligence has been widely used to solve the inherent optimization problem of hyperspectral band selection by maximizing the learning performance with the selected band subset. Computational intelligence based band selection methods often share similar individual representations,

as shown in Fig. 7, where a binary encoding scheme is used, with “1” (“0”) indicating the selection (exclusion) of the corresponding band. Computational intelligence based band selection methods for hyperspectral remote sensing images transform the band selection problem into a combinatorial optimization problem, in which some specially

IP T

designed objectives, such as maximizing the classification accuracy or minimizing the number of selected features, are used as fitness functions of the individuals in the whole

SC R

population. The best band subset can then be obtained through evolution of the population.

U

In Fig. 8, some of the representative computational intelligence based hyperspectral

N

band selection methods are depicted in a chronological manner. Depending on whether

A

or not a learning algorithm is used in the process of band selection, hyperspectral band

M

selection methods can be categorized into two groups, namely, wrapper and filter

ED

methods. Wrapper methods use the training classification accuracy of a specific algorithm (e.g. SVM) as the fitness function to evaluate the “goodness” of the selected

PT

band subset. In [15], a GA-based wrapper band selection method was first applied to

CC E

hyperspectral remote sensing images. Subsequently, various computational intelligence methods, including GAs [15, 37, 38], PSO [92], and ACO [46], have been used for band selection in a wrapper way to acquire the band subset of the hyperspectral remote

A

sensing images. Among these methods, in [38], an SVM classification system was designed, in which a GA is used to detect the most distinctive bands of the hyperspectral remote sensing images and to estimate the SVM parameters. In addition, memetic algorithms, such as the combination of a GA and PSO proposed in [39], have also been

used for hyperspectral band selection, due to their balanced optimization capability. In addition to selecting relevant bands, it is also helpful to know the contribution of each selected band. Hence, it is reasonable to select a suitable band subset and balance the weights of the bands. In [35], Zhang et al. proposed the clonal selection feature-

weight of each band is added into the encoding of the individuals.

IP T

weighting (CSFW) algorithm for hyperspectral remote sensing imagery, in which the

SC R

However, such wrapper methods consume a large amount of time when searching

for an optimal band subset. Therefore, the filter approach is preferable in the task of

U

band selection for hyperspectral remote sensing images as it is independent of the time-

N

consuming classification procedure. Unlike the wrapper-based methods, filter-based

A

band selection methods tend to use time-efficient objective functions based on the

M

statistical characteristics of the remote sensing data. Filter-based band selection

ED

methods include information theory based [36, 40, 41], graph-based, minimum estimated abundance covariance (MEAC) based [42], and Fisher’s ratio based [43]

PT

objective functions. Among these criteria, the information theory based methods are

CC E

widely used. In [44], a semi-supervised mutual information based band selection criterion for hyperspectral remote sensing images was proposed to select highly discriminative and informative band subsets by the use of both limited labeled samples

A

and a sufficient number of unlabeled samples, and a novel clonal selection algorithm (CSA) was proposed to optimize the proposed criterion. More recently, multi-objective optimization algorithms have shown great potential in hyperspectral band selection. In [45], the task of hyperspectral band selection was formulated as a multi-objective

optimization problem, in which one objective function is considered as the information entropy, to represent the information in the band subsets, and the other is designed as the number of selected bands. In addition, in [61], Paoli et al. presented a new methodology for the band selection and clustering of hyperspectral images within a

IP T

multi-objective PSO framework, in which the Bhattacharyya distance is used as one of

3.2

Remote sensing image classification

U

3.2.1 Supervised classification

SC R

the objective functions for the band selection.

N

Traditionally, for remote sensing images, the statistical learning based supervised

A

classification methods, such as MLC, have achieved good performances based on

M

certain conditions or assumptions (e.g. the Gaussian distribution of the data). However,

ED

as a result of the complexity of remote sensing images, these assumptions may not always be true. The SVM classifier can perform well with a small number of training

PT

samples, but its computational complexity is very high when the number of training

CC E

samples is large. ANNs and EAs, which are able to self-learn from the data, are possible ways to tackle the above problems.

A

1)

Artificial neural network based classification

The basic process of ANN-based classification for remote sensing images is shown

in Fig. 9, in which the training samples are input pixel by pixel to train the ANN, and the label or conditional probability of the corresponding pixel can then be acquired in the output layer. ANNs have been shown to perform better than statistical classifiers

such as the MLC [22-24, 93] and many other classifiers in remote sensing image classification [25]. For example, in [25], an ANN-based semi-supervised classifier was compared with two kernel-based methods (transductive SVM (TSVM) [94] and Laplacian SVM (LapSVM) [95]), and the results showed that the proposed ANN-based

IP T

semi-supervised classifier was computationally cheaper than the kernel-based semisupervised methods. With very large images, the difference in computational cost could

SC R

be of several orders of magnitude.

However, due to the difficult acquisition of training samples and the high

U

dimensionality of the remote sensing data, ANNs tend to suffer from overfitting. In

N

addition, this problem also results from the fixed ANN architecture. That is, most ANN-

A

based classifiers assume a fixed architecture which cannot be adapted automatically. In

M

order to resolve the above problem, by applying an EA to evolve the ANN (which is

ED

known as an “evolutionary artificial neural network” (EANN) [96]), the trained network structure can be automatically adjusted and its generalization capability can be

PT

enhanced. The main advantages of EANNs are their ability to escape overfitting, their

CC E

robustness, and their ability to adapt to complex remote sensing data. In [96], evolutionary programming (EP) was used to simultaneously evolve the ANN architecture and connection weights. Cruz-Ramírez et al. [26] developed a multi-

A

objective evolutionary neural network in which the Pareto differential evolution (PDE) algorithm is used to optimize the feedforward multilayer perceptron (MLP) neural network. Finally, the developed EANN was applied to crop identification in remotely sensed data.

2)

Evolutionary algorithm based classification

EA-based classifiers, in essence, are spectral matching methods. Thus, they are nonparametric methods, meaning that they do not make any assumption of the distribution of the remote sensing data. An example of AIS-based remote sensing image

IP T

classification is presented in Fig. 10, in which the training samples are used to train the AIS model (see Fig. 10(b)), and the antibody or memory cells (see Fig. 10(d)) in the

SC R

AIS are used to classify the original remote sensing image.

Compared with the application of EAs in remote sensing image clustering, the

U

application of EAs in remote sensing image supervised classification is less common.

N

In [97], Zhong et al. applied the CSA to remote sensing image classification, where the

A

diversity of the population was enhanced by combining the CSA with simulated

M

annealing (SA). In [21], Zhang et al. presented a resource-limited AIS supervised

ED

classifier for remote sensing images, in which the concept of “artificial recognition balls” (ARBs) was introduced into the whole process. With the help of the ARBs, the remote

PT

sensing image (antigens) can be classified more intelligently. In [12], an advanced version of this method was proposed, where the recognition radius can be determined

CC E

adaptively, without being defined by the user. Furthermore, in [98], operators such as preservation of the best antibody, the adaptive mutation rate, and the incorporation of

A

different distance metrics were used to optimize the traditional artificial immune network, which is then used to classify the remote sensing image. The performance of this method was found to be better than the BPNN, the decision tree, and the traditional artificial immune network algorithm. However, in the above “black-box” methods, it is difficult to find out which input

parameters are truly important, making it difficult for a new user to train an EA-based classifier for remote sensing image. The genetic fuzzy rule based classification system (GFRBCS) [99] can combine the high interpretability of the fuzzy rule based system (FRBCS) with the enhanced search capability of EAs, in order to automatically extract

IP T

an optimal fuzzy rule base. Thus, in [100, 101], GFRBCS was applied and used to generate or tune the classification rules for remote sensing images. In addition, GP-

SC R

based methods can perform better in this aspect due to their “open-box” characteristic. In [102], a GP was used to develop new vegetation indices, which were demonstrated

U

to be better than the traditional indices such as the normalized difference vegetation

Empirical comparison of the different computational intelligence based classification methods

M

A

3)

N

index (NDVI) and the soil-adjusted vegetation index (SAVI).

We undertook a real-data experiment on the AVIRIS Indian Pines image acquired in

ED

June 1992. The image, which is of 145 by 145 pixels, is shown in Fig. 11, representing

PT

an agricultural area in the northern part of Indiana. The image is composed of 220 spectral bands, which were all used for the classification. The 10 most representative

CC E

land-cover classes in the image are listed in Table I. In the classification process, 50% of the samples in each class were used as training samples, and the rest were used as

A

test samples.

Five different classifiers were used to classify the AVIRIS Indian Pines image: MLC, SVM, BPNN, resource-limited classification of remote sensing image (RLCRSI) [21], and the artificial antibody network (ABNet) [12]. A quantitative measurement of the

classification accuracy was obtained using the overall accuracy (OA) and the Kappa coefficient (Kappa). The classification results are listed in Fig. 12 and Table II. As can be seen from Fig. 12 and Table II, the traditional MLC classifier achieves the worst classification accuracy (57.51%), due to the fact that it makes an assumption of the

IP T

remote sensing data, and many Corn-min pixels are misclassified as Corn-no till pixels. In addition, because BPNN can easily suffer from overfitting, it does not perform as

SC R

well as the other computational intelligence based classifiers and SVM (81.85%). RLCRSI can acquire a better classification performance than the traditional classifiers,

U

but there is still some misclassification, and Corn-min pixels are misclassified as

N

Soybeans-min pixels in the bottom left in Fig. 12(d). Finally, the AIS-based RLCRSI

A

(82.04%) and ABNet (85.41%) classifiers achieve better performances because they

ED

M

are able to self-learn from the remote sensing data.

PT

3.2.2 Unsupervised classification (clustering) or segmentation

CC E

The task of unsupervised classification of remote sensing images, i.e. clustering, involves separating the images into a number of homogeneous regions, without using any prior information about the images [103]. Segmentation is characterized by

A

dividing an image into disjoint regions. It can be treated as a clustering problem, where the feature vector representing a pixel corresponds to a pattern, and each image region corresponds to a cluster. Since clustering and segmentation are very similar, we do not distinguish between them in the following text.

Traditionally, the task of remote sensing image clustering can be implemented by finding a set of cluster centers, as in k-means and fuzzy k-means, as shown in Fig. 13(b). Distance metrics are then used to assign all the pixels to the corresponding cluster centers. Hence, the task of clustering a remote sensing image can be transformed into

IP T

an optimization problem, the goal of which lies in the identification of correct cluster centers in the feature space. However, identifying correct cluster centers is a non-trivial

SC R

task because it is greatly affected by the initial values and easily gets stuck in local

optima. EAs have a powerful global search capability and are suited for the

U

optimization of cluster centers. In [104], an survey of EAs for clustering was presented

N

to demonstrate the effectiveness of EAs in remote sensing image clustering. In the EA-

A

based clustering methods, the basic individual encoding strategy is often formulated as

M

shown in Fig. 13(d), in which the cluster centers are encoded. The objective of the EA-

ED

based remote sensing image clustering methods is to find one (single-objective optimization) or a set of (multi-objective optimization) individuals (representing the

PT

cluster centers) with a high quality, using evolutionary operators. The quality of each

CC E

individual is then evaluated by a clustering objective function (e.g. a clustering validity index [105]).

In Fig. 14, the EA-based clustering methods for remote sensing images are depicted

A

in a chronological manner. According to the number of objectives to be optimized, the evolutionary clustering methods consist of evolutionary single-objective clustering and evolutionary multi-objective clustering. In addition, considering the imbalance between the global and local search capabilities of the traditional EAs, the memetic clustering

methods are also reviewed in the following. 1)

Evolutionary single-objective clustering

The evolutionary single-objective clustering algorithm was the first example of the application of an EA to the clustering problem in remote sensing image processing. In

IP T

[14, 106], Murthy and Maulik were the first to apply GAs to the clustering problem, and they tested this approach on both synthetic and real-life data sets. However, in these

SC R

methods, the optima of the objective function cannot indicate the optimal cluster number. In [49, 51], in order to implement the clustering task more automatically and

U

intelligently, automatic remote sensing image clustering was proposed, in which the

N

Davies-Bouldin (DB) index is used as the fitness function of automatic clustering,

A

optimized by a GA and DE. The minimum of the DB index can be used to

M

simultaneously indicate the optimal cluster number and clustering result. In [52], in

ED

order to make it possible to partition data that are linearly non-separable, a kernelized version of the method proposed in [51] was put forward for remote sensing images, in

PT

which a kernel-induced similarity measure is used instead of the conventional sum-of-

CC E

squares distance. In addition, considering the rich spatial information embedded in remote sensing images, which is a crucial difference with other types of data, some researchers have incorporated the spatial information into the objective functions of the

A

remote sensing image clustering methods [18, 55], in which the neighboring pixels located in the windows around the central pixel are incorporated into the objective function. The experimental results obtained in these studies showed that the spatial clustering methods can remove the effect of isolated pixels in the original remote

sensing image. 2)

Evolutionary multi-objective clustering

Different clustering objective functions are generally proposed from different perspectives, such as the Jm value [107] and the DB index, and represent a specific

IP T

structure on the remote sensing image. An effective clustering result can be acquired if there is a good match between the estimated remote sensing image structure and the

SC R

real remote sensing image structure, which is not known in practice. Hence, considering the complex structure of remote sensing data, there may be no single objective function

U

that can perform well on every remote sensing image. Thus, it is natural to

N

simultaneously optimize multiple objective functions, achieving a balance between

A

them. In general, this type of clustering method belongs to the problem of multi-

M

objective clustering [59]. Compared with the single-objective clustering methods,

ED

multi-objective clustering methods can acquire better results because the multiobjective optimization technique can take more information into consideration in the

PT

clustering process.

CC E

Recently, some evolutionary multi-objective clustering algorithms for remote sensing images have been presented [60-63]. In [62], two fuzzy clustering validity indices were utilized for optimization in the framework of the non-dominated sorting

A

genetic algorithm II (NSGA-II). However, in this method, the cluster number has to be specified manually. To tackle these problems, Zhong et al. [60] designed a two-layer clustering system comprising an optimization layer and a classification layer. In the classification layer, the Xie-Beni (XB) index is optimized to acquire the optimal cluster

number. In the classification layer, NSGA-II is utilized to minimize the Jm value and the XB index. However, this approach separates the process of automatic clustering into two layers: the layer for determination of the cluster number and the layer for clustering. In [61], Paoli et al. designed a new method for clustering hyperspectral images within

IP T

a multi-objective PSO framework, which can simultaneously implement band selection, determination of the cluster number, and clustering. A similar task was implemented in

SC R

a multi-objective PSO framework in [108]. Multi-objective clustering methods are

generally preferred over single-objective clustering methods. However, there is still one

U

problem that needs to be resolved for the multi-objective clustering methods, i.e., the

N

result of the multi-objective optimization is a set of candidate solutions rather than the

A

single solution obtained in single-objective optimization, so a strategy is needed to

M

select the most appropriate solution from the solution set. Although a number of

ED

decision-making techniques (such as the “technique for order of preference by similarity to ideal solution” (TOPSIS) [108], ratio cut [109], and the angle-based

PT

method proposed in [110]) have been proposed to locate these solutions, it is still an

CC E

open issue because these decision-making techniques do not consistently function well in multi-objective clustering methods for remote sensing images.

A

3)

Memetic clustering

EAs (e.g. GAs, DE) have a powerful global search capability and can quickly locate

the promising regions of the search space. However, the lack of a local search capability impedes the EAs from rapid convergence in local regions. Accordingly, memetic algorithms (MAs) [111], which can be considered as a hybrid between the population-

based EAs and one or more local search methods, were proposed to solve the problem by taking advantage of both global and local search. The effectiveness of the advanced MAs has been demonstrated in many areas, such as feature selection [112] and vehicle routing [113].

IP T

The performance of remote sensing image clustering is affected by the optimizer that functions to select the sets of cluster centers. Therefore, the MA-based clustering

SC R

methods can achieve better performances than the clustering methods based on

individual EAs, due to the fact that MAs can capture a more balanced optimization

U

between global search and local search. For example, in [18, 53], a local searcher (Fig.

N

15(a)) was designed to combine with DE in single-objective clustering and multi-

A

objective clustering, respectively, in which Gaussian mutation is performed on each

M

dimension of the individual, and a new individual is then generated. The better

ED

individual between the old one and the new one is then selected to survive in the new population. By introducing local search to the whole clustering process, an improved

PT

result can be acquired. In the method proposed in [54], instead of pixels, objects (see

CC E

Fig. 15(b)) are set as the processing units, and are acquired by a watershed segmentation method in a pre-processing step. Local search is then performed by reassigning the labels of regions which are dissimilar to their corresponding centers. This method can

A

acquire a promising visual performance that preserves the detail information. Although the introduction of local search can improve the performance of EA-based remote sensing image clustering, it is difficult to automatically achieve a balance between global search and local search. Furthermore, an inappropriate choice of local searcher

may destroy the diversity of the population, making the evolutionary progress premature. Hence, in the design of the MA-based remote sensing clustering methods, the local searcher must be selected carefully and adaptively, as in the adaptive memetic algorithm [114]. Empirical comparison of different computational intelligence based clustering methods

IP T

4)

SC R

In order to empirically compare the above-mentioned clustering methods, we used a 30-m resolution multi-spectral Landsat TM image of Wuhan City, China, with a size of 400 × 400 pixels, and six bands. The selected region of the image was expected to

U

contain five classes, which are listed in Table III. The original Wuhan TM image and

A

N

the ground-truth image are shown in Fig. 16(a) and (b).

M

The compared clustering methods were fuzzy c-means (FCM), the automatic fuzzy clustering using an improved differential evolution algorithm (FCIDE) [115], the

ED

adaptive memetic fuzzy clustering algorithm with spatial information (AMASFC) [18],

PT

the automatic fuzzy clustering method based on adaptive multi-objective differential evolution (AFCMDE) [60], and the adaptive multi-objective memetic fuzzy clustering

CC E

algorithm (AFCMOMA) [53]. It should be noted that FCIDE and AMASFC are evolutionary single-objective clustering methods, and AFCMDE and AFCMOMA are

A

evolutionary multi-objective clustering methods. In addition, AMASFC and AFCMOMA are memetic algorithm based clustering methods. The clustering results of the above methods were also evaluated quantitatively using the OA and Kappa. A comparison between the different clustering methods with the Wuhan TM image is shown in Fig. 17 and Table IV, where it can be seen that the computational intelligence

based clustering methods can acquire a better clustering accuracy than FCM (81.66%). For FCM, many Building pixels are misclassified as Vegetation pixels. In addition, the multi-objective clustering methods, AFCMDE (86.38%) and AFCMOMA (91.29%), perform better than the single-objective clustering method, FCIDE (81.75%), due to the

IP T

fact that they can take more information (characterized by the multiple objectives) into consideration. In Fig. 17(e), AFCMOMA obtains better visual results, particularly for

SC R

the Vegetation class. Finally, the memetic algorithm based clustering methods,

AMASFC (86.55%) and AFCMOMA (91.29%), obtain higher OA and Kappa values

U

than the corresponding single-objective clustering method (i.e. FCIDE (81.75%)) and

Change detection of remote sensing images

M

3.3

A

N

multi-objective clustering method (i.e. AFCMDE (86.38%)).

ED

Conventionally, change detection techniques, such as the maximum likelihood model

PT

used in the supervised change detection technique and the expectation maximization (EM) algorithm used in unsupervised change detection, tend to assume that the changed

CC E

and unchanged types follow a single Gaussian distribution and can be formulated as a multivariate Gaussian model. However, such prior assumptions may be unrealistic in

A

remote sensing images, as there will be many land-cover types in both the changed and unchanged classes. Recently, computational intelligence based techniques have been shown to be promising candidates for resolving the above problems in both supervised and unsupervised change detection for remote sensing images [13, 17]. The unsupervised change detection approaches need to analyze the dependency of

the multi-temporal images to identify the changed areas. In [27, 28], pulse coupled neural networks (PCNNs), which were developed under the mechanism of the visual cortex of small mammals, were used for change detection of high spatial resolution remote sensing images. In PCNNs, the threshold tuning required in many unsupervised

IP T

techniques is no longer necessary, which significantly reduces the computational burden. EA-based unsupervised change detection methods for remote sensing images

SC R

have also been proposed, in which the individual representation (i.e. the change map)

can be described as shown in Fig. 18(a). Among these methods, Celik [17] used a GA

U

to avoid the parameter estimation steps, and obtained the change map without making

N

any prior assumptions. However, the GA-based method used in [17] was designed to

A

handle a single-objective function, and it cannot take multiple criteria into consideration,

M

thus limiting the change detection performance. To deal with the problem, multi-

ED

objective optimization based change detection methods for remote sensing images have been proposed [64, 65]. In [64], a two-stage technique combining an AIS with a multi-

PT

objective optimization algorithm was proposed. In the first stage, the difference image

CC E

is classified into three groups (i.e., changed, unchanged, and uncertain) with the help of the AIS. Then, in the second stage, the uncertain samples are determined as either the changed or unchanged class using the multi-objective clustering algorithm. Moreover,

A

many ANN-based [29] and EA-based [30] change detection methods have taken the contextual information into consideration to alleviate the effect of registration and noise, especially with the emergence of very high resolution (VHR) satellites such as QuickBird and WorldView. In [30], Wan and Jiao explored a CSA-based change

detection method that acquires the change map at a regional level based on the contextual information. In [29], change detection techniques based on a Hopfield neural network (HNN) were used to make a tradeoff between each pixel and its neighbors, to overcome the drawback of the methods which only consider the inter-pixel

IP T

relationships. In [65], the multi-objective spatial clustering method was used for change detection by considering the neighboring pixels, in which two complementary objective

SC R

functions are constructed and optimized simultaneously, to preserve details and remove noise.

U

When extra information such as “from-to” information is needed, supervised change

N

detection algorithms are used, in which training samples are required to train a

A

supervised change detection model. Thus, the advantage of ANNs, i.e., they can

M

construct the function between the input and the output without specifying a parametric

ED

model, means that they can be used in supervised change detection for remote sensing images. As can be seen in Fig. 19, for the ANN-based change detection methods, the

PT

difference images calculated by the multi-temporal images and the corresponding

CC E

training samples can be input into the networks to generate accurate change maps. In [13], Chini demonstrated the superiority of the ANN-based change detection method over its MLC-based counterpart. In [31], for the task of detecting “unusual changes”, a

A

predictive model based on an ANN was provided to “predict” the pixel values. In order to detect changes in conifer mortality and urban areas, a multilayer feedforward architecture [32] and the MLP [33] have also been utilized, respectively.

4. Discussion and conclusion With the emergence of optical remote sensing images with a higher spectral-spatialtemporal resolution, computational intelligence techniques have been widely used in

IP T

remote sensing image processing. In supervised applications (including wrapper-based hyperspectral band selection, supervised classification, and supervised change

SC R

detection), the performance of the applied methods is greatly affected by the

assumptions made of the remote sensing data, such as the distribution in the feature space. EA-based and ANN-based methods are able to self-learn from the data, and thus

N

U

do not need to make assumptions about the remote sensing data. As a result, the EA-

A

based and ANN-based methods can obtain better performances in the corresponding

M

tasks. In unsupervised applications (including filter-based hyperspectral band selection, clustering, and unsupervised change detection), the performance is often affected by

ED

the optimization capability, due to the fact that the corresponding unsupervised tasks

PT

are usually transformed into optimization problems. Because the bio-inspired EAs possess a powerful global search capability, they can perform well in these

CC E

unsupervised applications.

Many computational intelligence based approaches have been introduced to remote

A

sensing image processing. However, it is worth noting that the massive amounts of remote sensing images with a high spatial-spectral-temporal resolution, the diverse feature descriptors that have been generated, and the rate at which these images are acquired and updated has resulted in an imperative need for rapid advancement in computational intelligence based technologies. In what follows, we reveal some of the

potential challenges for computational intelligence in the field of remote sensing image processing. 1) Deep learning of remote sensing images with a high spatial resolution While computational intelligence based techniques have performed relatively well in

IP T

the processing of remote sensing images of a low or medium spatial resolution, the rapid development of remote sensing sensors with a high spatial resolution is bringing

SC R

new and great challenges to the field. This is due to the fact that remote sensing images with a higher spatial resolution contain increased information about the objects on the

Earth. This in turn results in increased spectral variability, i.e., objects of the same type

N

U

may have different spectral characteristics. Recently, scene-based techniques [72]

A

which incorporate semantic information have attracted increasing attention since they

M

can embrace more complex information than the pixel- or object-based [116] representations. However, it is not an easy task to extract the semantic features of a

ED

scene. In recent research, scene classification based on the bag-of-words (BOW [117])

PT

model and topic models such as probabilistic latent semantic analysis (PLSA [118]) and latent Dirichlet allocation (LDA [72]) have been proposed and used in the computer

CC E

vision and remote sensing community. However, all these methods rely on the clustering results to map the individual scene image to the visual words, which limits

A

their feature representation ability. In addition, some studies have focused on feature descriptors (e.g. scale-invariant feature transform (SIFT)) for the scene classification of very high spatial resolution images. However, the design of these feature descriptors is subjective and relies on expert knowledge [119]. To extract the relevant features of the scenes automatically, deep learning can serve as an appropriate unsupervised feature

processing approach [120], and it can be used to learn the relevant features automatically from the remote sensing image, without elaborately designed descriptors. These features cover different levels, from the pixel level to the semantic level, from physical attributes to social attributes of the objects. As indicated in Section 3.1, deep

IP T

learning has achieved preliminary success in the context of remote sensing image scene classification [75-78] and other fields of remote sensing image processing, such as

SC R

classification, change detection [83], and object detection [86], due to its superiority in

feature representation for remote sensing images with a high spatial resolution [79].

U

This suggests that deep learning has great potential in remote sensing image processing.

N

2) Multi-objective optimization for the ill-posed problems in remote sensing image processing

A

An ill-posed problem can be defined as follows: a problem is well-posed if its

M

solution exists and is unique and stable; a problem is ill-posed if at least one of the three

ED

conditions does not hold [121]. It has been demonstrated that most inverse problems are ill-posed problems. In remote sensing image processing, many of the problems are

PT

actually inverse problems, including hyperspectral unmixing [122], hyperspectral sub-

CC E

pixel mapping [123], super-resolution [124], hyperspectral denoising [125], and so on. One possible solution for such problems is to introduce a priori information to penalize the model, which usually involves combining the fidelity term || Ax  y ||2 and the

A

regularization term  ( x) into a single objective function with a regularization parameter  , as shown in Eq. (1). The regularization-based ill-posed problems, such as hyperspectral sub-pixel mapping [126] and super-resolution [124], have been widely studied in remote sensing image processing.

min f || Ax  y ||2  ( x)

(1)

A special case of a regularization-based ill-posed problem is hyperspectral unmixing, which is a challenging, ill-posed problem because of the model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size

IP T

[122]. Traditionally, the unmixing of hyperspectral remote sensing images can be implemented using a linear mixture model, as in the sparse representation based

SC R

unmixing model [127], in which two conflicting cost function terms (measurement error and a regularization term) are combined via a regularization parameter. The

U

regularization parameter is crucial to the performance of unmixing, but is usually

N

determined by trial and error. In addition, the l0 norm constraint is usually imposed on

A

the solution in order to guarantee the sparsity of the endmembers. However, the

M

optimization of the l0 norm is a NP-hard problem, and hence the l1 norm is often

ED

considered instead, but this can lead to an unsatisfactory unmixing performance. The multi-objective optimization paradigm of computational intelligence has great potential

PT

[110] in resolving the above problem by simultaneously optimizing the fidelity term

CC E

and the regularization term, and providing multiple alternative hyperspectral unmixing solutions, without the need for a regularization parameter. Furthermore, the evolutionary multi-objective optimization has no constraint on the convexity of the

A

objective function, allowing the direct optimization of the l0 norm. In addition, other constraints such as spatial constraints can also be formulated as another term to be optimized simultaneously under the multi-objective optimization framework [20]. Therefore, the multi-objective optimization methods provide an alternative way to

resolve the regularization-based ill-posed problems in remote sensing image processing. 3) The efficiency of computational intelligence based optimization EAs have been established as universal problem solvers and have enjoyed significant success in obtaining high-quality solutions in many remote sensing image processing

IP T

tasks. However, as EAs involve an iterative process, they are generally slow and struggle to cope with today’s “big” [128, 129] image data with a high spatial, spectral,

SC R

and temporal resolution. Furthermore, the current state-of-the-art EAs in the remote sensing image processing literature have been mainly focused on achieving a high

accuracy in classification, change detection, and so on. Few works have focused on

N

U

improving the efficiency of EAs for large-scale remote sensing image processing.

A

In the context of computational intelligence, the meme-inspired computational model,

M

or memetic computation [111], has become an increasing focus of research in recent years, with the aim being to speed up the canonical forms of EAs. In the last decades,

ED

while a meme has been perceived as a form of individual learning procedure or local

PT

search operator to enhance the capability of population-based search algorithms, more recent works aimed at speeding up evolutionary search have perceived memes as units

CC E

of information or building blocks of structured knowledge that are derived from the data via machine learning methods. These atomized units of memes, metamemes, or

A

memeplexes [111] can then be expressed in hierarchical nested relationships or conceptual entities for higher-order learning, and subsequently transferred or reused to speed up the future solving of related problems [130]. Other works hybridizing EAs with machine learning to enhance the efficiency of remote sensing image processing have also been recently reported [131]. In addition, the efficiency of EAs can be

improved by taking advantage of their property of implicit parallelism. EAs, as population-based meta-heuristics, can be easily parallelized and effectively implemented on relatively cheap graphics processing units (GPUs) [132]. 4) Evolutionary multitasking for remote sensing image processing workflow

IP T

The workflow of remote sensing image processing consists of pre-processing, processing, and post-processing. Generally, many tasks in this workflow can be

SC R

transformed into optimization problems such as denoise-classification [133], denoisehyperspectral unmixing [134], hyperspectral unmixing-subpixel mapping [135] etc. Thereafter, solutions can be acquired by optimizing the corresponding objective

N

U

functions. In general, the final performance of remote sensing image processing usually

A

depends on all tasks and their interaction instead of only one task. However, for the

M

conventional remote sensing image processing methods, these tasks are usually regarded as independent procedures, despite of their interaction with each other.

ED

Motivated by nature-inspired computing, facilitating implicit information exchange

PT

across different numerical optimization tasks, evolutionary multitasking has attracted more attention in the field of computational intelligence, in which the solution of

CC E

multiple self-contained (but possibly similar) optimization tasks can be acquired at the same time using a single population of evolving individuals [136, 137]. Therefore,

A

evolutionary multitasking can take the interaction between different tasks of remote sensing image processing into consideration instead of conducting them separately, which may also be one interesting research direction.

Acknowledgments This work was supported by National Natural Science Foundation of China under Grant No. 41771385, 41622107 and 41371344, National Key Research and

IP T

Development Program of China under Grant No. 2017YFB0504202, Natural Science Foundation of Hubei Province in China under Grant No. 2016CFA029, and Key

A

CC E

PT

ED

M

A

N

U

Fuzhou University under Grant No. 2018LSDMIS04.

SC R

Laboratory of Spatial Data Mining & Information Sharing of Ministry of Education,

References [1]

J. A. Richards and J. Richards, Remote sensing digital image analysis: Springer, 1999.

[2]

J. A. Benediktsson, J. Chanussot, and W. M. Moon, "Very high-resolution remote sensing: Challenges and opportunities," Proc. IEEE vol. 100, no. 6, pp. 1907-1910, 2012.

[3]

M. Hansen, R. DeFries, J. R. Townshend, and R. Sohlberg, "Global land cover classification at 1 km spatial resolution using a classification tree approach," Int. J. Remote Sens. , vol. 21, no. 6-7, pp. 1331-1364, 2000. E. L. Hestir, S. Khanna, M. E. Andrew, M. J. Santos, J. H. Viers, J. A. Greenberg, S. S. Rajapakse,

IP T

[4]

and S. L. Ustin, "Identification of invasive vegetation using hyperspectral remote sensing in the California Delta ecosystem," Remote Sens. Environ. , vol. 112, no. 11, pp. 4034-4047, 2008.

Y. Chebud, G. Naja, R. Rivero, and A. Melesse, "Water Quality Monitoring Using Remote

SC R

[5]

Sensing and an Artificial Neural Network," Water Air Soil Pollut. , vol. 223, no. 8, pp. 4875-4887, 2012. [6]

Q. Weng, "Thermal infrared remote sensing for urban climate and environmental studies: Methods, applications, and trends," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 64, no. 4,

[7]

U

pp. 335-344, 2009.

L. Giglio, J. Descloitres, C. O. Justice, and Y. J. Kaufman, "An Enhanced Contextual Fire

[8]

N

Detection Algorithm for MODIS," Remote Sens. Environ. , vol. 87, no. 2–3, pp. 273-282, 2003. F. F. Sabins, "Remote sensing for mineral exploration," Ore Geol. Rev. , vol. 14, no. 3–4, pp. 157M. Fingas and C. Brown, "Review of oil spill remote sensing," Marine pollution bulletin, vol. 83, no. 1, pp. 9-23, 2014. [10]

M

[9]

A

183, 1999.

D. J. Mulla, "Twenty five years of remote sensing in precision agriculture: Key advances and

ED

remaining knowledge gaps," Biosystems Eng. , vol. 114, no. 4, pp. 358-371, 2013. [11]

Web. (2016). Available: https://directory.eoportal.org/web/eoportal/satellite-missions/

[12]

Y. Zhong and L. Zhang, "An Adaptive Artificial Immune Network for Supervised Classification

PT

of Multi-/Hyperspectral Remote Sensing Imagery," IEEE Trans. Geosci. Remote Sens. , vol. 50, no. 3, pp. 894-909, 2012. [13]

M. Chini, F. Pacifici, W. J. Emery, N. Pierdicca, and F. Del Frate, "Comparing Statistical and

CC E

Neural Network Methods Applied to Very High Resolution Satellite Images Showing Changes in Man-Made Structures at Rocky Flats," IEEE Trans. Geosci. Remote Sens. , vol. 46, no. 6, pp. 1812-1821, 2008.

[14]

C. A. Murthy and N. Chowdhury, "In search of optimal clusters using genetic algorithms," Pattern Recognit. Lett. , vol. 17, no. 8, pp. 825-832, 1996.

A

[15]

[16]

S. Yu, S. De Backer, and P. Scheunders, "Genetic feature selection combined with composite fuzzy nearest neighbor classifiers for hyperspectral satellite imagery," Pattern Recognit. Lett. , vol. 23, no. 1, pp. 183-190, 2002. Y. Zhong, L. Zhao, and L. Zhang, "An Adaptive Differential Evolution Endmember Extraction Algorithm for Hyperspectral Remote Sensing Imagery," IEEE Geosci. Remote Sens. Lett. , vol. 11, no. 6, pp. 1061-1065, 2014.

[17]

T. Celik, "Change detection in satellite images using a genetic algorithm approach," IEEE Geosci. Remote Sens. Lett. , vol. 7, no. 2, pp. 386-390, 2010.

[18]

Y. Zhong, A. Ma, and L. Zhang, "An Adaptive Memetic Fuzzy Clustering Algorithm With Spatial Information for Remote Sensing Imagery," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. (JSTARS), vol. 7, no. 4, pp. 1235-1248, 2014.

[19]

F. Zhu and P. Honeine, "Biobjective Nonnegative Matrix Factorization: Linear Versus KernelBased Models," IEEE Trans. Geosci. Remote Sens. , vol. 54, no. 7, pp. 4012-4022, 2016.

[20]

M. Gong, H. Li, E. Luo, and J. Liu, "A Multi-objective Cooperative Coevolutionary Algorithm for Hyperspectral Sparse Unmixing," IEEE Trans. Evol. Comput. , vol. PP, no. 99, pp. 1-1, 2016.

[21]

L. Zhang, Y. Zhong, B. Huang, and P. Li, "A resource limited artificial immune system algorithm for supervised classification of multi/hyper‐ spectral remote sensing imagery," Int. J. Remote

[22]

IP T

Sens. , vol. 28, no. 7, pp. 1665-1686, 2007.

B. Dixon and N. Candade, "Multispectral landuse classification using neural networks and support

vector machines: one or the other, or both?," Int. J. Remote Sens. , vol. 29, no. 4, pp. 1185-1206, [23]

SC R

2008.

C. Joshi, J. D. Leeuw, A. K. Skidmore, I. C. v. Duren, and H. Van Oosten, "Remotely sensed estimation of forest canopy density: A comparison of the performance of four methods," Int. J. Appl. Earth Obs. Geoinf. , vol. 8, no. 2, pp. 84-95, 2006.

[24]

H. Bagan, Q. Wang, M. Watanabe, Y. Yang, and J. Ma, "Land cover classification from MODIS

U

EVI times-series data using SOM neural network," Int. J. Remote Sens. , vol. 26, no. 22, pp. 49995012, 2005.

F. Ratle, G. Camps-Valls, and J. Weston, "Semisupervised neural networks for efficient

N

[25]

hyperspectral image classification," IEEE Trans. Geosci. Remote Sens. , vol. 48, no. 5, pp. 2271M. Cruz-Ramírez, C. Hervás-Martínez, M. Jurado-Expósito, and F. López-Granados, "A multi-

M

[26]

A

2282, 2010.

objective neural network based method for cover crop identification from remote sensed data," Expert Systems with Applications, vol. 39, no. 11, pp. 10038-10048, 2012. F. Pacifici and F. Del Frate, "Automatic change detection in very high resolution images with

ED

[27]

pulse-coupled neural networks," IEEE Geosci. Remote Sens. Lett. , vol. 7, no. 1, pp. 58-62, 2010. [28]

Y. Zhong, W. Liu, J. Zhao, and L. Zhang, "Change Detection Based on Pulse-Coupled Neural

PT

Networks and the NMI Feature for High Spatial Resolution Remote Sensing Imagery," IEEE Geosci. Remote Sens. Lett. , vol. 12, no. 3, pp. 537-541, 2015. [29]

S. Ghosh, L. Bruzzone, S. Patra, F. Bovolo, and A. Ghosh, "A context-sensitive technique for

CC E

unsupervised change detection based on Hopfield-type neural networks," IEEE Trans. Geosci. Remote Sens. , vol. 45, no. 3, pp. 778-789, 2007.

[30]

H. L. Wan and L. C. Jiao, "Change detection in SAR images by means of grouping connected regions using clone selection algorithm," Electron. Lett vol. 47, no. 5, pp. 338-339, 2011.

A

[31] [32] [33]

C. Clifton, "Change detection in overhead imagery using neural networks," Applied Intelligence, vol. 18, no. 2, pp. 215-234, 2003. S. Gopal and C. Woodcock, "Remote sensing of forest change using artificial neural networks," IEEE Trans. Geosci. Remote Sens. , vol. 34, no. 2, pp. 398-404, 1996. C. Pratola, F. D. Frate, G. Schiavon, and D. Solimini, "Toward Fully Automatic Detection of Changes in Suburban Areas From VHR SAR Images by Combining Multiple Neural-Network Models," IEEE Trans. Geosci. Remote Sens. , vol. 51, no. 4, pp. 2055-2066, 2013.

[34]

A. E. Eiben and J. Smith, "From evolutionary computation to the evolution of things," Nature, vol. 521, no. 7553, pp. 476-482, 2015.

[35]

L. Zhang, Y. Zhong, B. Huang, J. Gong, and P. Li, "Dimensionality reduction based on clonal selection for hyperspectral imagery," IEEE Trans. Geosci. Remote Sens. , vol. 45, no. 12, pp. 4172-4186, 2007.

[36]

F. Jie, L. C. Jiao, Z. Xiangrong, and S. Tao, "Hyperspectral Band Selection Based on Trivariate Mutual Information and Clonal Selection," IEEE Trans. Geosci. Remote Sens. , vol. 52, no. 7, pp. 4092-4105, 2014.

[37]

M. Pedergnana, P. R. Marpu, M. D. Mura, J. A. Benediktsson, and L. Bruzzone, "A novel technique for optimal feature selection in attribute profiles based on genetic algorithms," IEEE Trans. Geosci. Remote Sens. , vol. 51, no. 6, pp. 3514-3528, 2013. Y. Bazi and F. Melgani, "Toward an Optimal SVM Classification System for Hyperspectral

IP T

[38]

Remote Sensing Images," IEEE Trans. Geosci. Remote Sens. , vol. 44, no. 11, pp. 3374-3385, 2006.

P. Ghamisi and J. A. Benediktsson, "Feature Selection Based on Hybridization of Genetic

SC R

[39]

Algorithm and Particle Swarm Optimization," IEEE Geosci. Remote Sens. Lett. , vol. 12, no. 2, pp. 30-313, 2015. [40]

M. Mahrooghy, N. H. Younan, V. G. Anantharaj, J. Aanstoos, and S. Yarahmadian, "On the Use of the Genetic Algorithm Filter-Based Feature Selection Technique for Satellite Precipitation

[41]

U

Estimation," IEEE Geosci. Remote Sens. Lett. , vol. 9, no. 5, pp. 963-967, 2012.

J. Im, Z. Lu, J. Rhee, and J. R. Jensen, "Fusion of feature selection and optimized immune

N

networks for hyperspectral image classification of urban landscapes," Geocarto International, vol. 27, no. 5, pp. 373-393, 2011.

H. Yang, Q. Du, and G. Chen, "Particle Swarm Optimization-Based Hyperspectral Dimensionality

A

[42]

M

Reduction for Urban Land Cover Classification," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. (JSTARS), vol. 5, no. 2, pp. 544-554, 2012. [43]

M. Cui, S. Prasad, M. Mahrooghy, L. M. Bruce, and J. Aanstoos, "Genetic algorithms and linear

ED

discriminant analysis based dimensionality reduction for remotely sensed image analysis," in Geoscience and Remote Sensing Symposium (IGARSS), 2011 IEEE International, pp. 2373-2376, 2011.

J. Feng, L. Jiao, F. Liu, T. Sun, and X. Zhang, "Mutual-Information-Based Semi-Supervised

PT

[44]

Hyperspectral Band Selection With High Discrimination, High Information, and Low Redundancy," IEEE Trans. Geosci. Remote Sens. , vol. 53, no. 5, pp. 2956-2969, 2015. M. Gong, M. Zhang, and Y. Yuan, "Unsupervised Band Selection Based on Evolutionary

CC E

[45]

Multiobjective Optimization for Hyperspectral Images," IEEE Trans. Geosci. Remote Sens. , vol. 54, no. 1, pp. 544-557, 2016.

[46]

A. Ghosh, A. Datta, and S. Ghosh, "Self-adaptive differential evolution for feature selection in hyperspectral image data," Appl. Soft Comput., vol. 13, no. 4, pp. 1969-1977, 2013.

A

[47]

[48]

Y. Rezaei, M. R. Mobasheri, M. J. V. Zoej, and M. E. Schaepman, "Endmember extraction using a combination of orthogonal projection and genetic algorithm," IEEE Geosci. Remote Sens. Lett. , vol. 9, no. 2, pp. 161-165, 2012. Y. Zhong and L. Zhang, "Sub-pixel Mapping Based on Artificial Immune Systems for Remote Sensing Imagery," Pattern Recognit. , vol. 46, no. 11, pp. 2902-2926, 2013.

[49]

S. Bandyopadhyay and U. Maulik, "Genetic clustering for automatic evolution of clusters and application to image classification," Pattern Recognit. , vol. 35, no. 6, pp. 1197-1208, 2002.

[50]

Y. Zhong, L. Zhang, B. Huang, and P. Li, "An unsupervised artificial immune classifier for

multi/hyperspectral remote sensing imagery," IEEE Trans. Geosci. Remote Sens. , vol. 44, no. 2, pp. 420-431, 2006. [51]

S. Das, A. Abraham, and A. Konar, "Automatic Clustering Using an Improved Differential Evolution Algorithm," IEEE Trans. Syst. Man Cybern. Part A Syst. Humans vol. 38, no. 1, pp. 218-237, 2008.

[52]

S. Das and S. Sil, "Kernel-induced fuzzy clustering of image pixels with an improved differential evolution algorithm," Inf. Sci., vol. 180, no. 8, pp. 1237-1256, 2010.

[53]

A. Ma, Y. Zhong, and L. Zhang, "Adaptive Multiobjective Memetic Fuzzy Clustering Algorithm for Remote Sensing Imagery," IEEE Trans. Geosci. Remote Sens. , vol. 53, no. 8, pp. 4202-4217,

[54]

IP T

2015.

L. Jiao, M. Gong, S. Wang, B. Hou, Z. Zheng, and Q. Wu, "Natural and Remote Sensing Image

Segmentation Using Memetic Computing," IEEE Comput. Intell. Mag. , vol. 5, no. 2, pp. 78-91, [55]

SC R

2010.

S. Bandyopadhyay, "Satellite image classification using genetically guided fuzzy clustering with spatial information," Int. J. Remote Sens. , vol. 26, no. 3, pp. 579-593, 2005.

[56]

C. A. Coello, "Evolutionary multi-objective optimization: a historical view of the field," IEEE Comput. Intell. Mag. , vol. 1, no. 1, pp. 28-36, 2006.

Q. Zhang, A. Zhou, and Y. Jin, "RM-MEDA: A Regularity Model-Based Multiobjective

U

[57]

Estimation of Distribution Algorithm," IEEE Trans. Evol. Comput. , vol. 12, no. 1, pp. 41-63, [58]

N

2008.

A. Mukhopadhyay, U. Maulik, S. Bandyopadhyay, and C. A. Coello, "A Survey of Multiobjective

A

Evolutionary Algorithms for Data Mining: Part I," IEEE Trans. Evol. Comput. , vol. 18, no. 1, pp. [59]

M

4-19, 2014.

A. Mukhopadhyay, U. Maulik, S. Bandyopadhyay, and C. A. C. Coello, "Survey of Multiobjective Evolutionary Algorithms for Data Mining: Part II," IEEE Trans. Evol. Comput. , vol. 18, no. 1,

[60]

ED

pp. 20-35, 2014.

Y. Zhong, S. Zhang, and L. Zhang, "Automatic Fuzzy Clustering Based on Adaptive MultiObjective Differential Evolution for Remote Sensing Imagery," IEEE J. Sel. Topics Appl. Earth

PT

Observ. Remote Sens. (JSTARS), vol. 6, no. 5, pp. 2290-2301, 2013. [61]

A. Paoli, F. Melgani, and E. Pasolli, "Clustering of hyperspectral images based on multiobjective particle swarm optimization," IEEE Trans. Geosci. Remote Sens. , vol. 47, no. 12, pp. 4175-4188,

CC E

2009. [62]

S. Bandyopadhyay, U. Maulik, and A. Mukhopadhyay, "Multiobjective genetic clustering for pixel classification in remote sensing imagery," IEEE Trans. Geosci. Remote Sens. , vol. 45, no. 5, pp. 1506-1511, 2007.

A

[63]

[64]

A. Mukhopadhyay and U. Maulik, "Unsupervised pixel classification in satellite imagery using multiobjective fuzzy clustering combined with SVM classifier," IEEE Trans. Geosci. Remote Sens. , vol. 47, no. 4, pp. 1132-1138, 2009. R. Shang, L. Qi, L. Jiao, R. Stolkin, and Y. Li, "Change detection in SAR images by artificial immune multi-objective clustering," Eng. Appl. Artif. Intell. , vol. 31, no. 0, pp. 53-67, 2014.

[65]

H. Li, M. Gong, Q. Wang, J. Liu, and L. Su, "A multiobjective fuzzy clustering method for change detection in SAR images," Appl. Soft Comput., vol. 46, pp. 767-777, 2016.

[66]

A. Singh, "Review Article Digital change detection techniques using remotely-sensed data," Int. J. Remote Sens. , vol. 10, no. 6, pp. 989-1003, 1989.

[67]

L. Sicong, L. Bruzzone, F. Bovolo, and D. Peijun, "Hierarchical Unsupervised Change Detection in Multitemporal Hyperspectral Images," IEEE Trans. Geosci. Remote Sens. , vol. 53, no. 1, pp. 244-260, 2015.

[68]

M. Fauvel, Y. Tarabalka, J. A. Benediktsson, J. Chanussot, and J. C. Tilton, "Advances in SpectralSpatial Classification of Hyperspectral Images," Proc. IEEE vol. 101, no. 3, pp. 652-675, 2013.

[69]

H. Jiao, Y. Zhong, and L. Zhang, "Artificial DNA computing-based spectral encoding and matching algorithm for hyperspectral remote sensing data," IEEE Trans. Geosci. Remote Sens. , vol. 50, no. 10, pp. 4085-4104, 2012.

[70]

H. Jiao, Y. Zhong, and L. Zhang, "An Unsupervised Spectral Matching Classifier Based on

IP T

Artificial DNA Computing for Hyperspectral Remote Sensing Imagery," IEEE Trans. Geosci. Remote Sens. , vol. 52, no. 8, pp. 4524-4538, 2014. [71]

A. Ma, Y. Zhong, B. Zhao, H. Jiao, and L. Zhang, "Semisupervised Subspace-Based DNA Geosci. Remote Sens. , vol. 54, no. 8, pp. 4402-4418, 2016.

[72]

SC R

Encoding and Matching Classifier for Hyperspectral Remote Sensing Imagery," IEEE Trans. M. Lienou, H. Maitre, and M. Datcu, "Semantic Annotation of Satellite Images Using Latent Dirichlet Allocation," IEEE Geosci. Remote Sens. Lett. , vol. 7, no. 1, pp. 28-32, 2010. [73]

Y. Bengio, A. Courville, and P. Vincent, "Representation Learning: A Review and New

[74]

U

Perspectives," IEEE Trans. Pattern Anal. Mach. Intell. , vol. 35, no. 8, pp. 1798-1828, 2013. V. Mnih and G. Hinton, "Learning to Detect Roads in High-Resolution Aerial Images," in Springer Berlin Heidelberg, pp. 210-223, 2010.

F. Zhang, B. Du, and L. Zhang, "Saliency-Guided Unsupervised Feature Learning for Scene

A

[75]

N

Computer Vision – ECCV 2010. vol. 6316, K. Daniilidis, P. Maragos, and N. Paragios, Eds., ed:

[76]

M

Classification," IEEE Trans. Geosci. Remote Sens. , vol. 53, no. 4, pp. 2175-2184, 2015. Q. Zou, L. Ni, T. Zhang, and Q. Wang, "Deep Learning Based Feature Selection for Remote Sensing Scene Classification," IEEE Geosci. Remote Sens. Lett. , vol. 12, no. 11, pp. 2321-2325, [77]

ED

2015.

A. Romero, C. Gatta, and G. Camps-Valls, "Unsupervised Deep Feature Extraction for Remote Sensing Image Classification," IEEE Trans. Geosci. Remote Sens. , vol. 54, no. 3, pp. 1349-1362,

PT

2016. [78]

F. Zhang, B. Du, and L. Zhang, "Scene Classification via a Gradient Boosting Random Convolutional Network Framework," IEEE Trans. Geosci. Remote Sens. , vol. 54, no. 3, pp. 1793-

CC E

1802, 2016. [79]

L. Zhang and B. Du, "Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art," IEEE Geoscience and Remote Sensing Magazine, vol. 4, no. 2, pp. 22-40, 2016.

[80]

Y. Chen, Z. Lin, X. Zhao, G. Wang, and Y. Gu, "Deep Learning-Based Classification of

A

Hyperspectral Data," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. (JSTARS), vol. 7, no.

[81]

6, pp. 2094-2107, 2014. Y. Chen, X. Zhao, and X. Jia, "Spectral-Spatial Classification of Hyperspectral Data Based on Deep Belief Network," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. (JSTARS), vol. 8, no. 6, pp. 2381-2392, 2015.

[82]

W. Zhao and S. Du, "Learning multiscale and deep representations for classifying remotely sensed imagery," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 113, pp. 155-165, 2016.

[83]

M. Gong, J. Zhao, J. Liu, Q. Miao, and L. Jiao, "Change Detection in Synthetic Aperture Radar Images Based on Deep Neural Networks," IEEE Transactions on Neural Networks and Learning

Systems, vol. 27, no. 1, pp. 125-138, 2016. [84]

Y. Yu, H. Guan, D. Zai, and Z. Ji, "Rotation-and-scale-invariant airplane detection in highresolution satellite images based on deep-Hough-forests," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 112, pp. 50-64, 2016.

[85]

X. Chen, S. Xiang, C. L. Liu, and C. H. Pan, "Vehicle Detection in Satellite Images by Hybrid Deep Convolutional Neural Networks," IEEE Geosci. Remote Sens. Lett. , vol. 11, no. 10, pp. 1797-1801, 2014.

[86]

J. Han, D. Zhang, G. Cheng, L. Guo, and J. Ren, "Object Detection in Optical Remote Sensing Images Based on Weakly Supervised Learning and High-Level Feature Learning," IEEE Trans.

[87]

IP T

Geosci. Remote Sens. , vol. 53, no. 6, pp. 3325-3337, 2015.

J. Wang, J. Song, M. Chen, and Z. Yang, "Road network extraction: a neural-dynamic framework

based on deep learning and a finite state machine," Int. J. Remote Sens. , vol. 36, no. 12, pp. 3144[88]

SC R

3169, 2015.

J. Tang, C. Deng, G. B. Huang, and B. Zhao, "Compressed-Domain Ship Detection on Spaceborne Optical Image Using Deep Neural Network and Extreme Learning Machine," IEEE Trans. Geosci. Remote Sens. , vol. 53, no. 3, pp. 1174-1185, 2015.

[89]

A. W. Whitney, "A direct method of nonparametric measurement selection," IEEE Trans.

U

Comput. , vol. 100, no. 9, pp. 1100-1103, 1971.

[90] T. Marill and D. M. Green, "On the effectiveness of receptors in recognition systems," IEEE Trans. [91]

N

Inf. Theory vol. 9, no. 1, pp. 11-17, 1963.

P. Pudil, J. Novovičová, and J. Kittler, "Floating search methods in feature selection," Pattern R. Y. Nakamura, L. M. G. Fonseca, J. A. Dos Santos, R. d. S. Torres, X.-S. Yang, and J. P. Papa,

M

[92]

A

Recognit. Lett. , vol. 15, no. 11, pp. 1119-1125, 1994.

"Nature-inspired framework for hyperspectral band selection," IEEE Trans. Geosci. Remote Sens. , vol. 52, no. 4, pp. 2126-2137, 2014.

Y.-H. Chiang and L.-L. Lin, "Mapping overused slopelands from SPOT data using support vector

ED

[93]

machines and artificial neural networks," J. Appl. Remote Sens. , vol. 7, no. 1, pp. 073597-073597, 2013.

L. Bruzzone, C. Mingmin, and M. Marconcini, "A Novel Transductive SVM for Semisupervised

PT

[94]

Classification of Remote-Sensing Images," IEEE Trans. Geosci. Remote Sens. , vol. 44, no. 11, pp. 3363-3373, 2006.

L. Gomez-Chova, G. Camps-Valls, J. Munoz-Mari, and J. Calpe, "Semisupervised Image

CC E

[95]

Classification With Laplacian Support Vector Machines," IEEE Geosci. Remote Sens. Lett. , vol. 5, no. 3, pp. 336-340, 2008.

[96]

X. Yao and Y. Liu, "A new evolutionary system for evolving artificial neural networks," IEEE Trans. Neural Networks vol. 8, no. 3, pp. 694-713, 1997.

A

[97]

[98]

Y. Zhong, L. Zhang, and P. Li, "Multispectral remote sensing image classification based on simulated annealing clonal selection algorithm," in Preceedings IEEE International Geoscience and Remote Sensing Symposium (IGARSS '05), pp. 3745-3748, 2005. B. Gong, J. Im, and G. Mountrakis, "An artificial immune network approach to multi-sensor land use/land cover classification," Remote Sens. Environ. , vol. 115, no. 2, pp. 600-614, 2011.

[99]

O. Cordón, F. Herrera, F. Hoffmann, and L. Magdalena, Genetic Fuzzy Systems: Evolutionary Tuning and Learning of Fuzzy Knowledge Bases: Singapore: World Scientific, 2001.

[100] M.-H. Tseng, S.-J. Chen, G.-H. Hwang, and M.-Y. Shen, "A genetic algorithm rule-based

approach for land-cover classification," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 63, no. 2, pp. 202-212, 2008. [101] D. G. Stavrakoudis, G. N. Galidaki, I. Z. Gitas, and J. B. Theocharis, "A genetic fuzzy-rule-based classifier for land cover classification from hyperspectral imagery," IEEE Trans. Geosci. Remote Sens. , vol. 50, no. 1, pp. 130-148, 2012. [102] C. Puente, G. Olague, S. V. Smith, S. H. Bullock, A. Hinojosa-Corona, and M. A. GonzálezBotello, "A genetic programming approach to estimate vegetation cover in the context of soil erosion assessment," Photogrammetric Engineering & Remote Sensing, vol. 77, no. 4, pp. 363376, 2011.

IP T

[103] A. K. Jain, "Data clustering: 50 years beyond K-means," Pattern Recognit. Lett. , vol. 31, no. 8, pp. 651-666, 2010.

[104] E. R. Hruschka, R. J. G. B. Campello, A. A. Freitas, and A. P. L. F. De Carvalho, "A survey of

SC R

evolutionary algorithms for clustering," IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. , vol. 39, no. 2, pp. 133-155, 2009.

[105] U. Maulik and S. Bandyopadhyay, "Performance evaluation of some clustering algorithms and validity indices," IEEE Trans. Pattern Anal. Mach. Intell. , vol. 24, no. 12, pp. 1650-1654, 2002. [106] U. Maulik and S. Bandyopadhyay, "Genetic algorithm-based clustering technique," Pattern

U

Recognit. , vol. 33, no. 9, pp. 1455-1465, 2000.

[107] N. R. Pal and J. C. Bezdek, "On cluster validity for the fuzzy c-means model," IEEE Trans. Fuzzy

N

Syst. , vol. 3, no. 3, pp. 370-379, 1995.

[108] A. A. Naeini, S. Homayouni, and M. Saadatseresht, "Improving the Dynamic Clustering of

A

Hyperspectral Data Based on the Integration of Swarm Optimization and Decision Analysis,"

M

IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. (JSTARS), vol. 7, no. 6, pp. 2161-2173, 2014.

[109] J. Luo, L. Jiao, and J. A. Lozano, "A Sparse Spectral Clustering Framework via Multi-Objective

ED

Evolutionary Algorithm," IEEE Trans. Evol. Comput. , vol. PP, no. 99, pp. 1-1, 2015. [110] L. Li, X. Yao, R. Stolkin, M. Gong, and S. He, "An Evolutionary Multi-objective Approach to Sparse Reconstruction," IEEE Trans. Evol. Comput. , vol. PP, no. 99, pp. 1-1, 2013.

PT

[111] Y. S. Ong, M. Lim, and X. Chen, "Memetic computation—Past, present & future," IEEE Comput. Intell. Mag. , vol. 5, no. 2, pp. 24-31, 2010. [112] Z. Zhu, S. Jia, and Z. Ji, "Towards a Memetic Feature Selection Paradigm," IEEE Comput. Intell.

CC E

Mag. , vol. 5, no. 2, pp. 41-53, 2010. [113] X. Chen, L. Feng, and Y. S. Ong, "A self-adaptive memeplexes robust search scheme for solving stochastic demands vehicle routing problem," Int. J. Syst. Sci. , vol. 43, no. 7, pp. 1347-1366, 2012.

A

[114] Y. S. Ong, M. Lim, N. Zhu, and K. Wong, "Classification of adaptive memetic algorithms: a comparative study," IEEE Trans. Syst. Man Cybern. Part B Cybern. , vol. 36, no. 1, pp. 141-152, 2006.

[115] S. Das and A. Konar, "Automatic image pixel clustering with an improved differential evolution," Appl. Soft Comput., vol. 9, no. 1, pp. 226-236, 2009. [116] T. Blaschke, G. J. Hay, M. Kelly, S. Lang, P. Hofmann, E. Addink, R. Queiroz Feitosa, F. van der Meer, H. van der Werff, F. van Coillie, and D. Tiede, "Geographic Object-Based Image Analysis – Towards a new paradigm," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 87, no. 0, pp. 180-191, 2014.

[117] J. Sivic and A. Zisserman, "Video Google: A text retrieval approach to object matching in videos," in Proceedings Ninth IEEE International Conference on Computer Vision, pp. 1470-1477, 2003. [118] A. Bosch, A. Zisserman, and X. Muñoz, "Scene Classification Using a Hybrid Generative/Discriminative Approach," IEEE Trans. Pattern Anal. Mach. Intell. , vol. 30, no. 4, pp. 712-727, 2008. [119] Y. Zhong, Q. Zhu, and L. Zhang, "Scene Classification Based on the Multifeature Fusion Probabilistic Topic Model for High Spatial Resolution Remote Sensing Imagery," IEEE Trans. Geosci. Remote Sens. , vol. 53, no. 11, pp. 6207-6222, 2015. [120] G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural

IP T

networks," Science, vol. 313, no. 5786, pp. 504-507, 2006.

[121] S. Kabanikhin, "Definitions and examples of inverse and ill-posed problems," Journal of Inverse and Ill-Posed Problems, vol. 16, no. 4, pp. 317-357, 2008.

SC R

[122] J. M. Bioucas-Dias, A. Plaza, N. Dobigeon, M. Parente, Q. Du, P. Gader, and J. Chanussot,

"Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. (JSTARS), vol. 5, no. 2, pp. 354-379, 2012.

[123] X. Tong, X. Xu, A. Plaza, H. Xie, H. Pan, W. Cao, and D. Lv, "A New Genetic Method for

U

Subpixel Mapping Using Hyperspectral Images," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. (JSTARS), vol. PP, no. 99, pp. 1-12, 2016.

N

[124] L. Yue, H. Shen, J. Li, Q. Yuan, H. Zhang, and L. Zhang, "Image super-resolution: The techniques, applications, and future," Signal Process., vol. 128, pp. 389-408, 2016.

A

[125] W. He, H. Zhang, L. Zhang, and H. Shen, "Hyperspectral Image Denoising via Noise-Adjusted

M

Iterative Low-Rank Matrix Approximation," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. (JSTARS), vol. 8, no. 6, pp. 3050-3061, 2015.

[126] X. Xu, Y. Zhong, L. Zhang, and H. Zhang, "Sub-Pixel Mapping Based on a MAP Model With

ED

Multiple Shifted Hyperspectral Imagery," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. (JSTARS), vol. 6, no. 2, pp. 580-593, 2013. [127] Z. Zheng, X. Yong, Y. Jian, L. Xuelong, and D. Zhang, "A Survey of Sparse Representation:

PT

Algorithms and Applications," Access, IEEE, vol. 3, pp. 490-530, 2015. [128] D. E. O'Leary, "Artificial intelligence and big data," IEEE Intell. Syst. , vol. 28, no. 2, pp. 009699, 2013.

CC E

[129] Y. T. Zhai, Y. S. Ong, and I. W. Tsang, "The Emerging? Big Dimensionality?," IEEE Comput. Intell. Mag. , vol. 9, no. 3, pp. 14-26, 2014.

[130] L. Feng, Y. S. Ong, M. H. Lim, and I. W. Tsang, "Memetic Search With Interdomain Learning: A Realization Between CVRP and CARP," IEEE Trans. Evol. Comput. , vol. 19, no. 5, pp. 644-658, 2015.

A

[131] J. Y. Chia, C. K. Goh, V. A. Shim, and K. Tan, "A data mining approach to evolutionary optimisation of noisy multi-objective problems," Int. J. Syst. Sci. , vol. 43, no. 7, pp. 1217-1247, 2012.

[132] L. Mussi, F. Daolio, and S. Cagnoni, "Evaluation of parallel particle swarm optimization algorithms within the CUDA™ architecture," Inf. Sci., vol. 181, no. 20, pp. 4642-4657, 2011. [133] M. Ye, Y. Qian, and J. Zhou. "Multitask Sparse Nonnegative Matrix Factorization for Joint Spectral–Spatial Hyperspectral Imagery Denoising." IEEE Trans. Geosci. Remote Sens. , vol. 53, no. 5, pp. 2621-2639, 2015.

[134] W. He, H. Zhang, and L. Zhang, "Sparsity-Regularized Robust Non-Negative Matrix Factorization for Hyperspectral Unmixing," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. (JSTARS), vol. 9, no. 9, pp. 4267-4279, 2016. [135] X. Xu, X. Tong, A. Plaza, Y. Zhong, H. Xie, and L Zhang. Joint Sparse Sub-Pixel Mapping Model with Endmember Variability for Remotely Sensed Imagery[J]. Remote Sens., 9(1), pp.1-20, 2016. [136] A. Gupta, Y. S. Ong, and L. Feng, "Multifactorial Evolution: Towards Evolutionary Multitasking", IEEE Trans. Evol. Comput. vol. 20, no. 3, pp. 343 - 357, 2016. [137] A. Gupta, B. Da, Y. Yuan, YS Ong. On the Emerging Notion of Evolutionary Multitasking: A Computational Analog of Cognitive Multitasking. In: Bechikh S., Datta R., Gupta A. (eds) Recent

IP T

Advances in Evolutionary Multi-objective Optimization. Adaptation, Learning, and Optimization,

A

CC E

PT

ED

M

A

N

U

SC R

vol 20. Springer, Cham, 2017.

SC R

IP T CC E

PT

ED

M

A

N

U

Fig. 1. Analysis of the self-learning capability.

A

Fig. 2. Analysis of the global search capability of evolutionary algorithms.

IP T SC R U

CC E

PT

ED

M

A

N

Fig. 3. Analysis of the capability of handling a multi-objective optimization problem.

A

(a) Remote sensing, differential evolution, (b) Remote sensing, back-propagation neural particle swarm optimization network, deep learning Fig. 4. The number of published papers by year for the different techniques of computational intelligence in remote sensing image processing.

IP T SC R U N A M ED

A

CC E

PT

Fig. 5. The general workflow of optical remote sensing image processing.

Fig. 6. Different types of feature representation for remote sensing images.

IP T SC R

PT

ED

M

A

N

U

Fig. 7. The basic encoding strategy for hyperspectral band selection.

A

CC E

Fig. 8. The development of computational intelligence based hyperspectral band selection methods.

PT

ED

M

A

N

U

IP T

SC R

Fig. 9. The basic process of ANN-based classification for remote sensing images.

A

CC E

Fig. 10. The AIS-based classification method for remote sensing images.

IP T

ED

M

A

N

U

SC R

(a) RGB false-color image (b) The ground-truth image land-cover classes (52,27,17) Fig. 11. AVIRIS Indian Pines image.

(b) SVM

A

CC E

PT

(a) MLC

(d) RLCRSI (e) ABNet Fig. 12 Classification results for the Indian Pines image [12].

(c) BPNN

IP T

Fig. 13. The basic encoding strategy of the evolutionary algorithm based remote sensing

M

A

N

U

SC R

clustering methods.

A

CC E

PT

ED

Fig. 14. The development of evolutionary algorithm based clustering methods for remote sensing images.

Fig. 15. Two types of local searchers for clustering and segmentation, respectively.

IP T SC R

CC E

PT

ED

M

A

N

U

(a) RGB false-color image (b) The ground-truth image land-cover classes Fig. 16. Wuhan Thematic Mapper (TM) image.

(b) FCIDE

A

(a) FCM

(d) AFCMDE

(e) AFCMOMA

(c) AMASFC

A

CC E

PT

ED

M

A

N

U

SC R

IP T

Fig. 17. Clustering results for the Indian Pines image [18, 53, 60].

IP T SC R

PT

ED

M

A

N

U

Fig. 18. The basic encoding strategy of change detection for remote sensing images.

A

CC E

Fig. 19. The strategy of ANN-based change detection for remote sensing images.

Table I The land-cover classes and corresponding numbers of samples for the Indian Pines image. Number of Number of Class Class samples samples C1. Corn-no till 1434 C6. Soybeans-no till 968 834

C7. Soybeans-min till

2468

C3. Grass/Pasture

497

C8. Soybeans-clean till

614

C4. Grass/Tree

747

C9. Woods

1294

C5. Hay-windrowed

489

C10. Bldg-Grass-Tree-Drives

380

C6. Soybeans-no till

968

Overall

9725

SC R

IP T

C2. Corn-min till

Table II Empirical comparison of different classifiers for the Indian Pines image. SVM

BPNN

RLCRSI

ABNet

OA(%)

57.51

81.85

75.40

82.04

85.41

Kappa

0.4713

0.7869

0.7191

0.7910

0.8313

ED

M

A

N

U

MLC

River 2577

Vegetation 4098

Lake 1559

Bare soil 3037

Building 1666

CC E

Class Number

PT

Table III Land-cover classes and the corresponding numbers of training samples for the Wuhan TM image.

A

Table IV Comparison between different clustering methods for the Wuhan TM image. FCM

FCIDE

AMASFC

AFCMDE

AFCMOMA

OA(%)

81.66

81.75

86.55

86.38

91.29

Kappa

0.7619

0.7652

0.8276

0.8262

0.8882