Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture
Abstract
The separation of independent sources from an array of sensors is a classical but difficult problem in signal processing. Based on some biological observations, an adaptive algorithm is proposed to separate simultaneously all the unknown independent sources. The adaptive rule, which constitutes an independence test using non-linear functions, is the main original point of this blind identification procedure. Moreover, a new concept, that of INdependent Components Analysis (INCA), more powerful than the classical Principal Components Analysis (in decision tasks) emerges from this work.
Zusammenfassung
Die Trennung unabhängiger Quellen stellt ein klassiches jedoch schwieriges Problem bei der Signalverarbeitung dar. Aufgrund neurobiologischer Beobachtungen stellen wir in diesem Artikel einen selbstanpassenden Algorithmus vor, der gleichzeitig alle unbekannten, unabhängigkeitstest unter Anwendung von nicht linearen Funktionen darstellt, ist der zentralste Punkt dieses blindend Identifikationsverfahrens. Ausserdem hebt sich ein neues Konzept, das der unabhängigen Komponenten-Analyse (INCA), leistungsfähiger in den Entscheidungsvorgängen als die Analyse der Hauptkomponenten, aus dieser Arbeit hervor.
Résumé
La séparation de sources indépendantes constitue un problème classique mais difficile de traitement du signal. D'après des observations neurobiologiques, nous proposons dans cet article un algorithme auto-adaptatif capable de séparer simultanément toutes les sources indépendantes inconnues. La règle d'adaptation, qui effectue un test d'indépendance grâce à l'utilisation de fonctions non-linéaires, est le point le plus central de cette méthode d'identification aveugle. De plus, un nouveau concept, celui d'analyse en composantes indépendantes (INCA), plus puissant dans les opérations de décision que celui d'analyse en composantes principales, émerge de ce travail.
References (17)
- S. Grossberg
Nonlinear neural networks: Principles, mechanisms, and architectures
Neural Networks
(1988) - J.-F. Cardoso, “Eigen structure of the 4th order cumulant tensor with application to the blind sources separation...
- F. Christophe et al.
Elimination auto-adaptive de brouilleurs par une méthode de gradient appliquée à un réseau lacunaire
- P. Comon
Separation of stochastic processes whose linear mixture is observed
- P. Duvaut
Principes des méthodes de séparation de sources fondées sur les moments d'ordre supérieur
- L. Fety
Méthodes de traitement d'antenne adaptées aux radiocommunications
- D.O. Hebb
The Organization of the Behaviour
(1949) - J. Hérault et al.
Détection de grandeurs primitives dans un message composite par une architecture de calcul neuromimétique en apprentissage non supervisé
Cited by (2087)
The impact of global warming on ENSO from the perspective of objective signals
2024, Atmospheric ResearchMost studies on the relationship between global warming and El Niño-Southern Oscillation (ENSO) rely on simulation experiments, but they have produced contradictory results, which cannot guarantee the reliability of the experimental findings. To address this challenge, we approached the issue from the perspective of objective ENSO and global warming signals. The effects of global warming on ENSO are not limited to the Earth's surface, but also present in the atmosphere. While objective monitoring indices exist for surface global warming and ENSO signals, objective ENSO and global warming signals in the atmosphere are difficult to obtain. To obtain high-precision ENSO and global warming signals in the atmosphere, we propose using the strategy of ICA (Independent Component Analysis) combined with a non-parametric method. The results of the objective ENSO and global warming signals suggest that the global warming signal leads the ENSO signal by approximately three months and exhibits a clear positive correlation with the ENSO signal, especially the occurrence of strong ENSO events is closely related to global warming, indicating that global warming promotes the occurrence of ENSO events. Under the influence of global warming, the frequency and intensity of ENSO increase, the characteristics and complexity of ENSO atmospheric teleconnections may be altered, and there is a greater tendency for more destructive El Niño events to occur. Since the beginning of the 21st century, the probability of ENSO event has increased significantly. Furthermore, moderate or strong ENSO events tend to develop earlier, and extreme ENSO events not only develop earlier but also prolong their decay phase, leading to an extended lifespan of ENSO events. This paper investigates the effect of global warming on ENSO from the perspectives of objective ENSO and global warming signals, which well supplements the objective factual basis for the previous simulation experimental studies.
A review on the application of blind source separation in vibration analysis of mechanical systems
2024, Measurement: Journal of the International Measurement ConfederationVibration is a widespread phenomenon in mechanical systems, and vibration analysis is imperative for assessing the dynamic characteristics of mechanical structures. Vibration analysis extensively relies on analyzing the vibration signals. Blind source separation (BSS) was introduced as a pattern separation technique in vibration signal analysis in the 21st century. BSS has provided innovative solutions for various vibration problems by effectively recovering desired source signals from mixed signals despite limited prior information. This paper reviews the application research of BSS in vibration analysis in the past decade, focusing on detailed explanations in the areas of fault diagnosis, operational modal analysis (OMA), and load identification. Firstly, the paper presents the fundamental theory of BSS, encompassing several prominent algorithms: independent component analysis (ICA), non-negative matrix factorization (NMF), sparse component analysis (SCA), bounded component analysis (BCA) and deep learning-based BSS. Subsequently, corresponding to the three key elements of structural dynamics, namely, response, system, and excitation, the paper categorizes and organizes the research on the application of BSS in the areas of fault diagnosis, OMA, and load identification, summarizing the methodological approaches in relevant literature for specific problems in practical applications. Finally, the paper presents a future perspective on the application prospects of BSS in vibration analysis.
NeuroDAVIS: A neural network model for data visualization
2024, NeurocomputingThe task of dimensionality reduction and visualization of high-dimensional datasets remains a challenging problem since long. Modern high-throughput technologies produce large datasets having multiple views with relatively new data types. Visualization of these datasets require efficient algorithms that can uncover hidden patterns in the data without affecting the local and global structures, and bring out the inherent non-linearity within the data. To this end, however, very few such methodologies exist, which can realize this task. In this work, we have introduced a novel unsupervised deep neural network model, called NeuroDAVIS, for data visualization. NeuroDAVIS is capable of extracting important features from the data, without assuming any data distribution, visualize effectively in low dimension, and preserve both local and global structures simultaneously. It has been shown theoretically that neighbourhood relationship of the data in high dimension remains preserved in lower dimension. The performance of NeuroDAVIS has been evaluated on a wide variety of synthetic and real high-dimensional datasets including numeric, textual, image and biological data. NeuroDAVIS has been highly competitive against t-Distributed Stochastic Neighbor Embedding (t-SNE), Uniform Manifold Approximation and Projection (UMAP), Principal Component Analysis (PCA) and Random Projection (RP) with respect to visualization quality, preservation of data size, shape, and both local and global structure. It has outperformed Fast interpolation-based t-SNE (Fit-SNE), a variant of t-SNE, for most of the datasets. For the biological datasets, besides t-SNE, UMAP, Fit-SNE, PCA and RP, NeuroDAVIS has also performed well as compared to other state-of-the-art algorithms, like Potential of Heat-diffusion for Affinity-based Trajectory Embedding (PHATE) and the siamese neural network-based method, called IVIS. Downstream classification and clustering analyses have also revealed favourable results for NeuroDAVIS-generated embeddings.
A survey of artificial intelligence approaches in blind source separation
2023, NeurocomputingIn various signal processing applications, such as audio signal recovery, the extraction of desired signals from a mixture of other signals is a crucial task. To achieve superior performance and efficiency in separator systems, extensive research has been conducted. Blind source separation emerges as a relevant technique to address the challenge of separating and reconstructing unknown signals when only observations of their mixtures are available to end-users. Blind source separation involves retrieving a set of independent source signals mixed by an unknown and potentially destructive combining system. Notably, the separation process in blind source separation frameworks solely relies on observing the mixed sources without prior knowledge of the mixing algorithm or the source signal characteristics. The significance of blind source separation has garnered substantial attention, and its numerous applications have been demonstrated, which serves as the primary motivation for conducting this comprehensive study. This paper presents a systematic literature survey of blind source separation, encompassing existing methods, approaches, and applications, with a particular focus on artificial intelligence-based frameworks. Through a thorough review and examination, this work sheds light on the diverse techniques utilized in blind source separation and their performance in real-world scenarios. The study identifies research gaps in the current literature, highlighting areas that warrant further investigation and improvement. Moreover, potential avenues for future research are outlined to contribute to the ongoing development of blind source separation techniques.
Two phase cooperative learning for supervised dimensionality reduction
2023, Pattern RecognitionThe simultaneous minimization of the reconstruction and classification error is a hard non convex problem, especially when a non-linear mapping is utilized. To overcome this obstacle, motivated by the widespread success of Cooperative Neural Networks, an innovative supervised dimensionality reduction framework is proposed, based on a cooperative two phase optimization strategy. Specifically, the proposed framework that requires minimal parameter adjustment consists of an autoencoder for dimensionality reduction and a separator network for separability assessment of the embedding. This scheme results in meaningful and discriminable codes, which are optimized for the classification task and are exploitable by any trainable classifier. The experimental results showed that the proposed methodology achieved competitive results against the state-of-the-art competing methods, while being much more efficient in terms of parameter count. Finally, it was empirically justified that the proposed methodology introduces advanced behavioural explainability, while enabling applicability for image generation tasks.
Nonlinear independent component analysis for principled disentanglement in unsupervised deep learning
2023, PatternsA central problem in unsupervised deep learning is how to find useful representations of high-dimensional data, sometimes called “disentanglement.” Most approaches are heuristic and lack a proper theoretical foundation. In linear representation learning, independent component analysis (ICA) has been successful in many applications areas, and it is principled, i.e., based on a well-defined probabilistic model. However, extension of ICA to the nonlinear case has been problematic because of the lack of identifiability, i.e., uniqueness of the representation. Recently, nonlinear extensions that utilize temporal structure or some auxiliary information have been proposed. Such models are in fact identifiable, and consequently, an increasing number of algorithms have been developed. In particular, some self-supervised algorithms can be shown to estimate nonlinear ICA, even though they have initially been proposed from heuristic perspectives. This paper reviews the state of the art of nonlinear ICA theory and algorithms.