Prof. Sun-Yuan Kung, Princeton University, USA (Life Fellow of IEEE)
Title：Progressive and Regressive NAS for Deep Learning: Theory and Applications
Abstract: It has recently become trendy to explore an architecture engineering process so that a machine can automatically learn the network structure as well as its parameters. This is collectively known as ``Neural Architecture Search" (NAS). In this talk, we shall present mathematical analysis on the layer sensitivity vital for both PNAS and RNAS - short for Progressive and Regressive NAS. To this end, a notion of Discriminant Matrix (DM), originally rooted on the classic works by Gauss, Fisher, and Shannon, is adopted to assess the information/redundancy embedded in a network layer. Both its trace norm, i.e. Discriminant Information" (DI), and eigenvalue analysis will play vital roles on assessing the network sensitivity revealing how to grow/prune a layer and which layer(s) to grow/prune, etc. Our analysis can precisely predict the theoretical range of the resultant DI-score upon growing or dropping a fixed number of neurons to/from a layer. For example,
For RNAS, the lower and upper bounds of the said range are fully determined by the (major and minor) eigenvalues of the DM, a critical knowledge in order for us to (1) arrive at the highest possible score, (2 determine how many of such neurons should be dropped; and (2) identify/remove the most dispensable neurons, i.e. the ones whose removal would suffer the least DiLOSS.
For PNAS, the bounds depend solely on the (major and minor) eigenvalues of an ``Innovative DM", which stems from the optimal projection from the input-residue to output-residue. (Residues are defined as the subspaces orthogonal to the targeted layer.) In order to to maximize DI, the optimal direction to grow the network must be in the same subspace defined by the principal eigenvectors of the ``Innovative DM".
The DM/DI analysis ultimately leads to an X-learning paradigm where deleterious neurons will be gradually trimmed so as to reach an improved network structure. We have recently been developing an autonomous reinforcement software based on X-Learning, named XNAS. (If time permits, we shall make a brief demo on a real-time learning performance of XNAS.) In this talk, we shall highlight some (rather exciting) simulation results by applying XNAS to (1) classification-type application scenarios: e.g. CIFAR or ImageNet; as well as (2) regression-type application scenarios: e.g. super-resolution (SR) hetero-encoder and seeing-in-the-dark.
Prof. Dong Xu, University of Missouri-Columbia, USA (AAAS Fellow; AIMBE Fellow)
Title: Applications of Graph Neural Networks in Single-Cell Sequencing Data Analyses
Abstract: Single-cell RNA-sequencing (scRNA-Seq) is widely used to reveal the heterogeneity and dynamics of tissues, organisms, and complex diseases, but its analyses still suffer from multiple challenges, including the sequencing sparsity and complex differential patterns in gene expression. We introduce scGNN (single-cell Graph Neural Network) as a hypothesis-free deep learning framework for scRNA-Seq analyses. It integrates three iterative multi-modal autoencoders and outperforms existing tools for gene imputation and cell clustering on four benchmark scRNA-Seq datasets. In an Alzheimer's disease study with 13,214 single nuclei from postmortem brain tissues, scGNN successfully illustrates disease-related neural development and the differential mechanism by identifying ten cell clusters with enriched signature genes. scGNN provides an effective representation for gene expression and cell-cell relationships, as well as a general framework that can be applied to other types of single-cell sequencing data analyses, such as single-cell spatial transcriptomic analysis and single-cell multi-omic data integration.
Prof. Yuping Wang, Tulane University, USA
Title: Interpretable multimodal deep learning for brain imaging and genomics data fusion
Abstract: Deep network-based data fusion models have been developed to capture complex associations between multi-modal datasets such as brain imaging and genomics, resulting in improved diagnosis of mental diseases. However, deep learning models are often difficult to interpret, bringing about challenges for uncovering biological mechanisms using these models. In this work, we develop an interpretable multimodal fusion model to perform automated diagnosis and result interpretation simultaneously. We name it Grad-CAM guided convolutional collaborative learning (gCAM-CCL), which is achieved by combining intermediate feature maps with gradient-based weights. The gCAM-CCL model can generate interpretable activation maps to quantify pixel-level contributions of the input features. Moreover, the estimated activation maps are class-specific, which can therefore facilitate the identification of biomarkers underlying different groups. Finally, we apply and validate the gCAM-CCL model on a brain imaging-genomics study, and demonstrate its applications to both the classification of cognitive function groups and the discovery of underlying biological mechanisms.
Prof. Philippe Fournier-Viger, Shenzhen University, China
Title: Advances and challenges for the automatic discovery of interesting patterns in data
Abstract: Discovering interesting and useful patterns in symbolic data has been the goal of numerous studies. Several algorithms have been designed to extract patterns from data that meet a set of requirements specified by a user. Although many early research studies in this domain have focused on identifying frequent patterns (e.g. itemsets, episodes, rules), nowadays many other types of interesting patterns have been proposed and more complex data types and pattern types are considered. Identifying patterns in data has applications in many fields as they provide glass-box models that are generally easily interpretable by humans either to understand the data or support decision-making. This talk will first highlight limitations of early work on frequent pattern mining and provide an overview of current problems and state-of-the-art techniques for identifying interesting patterns in symbolic data. Topics that will be discussed include high utility patterns, locally interesting patterns, compressing patterns and periodic patterns. Lastly, the SPMF open-source software will be mentioned and opportunities related to the combination of pattern mining algorithms with traditional artificial intelligence techniques will be discussed.
Prof. Yulong Bai, Northwest Normal University, China
Title: Data Assimilation, Time Series Predication and AI methods
Abstract: As an important methodology for optimally merging Earth observation information and geophysical model dynamics, data assimilation has played an important role in Earth science. In this talk, a brief introduction of data assimilation will be presented firstly. Then, we will summarize all theoretical background of the existing data assimilation methods from mathematical points of view. To take a typical sequential data assimilation method as an example, Kalman filter will be discussed. Ensemble Kalman filter (EnKF) are also represented using an ensemble of model states. Besides this, in the light of data assimilation system, we will analysis error sources and the corresponding handling methods. After a review of the current error-handling methods, a new AI method was designed to combine the advantages of different error handling methods. A new data assimilation system, coupled with genetic algorithms, is proposed to solve the difficult problem of the error adjustment factor search, which is usually performed using trial-and-error methods. In the last part of our presentation, we will review our latest research in time series predication with AI methods. Our Conclusions of this talk are: 1) Observation and modeling are two basic methods for Earth System Science research. 2) Integration of observation and model simulation needs new methodology to be developed. The rapidly developing land/hydrological data assimilation will play a key role.