Manifold alignment through deep autoencoders

The focus of this thesis is on manifold alignment methods. They are applicable when aligning two or more high-dimensional datasets for analysis and extracting knowledge. However, conventional methods have shown limitations when aligning nonlinear, noisy datasets. We hypothesized that recent developments in deep learning could help to process complex cases such as noisy, incomplete, and cross-domain datasets. As one of the main contributions of this thesis, a novel parallel deep autoencoder architecture is introduced to overcome the limitations of conventional manifold alignment methods. The parallel autoencoder model consists of two autoencoders trained alongside each other to earn manifolds of two separate datasets. A new concept for correspondence error was introduced, which facilitated the alignment of the learned manifolds. The proposed parallel model incorporated a deep design by adding multiple layers of encoders and decoders. Another main contribution of this thesis is a new experimental framework to test manifold alignment and manifold learning methods on simulated data of increasing complexity that results in visualisations of latent manifolds with dimension in the range 2-4. The datasets of the framework were constructed from simulations of four different double pendulum motion configurations, which provided high-dimensional data with intrinsic 2, 3, and 4-manifolds. Two different types of noise were incrementally added to the datasets. For comparing and testing various manifold alignment methods, this thesis developed an approach that combines quantitative evaluation of the alignment success based on manifold proximity with a qualitative assessment using visualisations. Our visual and numerical evaluation results showed that the proposed parallel deep autoencoder method surpassed the conventional alignment techniques and overcame their limitations when aligning high-dimensional noisy data. The new parallel autoencoder model was further developed to be able to process image data by adding convolutional layers to the encoders and decoders. The advancement also allowed us to apply the new model to the task of aligning datasets from different domains, including data sets of different dimensionalities. The results demonstrated that the new model is useful in cross-domain data alignment, which constituted a significant limitation of conventional methods.