Improving neural network based spectral clustering

Abstract

Spectral Clustering is a well known clustering method which overcomes the limitations of traditional clustering algorithms like k-means clustering. The Algorithm involves finding eigenvectors of a graph Laplacian which has two main drawbacks, namely, scalability and out-of-sample predictions. SpectralNet is a deep neural network (NN) based method which overcomes these limitations but uses Cholesky factorization to obtain output orthogonal matrix and is not an end-to-end network. This method only performs well when the Laplacian matrix is highly sparse. The model is also highly sensitive to the hyperparameter setting. We developed an end-to-end neural network architecture called Extended Spectral Clustering (ExSC) which employes beta-VAE and cayley map to orthogonalize the input matrices and minimize the spectral loss to update the network weights so as to obtain orthogonal output which better resembles the eigenvector matrix of the Laplacian. The model performs better than the SpectralNet base model. Also, as the model learns an encoder and a decoder, it can also be used as a generative model or a feature extractor to simultaneously perform other tasks.