Towards co-evolution of fitness predictors and Deep Neural Networks

Abstract: Deep neural networks proved to be a very useful and powerful tool with many practical applications. They especially excel at learning from large data sets with labeled samples. However, in order to achieve good learning results, the network architecture has to be carefully designed. Creating an optimal topology requires a lot of experience and knowledge. Unfortunately there are no practically applicable algorithms which could help in this situation. Using an evolutionary process to develop new network topologies might solve this problem. The limiting factor in this case is the speed of evaluation of a single specimen (a single network architecture), which includes learning based on the whole large dataset. In this paper we propose to overcome this problem by using a fitness prediction technique: use subsets of the original training set to conduct the training process and use its results as an approximation of specimen's fitness. We discuss the feasibility of this approach in context of the desired fitness predictor features and analyze whether subsets obtained in an evolutionary process can be used to estimate the fitness of the network topology. Finally we draw conclusions from our experiments and outline plans for future work.