While neural networks nowadays are widely used in different areas and applications, many usages of such neural networks are hindered by lacking computation power, especially on smaller devices like mobile devices. For this reason, it is very attractive to find ways to enable such devices to use neural networks under their restricted hardware conditions. There are different ways to approach that goal, but I will focus on an approach that enhances a given neural network to perform better on the same device and data without any distribution on other devices or reduction of potential redundancy. To achieve a better performance, I will create a supporting structure consisting of at least one supporting network processing input data instead of the original network. To decide which network processes which input, I will create a deciding neural network which will allocate each input value to exactly one network. This requires more computation in the training process because I will have to search the supporting and the deciding network and train them accordingly, but will reduce the error in the output of the whole network construction compared to the original network.