During the last two decades, GPUs developed into powerful and massively parallel processors. That rose the attention of scientist who started using GPUs for large scale scientific computing, e.g. simulations. However, the architecture of GPUs is different from CPUs. Furthermore, graphic processors have their now fast access memory. Computing in a heterogeneous system consisting of a CPU and multiple GPUs has various challenges. In this work, we focus on how to distribute the load among the different components. We consider an iterative load that can be redistributed after each iteration. The goal of our scheduling methods is to minimise the computation time of the next iteration by estimating the performance of each component. After a short introduction to load balancing, we specify the iterative workload scenario and differentiate it from the typical task-based scenario often found in the literature. Then, we show the basics of GPU programming with the help of NVIDIAs CUDA API. Furthermore, we introduce the different kernels we use for our test and derive multiple schedulers. Our dynamic schedulers use the time each component took to compute its assigned workload in the last iteration as a basis of the performance estimation. After investigating the influence of previous run-time data on the scheduling decisions, we turn our attention towards the properties of the workloads and therefore compare different types of memory management.