Augmented Reality technologies are continually evolving and changing our interaction with the environment. While visual extensions of the environment are prevalent in leisure time as well as at the workplace, hardly any auditory changes are applied to the environment. The best-known application of Augmented Sound is active noise-canceling headphones, which enable the filtering of distracting background noise. Currently this approach of changing our soundscape is being pursued and expanded. Nowadays, we are capable of amplifying the sound coming from our point of view and integrating virtual sounds into the environment, but it is currently not possible to completely adjust our soundscape. In this work, we investigate the expectations and needs of potential end users regarding the control of Augmented Hearing. We have conducted three consecutive studies to identify not only desired actions but also control modalities. In an elicitation study, possible gestures/interactions for three different modalities were identified and verified through a large-scale validation study. Based on the study results, we can suggest applications for Augmented Sound and user-friendly control methods.