In a conversation with another person, it is totally normal to ask a clarifying ques- tion if one does not understand something or if something is ambiguous. A clarifying question is the most common and convenient way to solve such a problem. However, in a conversation with a conversational system, it does not respond with a clarifying question if there is an ambiguity. Moreover, the research in this are is still limited. There are primarily only two types of responses from the system. Either it answers the request of the user (which could be the wrong answer if the request is ambiguous), or it responds that it does not understand the request. Therefore, making the conversational system able to generate clarifying questions would lead to a better user experience, because more accurate final responses will be delivered to the user. We address this deficiency by developing a system that is able to generate clarifying questions in an open-domain dialogue setting. Our system is constructed as a pipeline so that several processing steps are easily adaptable and replaceable. Therefore, we used state-of-the-art methods to achieve a system that is able to generate clarifying questions dynamically and on the fly and not just selects the best matching from a predefined set. Thereby, our approach generates the clarifying questions based on the most probable answer that is extracted from web search results. Due to this, our approach offers the advantage of directly having the answer at hand to provide the user the answer in case the clarifying question covers the user’s intent of the request. This work gives a detailed insight into the complete development process, with all problems and the solutions we found to fix them. We automatically evaluate the generated clarifying questions on several aspects with metrics like BLEURT and Distinct-N. Moreover, we manually analyze the results to find conspicuities in the generated questions. We conduct a study to collect a quality evaluation from a broader range of people. It shows that our system is indeed able to generate appropriate clarifying questions and reveals interesting correlations between different quality aspects. Our work demonstrates that the generation of clarifying questions in an open-domain setting is possible with our novel approach.