The popularity of the Internet of Things is rapidly increasing and so is the range of IoT application scenarios. The prominent examples are Smart Homes, Industry 4.0 and Car2X. Undoubtedly, there are huge benefits for end-users in these scenarios. However, due to the pervasiveness of the Internet of Things, the amount of private and sensitive data captured in IoT environments is of great magnitude. A central IoT platform is often used to manage this data. That is, privacy concerns arise which have to be dealt with, e.g., by the vendors of IoT platforms. To address the privacy related issues in IoT environments, we propose a concept for privacy and access control particularly tailored for IoT environments. In a first step, an exhaustive evaluation of related work in the field of access control is conducted. We then present a generic end-to-end reference architecture for IoT environments. Based on this reference architecture, we examine the special requirements towards privacy and access control in IoT environments. Following that, we present a generic concept for IoT environments that allows different ways of integration, provides flexible yet expressive specification of privacy settings, is highly customizable, and introduces minimal overhead for access control enforcement. The concept provides details about the integration into existing IoT environments, the access control workflow and components as well as the suggested domain model and evaluation algorithm. A concrete implementation is illustrated with the Multi-purpose Binding and Provisioning Platform (MBP) developed at the University of Stuttgart.