Utility, defined as the perceived satisfaction with a service, provides the ideal means for decision making on the level of individual entities and collectives participating in a large-scale dynamic system. Previous works have already introduced the concept into the area of collective adaptive systems, and have discussed what is the necessary infrastructure to support the realization of the involved theoretical concepts into actual decision making. In this work we focus on two aspects. First, we provide a concrete utility model for a case study that is part of a larger research project. Second, we incorporate this model into our implementation of the proposed architecture. More importantly, we design and execute an experiment that aims to empirically evaluate the use of utility for decision making by comparing it against simpler decision making mechanisms.