In this paper we describe a policy based authorisation infrastructure that a cloud provider can run as an infrastructure service for its users. It will protect the privacy of usersʼ data by allowing the users to set their own privacy policies, and then enforcing them so that no unauthorised access is allowed to their data. The infrastructure ensures that the usersʼ privacy policies are stuck to their data, so that access will always be controlled by the policies even if the data is transferred between cloud providers or services. This infrastructure also ensures the enforcement of privacy policies which may be written in different policy languages by multiple authorities such as: legal, data subject, data issuer and data controller. A conflict resolution strategy is presented which resolves conflicts among the decisions returned by the different policy decision points (PDPs). The performance figures are presented which show that the system performs well and that each additional PDP only imposes a small overhead.