The proliferation of sensing device technologies, and the growing demand for data intensive IoT applications calls for a seamless interconnection of IoT, edge and cloud resources in one computing system, to form a Compute Continuum, also referred to as edge-to-cloud. This paper targets self-adaptive Machine Learning applications that rely on data coming from IoT sensors. These applications are often “context-aware”, with high context sensitivity, different physical settings and complex usage patterns. Their intelligence, deployed on the edge, is updated on the fly. We present two Compute Continuum strategies for the deployment of such applications: (1) a centralised approach, which involves training a model on a centralised server, and (2) a decentralised approach using Federated Learning. The former approach involves centralising data from multiple sources onto a single server, while the latter locally decentralises both the training process and the aggregation and communication tasks across edge devices. In both cases the inference model is deployed on edge devices close to the collected data. The decentralised architecture relies on a coordination platform favouring self-adaptation and decentralised Federated Learning. Results show that the decentralised Federated Learning approach offers networking performances and privacy-preserving advantages compared to non-private centralised models, with a slight trade-off in prediction accuracy. According to our simulations, the deployment cost of the decentralised architecture is much lower than that of deployment on the centralised architecture.