The rationale behind the ever increasing combined adoption of Artificial Intelligence and Internet of Things (IoT) technologies in the industry lies in its potential for improving resource efficiency of the manufacturing process, reducing capital and operational expenditures while minimizing its carbon footprint. Nonetheless, the synergetic application of these technologies is hampered by several challenges related to the complexity, heterogeneity and dynamicity of industrial scenarios. Among these, a key issue is how to reliably deliver target levels of data quality and veracity, while effectively supporting a heterogeneous set of applications and services, ensuring scalability and adaptability in dynamic settings. In this paper we perform a first step towards addressing this issue. We outline ABIDI, an innovative and comprehensive Industrial IoT reference architecture, enabling context-aware and veracious data analytics, as well as automated knowledge discovery and reasoning. ABIDI is based on the dynamic selection of the most efficient IoT, networking and cloud/edge technologies for different scenarios, and on an edge layer that efficiently supports distributed learning, inference and decision making, enabling the development of real-time analysis, monitoring and prediction applications. We exemplify our approach on a smart building use case, outlining the key design and implementation steps which our architecture implies.