A common problem across science and engineering is that aspects of models have to be estimated from observed data. An instance of this familiar to control engineers is system identification. Bayesian inference is a principled way to estimate parameters: exploiting Bayes~ theorem, an equational probabilistic model is “inverted”, yielding a probability distribution for the unknown parameters given the observations. This paper presents Ebba, a declarative language for proba- bilistic modelling where models can be used both “forwards” for probabilistic computation and “backwards” for parameter estimation. The novel aspect of Ebba is its implementation: a shallow, arrows-based, embedding. This provides a clear semantical account and ensures that only models that support estimation can be expressed. As arrow-like notions have proved useful in modelling dynamical systems, this might also suggest an approach to an integrated language for modelling dynamical systems and parameter estimation.