We consider a hybrid approach for the approximation of the solution to parametric partial differential equations based on finite elements and deep neural net-works. Finite element simulations with adaptive mesh refinement are used to generate input data for the training of a neural network. A deep feedforward neural network is then used to approximate the solution of the partial differential equation. We aim at balancing the numerical errors introduced by the finite element method and the neural network approximation respectively. Numerical results are presented for the transport equation.