We will introduce the basic notion of Functional Data Analysis (FDA) - that the observed data are functions of a continuous variable. Practical FDA requires finite dimensional representation. We argue that a foundational requirement of FDA is the Grid Refinement Invariance Principle: as the funite dimensional representation becomes more accurate, any statistical inferences should converge to the equivalent inference on the idealized functional observations. For Bayesians, the easiest way to achieve this is to put a prior on the infinite dimensional function space, which can be challenging. We shall consider a Gaussian sampling model for functional data and discuss the challenges of creating a prior for the unknown covariance of the Gaussian measure. We will give a practical prior which can be used and give some numerical results. We will also present preliminary work on priors obtained by limits of finite dimensional inverted Wishart distributions. This is joint work with Hong Xiao Zhu.