It looks like you are wanting to find the relationship between displacement measurements/estimates between two locations. Let's define your target displacement variables as an input $x(t)$ and an output $y(t)$ (both in meters). Your input, $x(t)$, is a direct measurement of displacement (having been converted to meters), but your output is instead a measure of velocity, $v(t)$, (having been converted to meters per second). We can either recover the output displacement in the time domain, or the frequency domain.
Given that velocity is the time derivative of displacement,
$$\frac{d}{dt}y(t)=v(t)\tag{1}$$
to recover displacement in the time domain we perform an integration.
$$y(t) = \int{v(t)dt}\tag{2}$$
Let $\mathcal{F}$ be the Fourier transform such that $G(f)=\mathcal{F}[g(t)]$. By the integration property of the Fourier transform,
$$\mathcal{F}\left[\int{g(t)dt}\right]=\frac{1}{j2\pi f}G(f)+\pi G(0)\delta(f)\tag{3}$$
where the second term arises to define the value at $f=0$, given the undefined nature of the first term. Taking the Fourier transform of $(2)$ then gives:
$$Y(f)=\frac{1}{j2\pi f}V(f)+\pi V(0)\delta(f)\tag{4}$$
If you remove any rigid body motion, the average value of $v(t)$, prior to any "integration" step by direct subtraction of the mean or by low pass filtering, then $V(0)=0$. Depending on your application and desired result, this might make sense to do. Assuming you do remove the average value, your frequency domain estimate of the displacement becomes
$$Y(f)=\frac{1}{j2\pi f}V(f)$$
Now that you have the proper terms, it is worth reviewing the units of the Fourier transform. Assuming you operate in the discrete domain (i.e., typical Discrete Fourier Transform or DFT), your units will not change between the time and frequency domains.
To then find the relationship between the input and output, you simply divide the Fourier transform of the output by the input. This contains the phase information that Tim was referring to. Recalling your units, you should note that this result would be in units meter per meter.
$$H(f)=\frac{Y(f)}{X(f)}$$
The associated gain between the input and output is described by the Frequency Response Function (FRF):
$$FRF(f)=\frac{|Y(f)|}{|X(f)|}$$
As an important caveat, $H(f)$ and $FRF(f)$ are only valid for describing Linear time-invariant (LTI) systems in which there is a linear relationship between the input and output at a given frequency. You can (and should) verify that this linear relationship holds by calculating a Coherence Function. This function will have a value near $1$ for frequencies with strong linear relationships and values near $0$ for frequencies with non-linear relationships. Any values of $H(f)$ and $FRF(f)$ that do not have a high coherence value are essentially nonsensical. The coherence function may be calculated from:
$$C_{xy}(f)=\frac{|S_{xy}(f)|^2}{S_{xx}(f)S_{yy}(f)}$$
where $S_{xy}=H(f)S_{xx}(f)$ is the cross-spectral density, and $S_{xx}(f)$ and $S_{yy}(f)$ are the respective auto-spectral densities of $x(t)$ and $y(t)$, respectively. The auto-spectral densities can be estimated from
$$S_{xx}(f)=|X(f)|^2$$
$$S_{yy}(f)=|Y(f)|^2$$
roughly, or more robustly in the presence of noise using an averaging approach like Welch's method.
