Let $\mathcal{P}$ be the space of probability distributions on $\mathbf{V}$, and $f: \mathcal{P} \mapsto \mathbb{R}$ be a functional. Our working example is to estimate the following back-door adjustment:
$$ f(P) = \mathbb{E}{P}[\mathbb{E}{P}[Y \mid x,Z]], $$
where $Y \in \mathtt{R}$ is an outcome, $X$ is a binary treatment, and $Z \in \mathbb{R}^d$ is a covariate. For the analysis of the working example, we will
$$ \begin{align*} \mu(XZ) &\triangleq \mathbb{E}[Y \mid X,Z], \\ \pi(XZ) &\triangleq \mathbb{I}_{x}(X)/P(X \mid Z). \end{align*} $$
We first assume the following smoothness.
<aside> 💡
Definition (Unifrom Hadamard Differentiability)
We say $f$ admits a uniform Hadamard Derivative over $\mathcal{Q} \subseteq \mathcal{P}$ if the following limit exists for all $Q \in \mathcal{Q}$:
$$ \begin{align} \dot{f}{Q}(h) \triangleq \lim{t \rightarrow 0} \frac{f(Q (1+ t h)) - f(Q)}{t}, \quad \forall h \text{ such that } Q(1+t h) \in \mathcal{P}. \end{align} $$
</aside>
Using this, we can approximate $f(Q_t)$ for $Q_t \triangleq Q(1+th)$ with $f(Q)$ through a functional analogue of Taylor expansion, called von Mises expansion.
<aside> 💡
Definition (von Mises Expansion)
Suppose $f$ admits a Gateaux differentiability at $Q$ in a direction $h$. Then,
$$ \begin{align} f(Q_t) = f(Q) + t\dot{f}_{Q}(h) + R_2(h), \end{align} $$
where $R_2$ is the second-order remainder.
</aside>
Now, we will rewrite $\dot{f}_{Q}(h)$ using the Riesz representation learning theorem:
<aside> 💡
Theorem (Riesz Representation Theorem)
Let $\mathbb{H}$ denote Hilbert space with the inner product $\langle,\rangle$. For each bounded linear functional $f$ on $\mathbb{H}$, there exists a unique element $\varphi \in \mathbb{H}$ such that
$$ \begin{align} f(x) = \langle \varphi, x\rangle , \quad \forall x \in \mathbb{H}, \end{align} $$
where $\|f\| = \|\varphi\|$.
</aside>
Since $Q(1+th)$ is a valid distribution, $h$ is defined in the space $\mathcal{H}Q$, which is a collection of functions satisfying three conditions: $\mathbb{E}{Q}[h] = 0$, $Q(1+th)$ is a valid distribution, and $\mathbb{E}_{Q}[h^2] < \infty$. This space $\mathcal{H}_Q$ is a subset of $L^2_0(Q)$, the collection of mean-zero functions over $Q$.
Since $\dot{f}{Q}$ is a bounded linear functional over $\mathcal{H}{Q}$, applying the Riesz Representation Theorem shows there exists a unique function $\varphi_{Q} \in \mathcal{H}_{Q}$ such that
$$ \begin{align} \dot{f}{Q}(h) = \langle \varphi{Q}, h \rangle_{L^2(Q)} = \mathbb{E}{Q}[\varphi{Q}(\mathbf{V}) h(\mathbf{V})] \quad \forall h \in \mathcal{H}_{Q}. \end{align} $$
This unique function $\varphi_{Q}$ is called the influence funciton:
<aside> 💡
Definition (Influence Function, IF)
If $f$ admits Gateaux differentiability at $Q$ along a direction $h$, then there exists an element $\varphi$ such that
$$ \dot{f}{Q}(h) = \mathbb{E}{Q}[\varphi_{Q}(\mathbf{V})h(\mathbf{V})]. $$
Such element $\varphi_{Q}$ is the influence function of $f$ at $Q$.
</aside>
Now, consider $h = (P-Q)/Q$. This is a valid choice since $\mathbb{E}_{Q}[(P-Q)/Q] = 0$ and $Q(1+t(P-Q)/Q) = Q + t(P-Q)$ is a valid distribution. Let's set $t=1$. From Equation (2) and (3), we have a following specialization of the von Mises expansion:
$$ \begin{align} f(P) = f(Q) + \mathbb{E}{P}[\varphi{Q}(\mathbf{V})] + R_2. \end{align} $$