Skip to article frontmatterSkip to article content
ϕℓ=σ(ωℓ(wx+b))\phi_\ell = \sigma\left( \omega_\ell\left(\mathbf{wx} + \mathbf{b} \right) \right)
(1)

where σ is an arbitrary activation function.

h(x;θ(x))\boldsymbol{h}(\mathbf{x};\boldsymbol{\theta}(\mathbf{x}))
(2)

For example we have the hypernetwork:

ϕℓ(x;θ)=wθx+bθ[wθ,bθ]=h(z;θ)\begin{aligned} \phi_\ell(\mathbf{x};\theta) &= \mathbf{w}_\theta\mathbf{x} + \mathbf{b}_\theta \\ [\mathbf{w}_\theta,\mathbf{b}_\theta] &= \boldsymbol{h}(\mathbf{z};\boldsymbol{\theta}) \end{aligned}
(3)

We have the additive transformation

ϕℓ(x;θ)=wℓx+bℓ+sθsθ=h(z;θ)\begin{aligned} \phi_\ell(\mathbf{x};\theta) &= \mathbf{w}_\ell\mathbf{x} + \mathbf{b}_\ell + \mathbf{s}_\theta \\ \mathbf{s}_\theta &= \boldsymbol{h}(\mathbf{z};\boldsymbol{\theta}) \end{aligned}
(4)

We have the affine transformation

ϕℓ(x;θ)=(wℓx+bℓ)sθ+aθ[sθ,aθ]=h(z;θ)\begin{aligned} \phi_\ell(\mathbf{x};\theta) &= \left(\mathbf{w}_\ell\mathbf{x} + \mathbf{b}_\ell\right)\mathbf{s}_\theta + \mathbf{a}_\theta \\ [\mathbf{s}_\theta, \mathbf{a}_\theta] &= \boldsymbol{h}(\mathbf{z};\boldsymbol{\theta}) \end{aligned}
(5)

Example: 2D Sea Surface Height

In this example, we will train the same SIREN model from before but we will add a modulation extension.

Neural Fields
QG PDE
Neural Fields
Predictive Uncertainty