Inductively Scalable, Single-Step Neural Surrogates for Wave-Scattering Inverse Problems

TL;DR: Single-step neural surrogates for electromagnetic simulation—networks that replace an FDTD solve with one forward pass—have been stuck at a few tens of controllable variables because the training data required grows exponentially with problem dimension. Dove and Waller sidestep this by never building a fixed dataset at all: a parallel process runs gradient ascent over refractive-index and source fields to find configurations where the surrogate disagrees most with a full-wave solver, labels them with FDTD, and feeds them through a replay buffer into training. The result is a surrogate trained with up to 41,772 variables that generalizes, without retraining, to domains with 3 million variables (73.8× more), and produces FDTD-validated photonic inverse designs up to 98 wavelengths wide with 1.29–26.5× optimization speedups. The interesting part isn't the U-Net—it's the demonstration that active, adversarial data generation can break a data-scaling wall that random sampling can't, in a setting where you can query the ground truth anywhere.

The problem: a data wall, not a model wall

Photonic inverse design works like this: you want a device—a lens, a beam splitter, a mode converter—defined by a spatial map of refractive indices. You pick a figure of merit (intensity at a focal spot, power coupled into a waveguide mode), simulate light propagating through the current design, backpropagate through the simulator to get gradients on the index map, update, and repeat for hundreds or thousands of iterations. The bottleneck is the simulator. Full-wave solvers like FDTD (finite-difference time-domain—march Maxwell's equations forward on a grid) are the gold standard, but each solve is expensive, and you need thousands to millions of them.

The obvious dream is a neural surrogate: a network that maps (refractive-index field, source field) → electric field in one pass. Fast, differentiable, done. Iterative neural solvers—domain decomposition, learned preconditioners—have scaled up spatially but require tens to hundreds of sequential network calls per simulation, which mostly defeats the purpose inside an already-iterative design loop. Single-step "direct" surrogates are what you actually want, and they've been stuck.

Why stuck? Data. A prior review found the number of ground-truth training examples needed for robust accuracy scales exponentially with the number of simulation variables. The paper does the arithmetic: extrapolating from existing studies, a 1000-variable surrogate would need on the order of 10^9 labeled FDTD simulations—roughly 3.17 years of serial compute and ~393 TB of storage. Meanwhile, real photonic design problems routinely have millions of variables. That's not a gap you close with a bigger model.

The word "robust" is doing heavy lifting here, and it's worth understanding why. In an inverse-design loop, the optimizer traverses a trajectory through configuration space. If the surrogate is accurate on 99.9% of configurations but the optimizer wanders into the 0.1% where it hallucinates, the gradients go bad and the design fails—or worse, the optimizer actively exploits the surrogate's errors, converging to designs that look great through the surrogate and terrible under real physics. Average-case accuracy on a random test set is nearly worthless; you need something closer to worst-case coverage. And random sampling of dense index/source grids is spectacularly bad at covering the worst cases, because the hard cases are structured: ring resonators, high-contrast gratings, waveguides—configurations with measure zero under i.i.d. per-pixel sampling but exactly the configurations an optimizer will steer toward.

The idea: let the model choose its own curriculum

Here's the reframe. In most ML settings, data is a fixed resource you sample from. In a simulation surrogate setting, you have an oracle—FDTD—that can label any input you construct. So don't sample the input space. Search it. Treat the training inputs themselves as optimization variables, and continuously hill-climb toward the configurations where your current surrogate is most wrong.

Concretely, two processes run in parallel on separate GPUs:

The generator holds the current surrogate weights \theta fixed and solves

where s is the complex source field, n the refractive-index field, \mathcal{C} enforces physical validity (index clipped to [1,2], sources zeroed inside the absorbing boundary, source projected to unit energy), and \mathcal{L} is the mismatch between the surrogate's predicted field and the FDTD ground truth. Each generator step is one projected Adam ascent step on a persistent population of candidates—so the search doesn't restart from scratch, it keeps climbing from the last batch of hard examples, with a small reinitialization probability (10^{-3} per candidate) to maintain exploration. Every discovered hard example gets labeled by FDTD and appended to the dataset.

The trainer samples minibatches from that dataset and does ordinary supervised learning on the surrogate, periodically syncing updated weights back to the generator.

If this smells like adversarial training crossed with active learning crossed with self-play, that's exactly the lineage the authors cite. But the framing differs in an important way from adversarial robustness in vision: they aren't hardening against imperceptible perturbations around a fixed dataset. They're using the adversary as a data-efficient exploration policy over the full physically-valid input space, because the oracle can label anything the adversary finds. As the surrogate improves, the generator's discoveries shift automatically—early hard examples are simple configurations with gross errors; late ones (after 200k trainer steps) are structurally complex, high-contrast resonant configurations that produce ever-smaller mismatch. It's an emergent curriculum with no hand design.

Two stabilizers turn out to be non-negotiable, and the ablations in Figure 2 of the paper show both:

Resonance normalization. Near-resonant configurations produce fields orders of magnitude more intense than typical ones. Under raw MSE, a handful of resonant examples dominates both the training gradient and the hard-example search—the generator would just farm resonances forever. So the loss normalizes each example's error by the RMS amplitude r(E) of its ground-truth field, measuring structural error rather than error inflated by field energy. A unit-energy constraint on sources similarly stops the generator from "cheating" by scaling amplitude.

The replay buffer. If freshly generated hard examples are used for training immediately, the system fails to converge—the paper is explicit about this. Generator and trainer form an unstable feedback loop: the generator shifts the input distribution to maximize error faster than the trainer can learn from strongly correlated consecutive batches. The fix is the same one that stabilized deep RL a decade ago: a FIFO replay buffer (100k examples here, with 8.28 million appended over the run) that mixes recent failures with older ones, decorrelating updates and slowing distribution drift. It's a nice reminder that any system with a learner chasing a moving, self-induced data distribution—GANs, DQN, self-play, and now adversarial surrogate training—hits the same instability and admits the same medicine.

The second trick: why it scales to grids 64× larger

Training happened on 64×64 and 128×128 grids (D = 3(N-2t)^2 controllable variables counting one index and two source components per non-boundary pixel: 8,748 and 41,772 respectively). Evaluation goes up to 1024×1024—D = 3{,}084{,}588, a 73.8× jump—with no retraining. Why does that work, when size generalization is usually where convolutional PDE surrogates quietly die?

The answer is a deliberate alignment of three localities. First, the ground truth isn't a steady-state Helmholtz solution—it's FDTD run for a fixed 300 timesteps. Since information in Maxwell's equations propagates at finite speed, each output pixel depends only on inputs within a finite causal cone. The target function is genuinely spatially local. Second, away from boundaries the physics is translation-equivariant. Third, the surrogate is a fully convolutional U-Net (253M parameters, no dense layers tied to grid size)—an architecture whose inductive biases are exactly locality and translation equivariance. Training on two grid sizes simultaneously discourages any features tied to a specific input dimension. The learned operator is a local, multiscale scattering stencil that tiles to any domain size, the same way a convolution kernel does.

The paper reports that RMS-normalized MSE holds essentially constant across scales up to 1024×1024 (Fig. 5 of the paper), with the ceiling set by GPU memory, not accuracy. But note the fine print, which the authors state plainly: the same 300 FDTD cycles are used at every scale, so long-distance interactions beyond the causal horizon are simply absent from both the teacher and the surrogate at large scales. For GRIN lenses—where the dominant interactions are fairly local—this doesn't bite. For a large high-Q resonator or a long feedback path, it would. The "steady-state field" language in the abstract deserves this asterisk: it's steady-state-after-300-steps, which is also what makes the whole inductive-scaling story possible. The trick and the limitation are the same object.

Does it actually work for design?

Three FDTD-validated inverse-design tasks, each optimized twice—once through the surrogate, once through the authors' own differentiable GPU FDTD—with all final designs re-simulated in FDTD:

• 48.5λ single-focus GRIN lens (512² grid): validated focal intensity 1669 (surrogate-designed) vs. 1839 (FDTD-designed); focal FWHM 0.434λ vs 0.423λ. Optimization: 92.2 s vs 474 s, 5.14× faster.
• 98λ periodic-focus GRIN lens array (1024² grid, i.e., deep in inductive-scaling territory): mean target intensity 1221 vs 1147—the surrogate design beats the FDTD design—with more uniform spots (CoV 0.222 vs 0.246). But only 1.29× faster (372 s vs 479 s).
• 8λ×8λ waveguide 50:50 beam splitter (128² grid): 51.2/48.8% split vs 49.9/50.1%, but higher total guided power (10.55 vs 10.24) and lower leakage. 26.5× faster (8.97 s vs 238 s).

Two things stand out. First, the surrogate's gradients are good enough that a design optimizer running entirely inside the neural model lands on structures that survive full-wave validation—this is the robustness claim cashing out, and it's the right test. An optimizer is a far more hostile probe of a surrogate than any static test set. Second, the wide speedup spread is informative, not noise: the surrogate's cost grows with grid area just like FDTD's, and at 1024² the 253M-parameter U-Net is itself expensive, so the advantage compresses to 1.29×. The big wins are at small-to-medium grids. The abstract's "orders of magnitude faster" framing describes the field's aspiration, not this paper's measured result.

Intriguingly, the surrogate-optimized designs sometimes outperform FDTD-optimized ones (the 98λ array, the splitter's total power). The authors' hypothesis—the network smooths a spiky loss landscape, acting as an implicit regularizer that keeps the optimizer out of bad local optima—is plausible and reportedly stable across hyperparameters, but it's a conjecture, not a demonstrated mechanism.

What to make of it, and what to be skeptical about

The genuinely transferable idea: when your training data comes from a queryable simulator, the data distribution is a control variable, and adversarial search over it can beat random sampling by enough to move a scaling wall. Tens of variables → 41,772 trained → 3M deployed is not an incremental bump. And the recipe—adversarial generator + oracle labeling + replay buffer + scale-invariant loss normalization + fully convolutional architecture over a local operator—is domain-agnostic. The authors flag heat and fluids; anything with a differentiable teacher and a local finite-time propagator is a candidate.

Honest caveats:

• Scope is narrow physics: 2D TE, single wavelength, index range fixed to [1,2], one grid resolution, one boundary convention. No dispersion, no 3D vectorial fields, no fabrication constraints. The authors say plainly it won't transfer outside these settings without retraining.
• Training cost is real: 8.28 million FDTD-labeled examples and 200k training steps on two 96 GB GPUs. This amortizes only if the surrogate is reused across many design problems—which the inductive scaling makes plausible, but the paper doesn't do the full amortization accounting.
• No worst-case guarantee: adversarial coverage is empirical. The generator is itself a local hill-climber; failure modes it can't reach by gradient ascent from its candidate population remain uncovered. For a tool meant to be trusted inside optimizers, formal or statistical error bounds (which the authors name as future work) matter.
• Baseline choice: speedups are measured against the authors' own GPU differentiable FDTD, which is a fair and reasonably strong baseline, but there's no comparison against the iterative neural surrogates (domain decomposition, learned preconditioners) that constitute the other branch of this literature.
• The 300-timestep horizon means both teacher and student ignore long-range resonant physics at large scales. Fine for GRIN optics; disqualifying for high-Q cavity design.

If the approach extends to 3D and broadband—a big if, since 3D multiplies both FDTD cost and surrogate memory—this is a credible path to the "foundation model for wave simulation" the authors gesture at, and the claim that distributed training could reach domains 100,000s of wavelengths across is at least architecturally coherent given the convolutional tiling argument. Even if it doesn't, the negative-result-turned-recipe about tightly-coupled adversarial data generation (it diverges; buffer it) is worth remembering.

Where to spend your reading time: the "Efficient training data optimization by dynamic hard-example generation" section, especially the replay-buffer instability discussion, plus the "Inductive scaling" section's locality argument. Figure 4 (hard examples evolving from crude to intricate over training) is the best single visual for what the adversarial curriculum is actually doing.