CVPR 2026Highlight
MSPT: Efficient Large-Scale Physical Modeling via Parallelized Multi-Scale Attention
University of Amsterdam
Abstract
Industrial-scale neural solvers must capture fine spatial interactions and long-range dependencies across large, irregular geometries. The Multi-Scale Patch Transformer (MSPT) combines local point attention with pooled global representations through Parallelized Multi-Scale Attention (PMSA).
Ball trees organize the domain into spatially coherent patches. Local tokens and patch-level supernodes communicate in one attention pattern, allowing MSPT to handle millions of points on a single GPU. Evaluations cover standard PDE tasks and large aerodynamic datasets, measuring predictive accuracy alongside memory and runtime.
Key contributions
Parallelized multi-scale attention
Local point interactions and global patch communication share a single attention pattern.
Irregular geometry at scale
Ball-tree patches and pooled supernodes organize large domains without requiring a regular grid.
Physics benchmark evaluation
PDE and aerodynamic experiments assess accuracy, memory use, and runtime as point counts grow.
Parallelized Multi-Scale Attention
Given point features, MSPT partitions the domain into K patches of size L. Each patch is pooled into Q supernodes. Attention is then computed on augmented tokens that include both local patch tokens and global pooled tokens, enabling simultaneous local detail modeling and long-range communication.
- Partition N spatial elements into K coherent patches using ball trees.
- Pool each patch into Q supernodes to expose long-range structure without paying full global-attention cost.
- Run attention over local tokens plus pooled global tokens, yielding complexity O(NL + N^2Q/L).

Complexity
O(NL + N^2Q/L)
With practical settings where KQ << N, the architecture preserves local fidelity while keeping global context affordable.
Benchmark results
PDE Standard Bench
Relative L2 error where lower is better
| Model | Elasticity | Plasticity | Airfoil | Pipe | Navier-Stokes | Darcy | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FNO | / | / | / | / | 15.56 | 1.08 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
... show 14 more baselines
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MSPT (Ours) | 0.48 | 0.10 | 0.51 | 0.31 | 6.32 | 0.63 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Relative Promotion | -41% | 17% | 4% | 6% | 30% | -10% | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ShapeNet-Car
Relative L2 error where lower is better
| Model | Volume | Surf | CD | rhoD | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Simple MLP | 5.12 | 13.04 | 3.07 | 94.96 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
... show 11 more baselines
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| MSPT (Ours) | 1.89 | 7.41 | 0.98 | 99.41 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Relative Promotion | +8.7% | +0.5% | +4.9% | +0.06% | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
| AB-UPT | 1.16 | 4.81 | N/A | N/A | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
| AB-UPT (repr.) | 2.51 | 7.67 | 2.20 | 97.48 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
AhmedML
Relative L2 error where lower is better
| Model | Volume | Surf | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PointNet | 5.44 | 8.02 | |||||||||||||||||||||
... show 7 more baselines
| |||||||||||||||||||||||
| MSPT (Ours) | 2.04 | 3.22 | |||||||||||||||||||||
| Relative Promotion | +0.49% | +6.67% | |||||||||||||||||||||
| AB-UPT | 1.90 | 3.01 | |||||||||||||||||||||
| AB-UPT (repr.) | 2.39 | 4.33 | |||||||||||||||||||||
Ablation and efficiency

Q) ablation on ShapeNet-Car.
Quick start
git clone --recurse-submodules https://github.com/pedrocurvo/mspt.git
cd mspt/Neural-Solver-Library-MSPT
pip install -r requirements.txt
bash ./scripts/StandardBench/plasticity/MSPT.shBibTeX
@inproceedings{curvo2026mspt,
title={MSPT: Efficient Large-Scale Physical Modeling via Parallelized Multi-Scale Attention},
author={Pedro M. P. Curvo and Jan-Willem van de Meent and Maksim Zhdanov},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2026},
url={https://arxiv.org/abs/2512.01738}
}