no code implementations • 27 May 2024 • Hang Zhou, Yuezhou Ma, Haixu Wu, Haowen Wang, Mingsheng Long
Inspired by the mathematical structure of PDEs, we define a complete set of PDE components and correspondingly embed them as domain-wise (e. g. equation symbols) and point-wise (e. g. boundaries) conditions for Transformer PDE solvers.
no code implementations • 23 May 2024 • Haixu Wu, Huakun Luo, Yuezhou Ma, Jianmin Wang, Mingsheng Long
To mitigate this inherent deficiency of the default scatter-point optimization, this paper proposes and theoretically studies a new training paradigm as region optimization.
1 code implementation • ICLR 2024 • Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y. Zhang, Jun Zhou
Going beyond the mainstream paradigms of plain decomposition and multiperiodicity analysis, we analyze temporal variations in a novel view of multiscale-mixing, which is based on an intuitive but important observation that time series present distinct patterns in different sampling scales.
no code implementations • 29 Feb 2024 • Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Yong liu, Yunzhong Qiu, Haoran Zhang, Jianmin Wang, Mingsheng Long
Experimentally, TimeXer significantly improves time series forecasting with exogenous variables and achieves consistent state-of-the-art performance in twelve real-world forecasting benchmarks.
no code implementations • 4 Feb 2024 • Jiaxiang Dong, Haixu Wu, Yuxuan Wang, Yunzhong Qiu, Li Zhang, Jianmin Wang, Mingsheng Long
To emphasize temporal correlation modeling, this paper proposes TimeSiam as a simple but effective self-supervised pre-training framework for Time series based on Siamese networks.
no code implementations • 4 Feb 2024 • Qilong Ma, Haixu Wu, Lanxiang Xing, Shangchen Miao, Mingsheng Long
Accurately predicting the future fluid is vital to extensive areas such as meteorology, oceanology, and aerodynamics.
1 code implementation • 4 Feb 2024 • Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, Mingsheng Long
Transformers have empowered many milestones across various fields and have recently been applied to solve partial differential equations (PDEs).
1 code implementation • 16 Oct 2023 • Lanxiang Xing, Haixu Wu, Yuezhou Ma, Jianmin Wang, Mingsheng Long
Compared with previous velocity estimating methods, HelmFluid is faithfully derived from Helmholtz theorem and ravels out complex fluid dynamics with physically interpretable evidence.
7 code implementations • 10 Oct 2023 • Yong liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, Mingsheng Long
These forecasters leverage Transformers to model the global dependencies over temporal tokens of time series, with each token formed by multiple variates of the same timestamp.
1 code implementation • NeurIPS 2023 • Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, Mingsheng Long
By relating masked modeling to manifold learning, SimMTM proposes to recover masked time points by the weighted aggregation of multiple neighbors outside the manifold, which eases the reconstruction task by assembling ruined but complementary temporal variations from multiple masked series.
1 code implementation • 30 Jan 2023 • Haixu Wu, Tengge Hu, Huakun Luo, Jianmin Wang, Mingsheng Long
A burgeoning paradigm is learning neural operators to approximate the input-output mappings of PDEs.
3 code implementations • 5 Oct 2022 • Haixu Wu, Tengge Hu, Yong liu, Hang Zhou, Jianmin Wang, Mingsheng Long
TimesBlock can discover the multi-periodicity adaptively and extract the complex temporal variations from transformed 2D tensors by a parameter-efficient inception block.
2 code implementations • 28 May 2022 • Yong liu, Haixu Wu, Jianmin Wang, Mingsheng Long
However, their performance can degenerate terribly on non-stationary real-world data in which the joint distribution changes over time.
3 code implementations • 13 Feb 2022 • Jialong Wu, Haixu Wu, Zihan Qiu, Jianmin Wang, Mingsheng Long
Policy constraint methods to offline reinforcement learning (RL) typically utilize parameterization or regularization that constrains the policy to perform actions within the support set of the behavior policy.
1 code implementation • 13 Feb 2022 • Haixu Wu, Jialong Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long
By respectively conserving the incoming flow of sinks for source competition and the outgoing flow of sources for sink allocation, Flow-Attention inherently generates informative attentions without using specific inductive biases.
Ranked #4 on D4RL on D4RL
1 code implementation • 8 Oct 2021 • Zhiyu Yao, Yunbo Wang, Haixu Wu, Jianmin Wang, Mingsheng Long
To this end, we propose ModeRNN, which introduces a novel method to learn structured hidden representations between recurrent states.
3 code implementations • ICLR 2022 • Jiehui Xu, Haixu Wu, Jianmin Wang, Mingsheng Long
Unsupervised detection of anomaly points in time series is a challenging problem, which requires the model to derive a distinguishable criterion.
2 code implementations • NeurIPS 2021 • Haixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long
Going beyond Transformers, we design Autoformer as a novel decomposition architecture with an Auto-Correlation mechanism.
3 code implementations • 17 Mar 2021 • Yunbo Wang, Haixu Wu, Jianjin Zhang, Zhifeng Gao, Jianmin Wang, Philip S. Yu, Mingsheng Long
This paper models these structures by presenting PredRNN, a new recurrent network, in which a pair of memory cells are explicitly decoupled, operate in nearly independent transition manners, and finally form unified representations of the complex environment.
Ranked #1 on Video Prediction on KTH (Cond metric)
1 code implementation • CVPR 2021 • Haixu Wu, Zhiyu Yao, Jianmin Wang, Mingsheng Long
With high flexibility, this framework can adapt to a series of models for deterministic spatiotemporal prediction.