1 code implementation • ACL 2022 • Jue Wang, Ke Chen, Gang Chen, Lidan Shou, Julian McAuley
In this paper, we propose SkipBERT to accelerate BERT inference by skipping the computation of shallow layers.
no code implementations • 7 Jun 2024 • Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, James Zou
With the growing number of LLMs, how to harness the collective expertise of multiple LLMs is an exciting open direction.
no code implementations • 29 May 2024 • Zhangyang Gao, Jue Wang, Cheng Tan, Lirong Wu, Yufei Huang, Siyuan Li, Zhirui Ye, Stan Z. Li
We do such unification in two levels: 1) Data-Level: We propose a unified block graph data form for all molecules, including the local frame building and geometric feature initialization.
no code implementations • 26 Mar 2024 • Jue Wang, Yuxiang Lin, Qi Zhao, Dong Luo, Shuaibao Chen, Wei Chen, Xiaojiang Peng
The widespread use of various chemical gases in industrial processes necessitates effective measures to prevent their leakage during transportation and storage, given their high toxicity.
no code implementations • 4 Feb 2024 • Zhangyang Gao, Cheng Tan, Jue Wang, Yufei Huang, Lirong Wu, Stan Z. Li
Is there a foreign language describing protein sequences and structures simultaneously?
no code implementations • 21 Jan 2024 • Yinhe Liu, Sunan Shi, Zhuo Zheng, Jue Wang, Shiqi Tian, Yanfei Zhong
Semantic Change Detection (SCD) is recognized as both a crucial and challenging task in the field of image analysis.
no code implementations • 21 Nov 2023 • Yinuo Ren, Feng Li, Yanfei Kang, Jue Wang
Forecast combination integrates information from various sources by consolidating multiple forecast results from the target time series.
1 code implementation • 26 Oct 2023 • Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Re, Beidi Chen
We show that contextual sparsity exists, that it can be accurately predicted, and that we can exploit it to speed up LLM inference in wall-clock time without compromising LLM's quality or in-context learning ability.
2 code implementations • 23 Oct 2023 • Maomao Li, Ge Yuan, Cairong Wang, Zhian Liu, Yong Zhang, Yongwei Nie, Jue Wang, Dong Xu
Based on this disentanglement, face swapping can be simplified as style and mask swapping.
1 code implementation • 19 Oct 2023 • Yuanduo Hong, Jue Wang, Weichao Sun, Huihui Pan
Building upon the original motivations of plain ViTs, which are simplicity and generality, we explore high-performance `minimalist' systems to this end.
Ranked #1 on Semantic Segmentation on PASCAL Context
no code implementations • 9 Oct 2023 • Cheng Tan, Jue Wang, Zhangyang Gao, Siyuan Li, Lirong Wu, Jun Xia, Stan Z. Li
In this paper, we re-examine the two dominant temporal modeling approaches within the realm of spatio-temporal predictive learning, offering a unified perspective.
1 code implementation • 15 Sep 2023 • Jun Zhang, Jue Wang, Huan Li, Lidan Shou, Ke Chen, Gang Chen, Sharad Mehrotra
This approach is characterized by a two-stage process: drafting and verification.
no code implementations • ICCV 2023 • David Fan, Jue Wang, Shuai Liao, Yi Zhu, Vimal Bhat, Hector Santos-Villalobos, Rohith MV, Xinyu Li
This suggests that the random masking strategy that is inherited from the image MAE is less effective for video MAE.
no code implementations • 17 May 2023 • Zhaozhuo Xu, Zirui Liu, Beidi Chen, Yuxin Tang, Jue Wang, Kaixiong Zhou, Xia Hu, Anshumali Shrivastava
Thus, optimizing this accuracy-efficiency trade-off is crucial for the LLM deployment on commodity hardware.
no code implementations • CVPR 2023 • Weiyu Li, Xuelin Chen, Jue Wang, Baoquan Chen
We target a 3D generative model for general natural scenes that are typically unique and intricate.
no code implementations • CVPR 2023 • Fang Zhao, Zekun Li, Shaoli Huang, Junwu Weng, Tianfei Zhou, Guo-Sen Xie, Jue Wang, Ying Shan
Once the anchor transformations are found, per-vertex nonlinear displacements of the garment template can be regressed in a canonical space, which reduces the complexity of deformation space learning.
no code implementations • 1 Apr 2023 • Xiaojun Jia, Yong Zhang, Xingxing Wei, Baoyuan Wu, Ke Ma, Jue Wang, Xiaochun Cao
This initialization is generated by using high-quality adversarial perturbations from the historical training process.
no code implementations • CVPR 2023 • Jue Wang, Wentao Zhu, Pichao Wang, Xiang Yu, Linda Liu, Mohamed Omar, Raffay Hamid
To address this limitation, we present a novel Selective S4 (i. e., S5) model that employs a lightweight mask generator to adaptively select informative image tokens resulting in more efficient and accurate modeling of long-term spatiotemporal dependencies in videos.
Ranked #2 on Video Classification on Breakfast
1 code implementation • 15 Mar 2023 • Weihuang Liu, Xiaodong Cun, Chi-Man Pun, Menghan Xia, Yong Zhang, Jue Wang
Thanks to the proposed structure, we only encode the high-resolution image in a relatively low resolution for larger reception field capturing.
1 code implementation • CVPR 2023 • Jiaxu Zhang, Junwu Weng, Di Kang, Fang Zhao, Shaoli Huang, Xuefei Zhe, Linchao Bao, Ying Shan, Jue Wang, Zhigang Tu
Driven by our explored distance-based losses that explicitly model the motion semantics and geometry, these two modules can learn residual motion modifications on the source motion to generate plausible retargeted motion in a single inference without post-processing.
1 code implementation • CVPR 2023 • Zhengdi Yu, Shaoli Huang, Chen Fang, Toby P. Breckon, Jue Wang
Our method significantly outperforms the best interacting-hand approaches on the InterHand2. 6M dataset while yielding comparable performance with the state-of-the-art single-hand methods on the FreiHand dataset.
Ranked #2 on 3D Interacting Hand Pose Estimation on InterHand2.6M
1 code implementation • CVPR 2023 • Jinbo Xing, Menghan Xia, Yuechen Zhang, Xiaodong Cun, Jue Wang, Tien-Tsin Wong
In this paper, we propose to cast speech-driven facial animation as a code query task in a finite proxy space of the learned codebook, which effectively promotes the vividness of the generated motions by reducing the cross-modal mapping uncertainty.
Ranked #4 on 3D Face Animation on BEAT2
no code implementations • 3 Jan 2023 • Boyu Zhang, Hongliang Yuan, Mingyan Zhu, Ligang Liu, Jue Wang
Generating high-quality, realistic rendering images for real-time applications generally requires tracing a few samples-per-pixel (spp) and using deep learning-based approaches to denoise the resulting low-spp images.
no code implementations • 27 Dec 2022 • Zixiao Wang, Junwu Weng, Chun Yuan, Jue Wang
Thanks to Noise Contrastive Learning, the average classification accuracy improvement on Mini-Kinetics and Sth-Sth-V1 is over 1. 6\%.
1 code implementation • SIGGRAPH 2022 • Menghan Xia, WenBo Hu, Tien-Tsin Wong, Jue Wang
Our key insight is that several carefully located anchors could approximately represent the color distribution of an image, and conditioned on the anchor colors, we can predict the image color in a deterministic manner by utilizing internal correlation.
1 code implementation • 27 Nov 2022 • Kun Cheng, Xiaodong Cun, Yong Zhang, Menghan Xia, Fei Yin, Mingrui Zhu, Xuan Wang, Jue Wang, Nannan Wang
Our system disentangles this objective into three sequential tasks: (1) face video generation with a canonical expression; (2) audio-driven lip-sync; and (3) face enhancement for improving photo-realism.
no code implementations • CVPR 2023 • Zhian Liu, Maomao Li, Yong Zhang, Cairong Wang, Qi Zhang, Jue Wang, Yongwei Nie
We rethink face swapping from the perspective of fine-grained face editing, \textit{i. e., ``editing for swapping'' (E4S)}, and propose a framework that is based on the explicit disentanglement of the shape and texture of facial components.
1 code implementation • 16 Nov 2022 • Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda
We present Holistic Evaluation of Language Models (HELM) to improve the transparency of language models.
1 code implementation • 14 Oct 2022 • Yiming Zhu, Hongyu Liu, Yibing Song, Ziyang Yuan, Xintong Han, Chun Yuan, Qifeng Chen, Jue Wang
Based on the visual latent space of StyleGAN[21] and text embedding space of CLIP[34], studies focus on how to map these two latent spaces for text-driven attribute manipulations.
3 code implementations • 12 Oct 2022 • Zeyu Qin, Yanbo Fan, Yi Liu, Li Shen, Yong Zhang, Jue Wang, Baoyuan Wu
Furthermore, RAP can be naturally combined with many existing black-box attack techniques, to further boost the transferability.
1 code implementation • 3 Oct 2022 • Jiancong Xiao, Yanbo Fan, Ruoyu Sun, Jue Wang, Zhi-Quan Luo
In adversarial machine learning, deep neural networks can fit the adversarial examples on the training dataset but have poor generalization ability on the test set.
1 code implementation • 2 Oct 2022 • Jiancong Xiao, Liusha Yang, Yanbo Fan, Jue Wang, Zhi-Quan Luo
On synthetic datasets, theoretically, We prove that on-manifold adversarial examples are powerful, yet adversarial training focuses on off-manifold directions and ignores the on-manifold adversarial examples.
1 code implementation • 2 Oct 2022 • Jiancong Xiao, Zeyu Qin, Yanbo Fan, Baoyuan Wu, Jue Wang, Zhi-Quan Luo
Therefore, adversarial training for multiple perturbations (ATMP) is proposed to generalize the adversarial robustness over different perturbation types (in $\ell_1$, $\ell_2$, and $\ell_\infty$ norm-bounded perturbations).
no code implementations • 31 Aug 2022 • Chengleyang Lei, Wei Feng, Jue Wang, Shi Jin, Ning Ge
This letter presents a sensing-communication-computing-control (SC3) integrated satellite unmanned aerial vehicle (UAV) network, where the UAV is equipped with on-board sensors, mobile edge computing (MEC) servers, base stations and satellite communication module.
1 code implementation • 28 Aug 2022 • Mingdeng Cao, Zhihang Zhong, Yanbo Fan, Jiahao Wang, Yong Zhang, Jue Wang, Yujiu Yang, Yinqiang Zheng
We believe the novel realistic synthesis pipeline and the corresponding RAW video dataset can help the community to easily construct customized blur datasets to improve real-world video deblurring performance largely, instead of laboriously collecting real data pairs.
1 code implementation • 14 Aug 2022 • Chengyin Xu, Zenghao Chai, Zhengzhuo Xu, Chun Yuan, Yanbo Fan, Jue Wang
Image retrieval has become an increasingly appealing technique with broad multimedia application prospects, where deep hashing serves as the dominant branch towards low storage and efficient retrieval.
1 code implementation • 21 Jul 2022 • Meng Cao, Tianyu Yang, Junwu Weng, Can Zhang, Jue Wang, Yuexian Zou
To further enhance the temporal reasoning ability of the learned feature, we propose a context projection head and a temporal aware contrastive loss to perceive the contextual relationships.
1 code implementation • 18 Jul 2022 • Xiaojun Jia, Yong Zhang, Xingxing Wei, Baoyuan Wu, Ke Ma, Jue Wang, Xiaochun Cao
Based on the observation, we propose a prior-guided FGSM initialization method to avoid overfitting after investigating several initialization strategies, improving the quality of the AEs during the whole training process.
Adversarial Attack Adversarial Attack on Video Classification
no code implementations • 1 Jul 2022 • Li Ma, Xiaoyu Li, Jing Liao, Xuan Wang, Qi Zhang, Jue Wang, Pedro Sander
Implicit radiance functions emerged as a powerful scene representation for reconstructing and rendering photo-realistic views of a 3D scene.
no code implementations • 6 Jun 2022 • Zhichao Huang, Yanbo Fan, Chen Liu, Weizhong Zhang, Yong Zhang, Mathieu Salzmann, Sabine Süsstrunk, Jue Wang
While adversarial training and its variants have shown to be the most effective algorithms to defend against adversarial attacks, their extremely slow training process makes it hard to scale to large datasets like ImageNet.
1 code implementation • 2 Jun 2022 • Jue Wang, Binhang Yuan, Luka Rimanic, Yongjun He, Tri Dao, Beidi Chen, Christopher Re, Ce Zhang
Communication compression is a crucial technique for modern distributed learning systems to alleviate their communication bottlenecks over slower networks.
1 code implementation • 31 May 2022 • Jingxiang Sun, Xuan Wang, Yichun Shi, Lizhen Wang, Jue Wang, Yebin Liu
Existing 3D-aware facial generation methods face a dilemma in quality versus editability: they either generate editable results in low resolution or high-quality ones with no editing flexibility.
2 code implementations • 26 May 2022 • Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, Ping Luo
To address this challenge, we propose an effective adaptation approach for Transformer, namely AdaptFormer, which can adapt the pre-trained ViTs into many different image and video tasks efficiently.
no code implementations • 24 May 2022 • Yunpeng Bai, Cairong Wang, Chun Yuan, Yanbo Fan, Jue Wang
The content contrastive loss enables the encoder to retain more available details.
1 code implementation • 17 Apr 2022 • Mingdeng Cao, Yanbo Fan, Yong Zhang, Jue Wang, Yujiu Yang
For multi-frame temporal modeling, we adapt Transformer to fuse multiple spatial features efficiently.
no code implementations • CVPR 2022 • Jue Wang, Lorenzo Torresani
Video transformers have recently emerged as an effective alternative to convolutional networks for action classification.
no code implementations • CVPR 2022 • Kai Ye, Siyan Dong, Qingnan Fan, He Wang, Li Yi, Fei Xia, Jue Wang, Baoquan Chen
Previous approaches either choose the frontier as the goal position via a myopic solution that hinders the time efficiency, or maximize the long-term value via reinforcement learning to directly regress the goal position, but does not guarantee the complete map construction.
1 code implementation • CVPR 2023 • Yue Chen, Xuan Wang, Xingyu Chen, Qi Zhang, Xiaoyu Li, Yu Guo, Jue Wang, Fei Wang
Neural volume rendering enables photo-realistic renderings of a human performer in free-view, a critical task in immersive VR/AR applications.
1 code implementation • CVPR 2022 • Can Zhang, Tianyu Yang, Junwu Weng, Meng Cao, Jue Wang, Yuexian Zou
These pre-trained models can be sub-optimal for temporal localization tasks due to the inherent discrepancy between video-level classification and clip-level localization.
1 code implementation • CVPR 2022 • Liang Chen, Yong Zhang, Yibing Song, Lingqiao Liu, Jue Wang
Following this principle, we propose to enrich the "diversity" of forgeries by synthesizing augmented forgeries with a pool of forgery configurations and strengthen the "sensitivity" to the forgeries by enforcing the model to predict the forgery configurations.
4 code implementations • 23 Mar 2022 • Zhan Tong, Yibing Song, Jue Wang, LiMin Wang
Pre-training video transformers on extra large-scale datasets is generally required to achieve premier performance on relatively small datasets.
Ranked #5 on Self-Supervised Action Recognition on HMDB51
1 code implementation • CVPR 2022 • Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang, Xiaochun Cao
In this paper, we propose a novel framework for adversarial training by introducing the concept of "learnable attack strategy", dubbed LAS-AT, which learns to automatically produce attack strategies to improve the model robustness.
1 code implementation • 8 Mar 2022 • Fei Yin, Yong Zhang, Xiaodong Cun, Mingdeng Cao, Yanbo Fan, Xuan Wang, Qingyan Bai, Baoyuan Wu, Jue Wang, Yujiu Yang
Our framework elevates the resolution of the synthesized talking face to 1024*1024 for the first time, even though the training dataset has a lower resolution.
1 code implementation • 16 Feb 2022 • Youwei Liang, Chongjian Ge, Zhan Tong, Yibing Song, Jue Wang, Pengtao Xie
Second, by maintaining the same computational cost, our method empowers ViTs to take more image tokens as input for recognition accuracy improvement, where the image tokens are from higher resolution images.
Ranked #4 on Efficient ViTs on ImageNet-1K (with DeiT-S)
no code implementations • 27 Jan 2022 • Peng Wei, Kun Guo, Ye Li, Jue Wang, Wei Feng, Shi Jin, Ning Ge, Ying-Chang Liang
Mobile edge computing (MEC) is considered a novel paradigm for computation-intensive and delay-sensitive tasks in fifth generation (5G) networks and beyond.
no code implementations • CVPR 2022 • Jingjing Li, Tianyu Yang, Wei Ji, Jue Wang, Li Cheng
Inspired by recent success in unsupervised contrastive representation learning, we propose a novel denoised cross-video contrastive algorithm, aiming to enhance the feature discrimination ability of video snippets for accurate temporal action localization in the weakly-supervised setting.
no code implementations • 21 Dec 2021 • Jue Wang, Shaoli Huang, Xinchao Wang, DaCheng Tao
Conventional 3D human pose estimation relies on first detecting 2D body keypoints and then solving the 2D to 3D correspondence problem. Despite the promising results, this learning paradigm is highly dependent on the quality of the 2D keypoint detector, which is inevitably fragile to occlusions and out-of-image absences. In this paper, we propose a novel Pose Orientation Net (PONet) that is able to robustly estimate 3D pose by learning orientations only, hence bypassing the error-prone keypoint detector in the absence of image evidence.
Ranked #82 on 3D Human Pose Estimation on MPI-INF-3DHP
1 code implementation • 10 Dec 2021 • Chaochen Gao, Xing Wu, Peng Wang, Jue Wang, Liangjun Zang, Zhongyuan Wang, Songlin Hu
To tackle that, we propose an effective knowledge distillation framework for contrastive sentence embeddings, termed DistilCSE.
1 code implementation • NeurIPS 2021 • Chongjian Ge, Youwei Liang, Yibing Song, Jianbo Jiao, Jue Wang, Ping Luo
Motivated by the transformers that explore visual attention effectively in recognition scenarios, we propose a CNN Attention REvitalization (CARE) framework to train attentive CNN encoders guided by transformers in SSL.
1 code implementation • CVPR 2022 • Jingxiang Sun, Xuan Wang, Yong Zhang, Xiaoyu Li, Qi Zhang, Yebin Liu, Jue Wang
2D GANs can generate high fidelity portraits but with low view consistency.
1 code implementation • CVPR 2022 • Xingyu Chen, Qi Zhang, Xiaoyu Li, Yue Chen, Ying Feng, Xuan Wang, Jue Wang
This paper studies the problem of hallucinated NeRF: i. e., recovering a realistic NeRF at a different time of day from a group of tourism images.
1 code implementation • CVPR 2022 • Li Ma, Xiaoyu Li, Jing Liao, Qi Zhang, Xuan Wang, Jue Wang, Pedro V. Sander
We demonstrate that our method can be used on both camera motion blur and defocus blur: the two most common types of blur in real scenes.
no code implementations • 30 Oct 2021 • Jue Wang, Haofan Wang, Xing Wu, Chaochen Gao, Debing Zhang
In this paper, we present TransAug (Translate as Augmentation), which provide the first exploration of utilizing translated sentence pairs as data augmentation for text, and introduce a two-stage paradigm to advances the state-of-the-art sentence embeddings.
no code implementations • 11 Oct 2021 • Xiaojun Jia, Yong Zhang, Baoyuan Wu, Jue Wang, Xiaochun Cao
Adversarial training (AT) has been demonstrated to be effective in improving model robustness by leveraging adversarial examples for training.
1 code implementation • 11 Oct 2021 • Chongjian Ge, Youwei Liang, Yibing Song, Jianbo Jiao, Jue Wang, Ping Luo
Motivated by the transformers that explore visual attention effectively in recognition scenarios, we propose a CNN Attention REvitalization (CARE) framework to train attentive CNN encoders guided by transformers in SSL.
1 code implementation • CVPR 2022 • Shuangrui Ding, Maomao Li, Tianyu Yang, Rui Qian, Haohang Xu, Qingyi Chen, Jue Wang, Hongkai Xiong
To alleviate such bias, we propose \textbf{F}oreground-b\textbf{a}ckground \textbf{Me}rging (FAME) to deliberately compose the moving foreground region of the selected video onto the static background of others.
1 code implementation • ICLR 2022 • Youwei Liang, Chongjian Ge, Zhan Tong, Yibing Song, Jue Wang, Pengtao Xie
Second, by maintaining the same computational cost, our method empowers ViTs to take more image tokens as input for recognition accuracy improvement, where the image tokens are from higher resolution images.
no code implementations • 20 Sep 2021 • Xin Zheng, Yanbo Fan, Baoyuan Wu, Yong Zhang, Jue Wang, Shirui Pan
Face recognition has been greatly facilitated by the development of deep neural networks (DNNs) and has been widely applied to many safety-critical applications.
1 code implementation • CVPR 2022 • Tengfei Wang, Yong Zhang, Yanbo Fan, Jue Wang, Qifeng Chen
With a low bit-rate latent code, previous works have difficulties in preserving high-fidelity details in reconstructed and edited images.
2 code implementations • 13 Sep 2021 • Jingtang Liang, Xiaodong Cun, Chi-Man Pun, Jue Wang
To this end, we propose a novel spatial-separated curve rendering network(S$^2$CRNet) for efficient and high-resolution image harmonization for the first time.
Ranked #12 on Image Harmonization on iHarmony4
no code implementations • 10 Sep 2021 • Jue Wang, Haofan Wang, Jincan Deng, Weijia Wu, Debing Zhang
Extra rich non-paired single-modal text data is used for boosting the generalization of text branch.
no code implementations • 6 Sep 2021 • Xin Tong, Zhaoyang Zhang, Jue Wang, Chongwen Huang, Merouane Debbah
As a potential technology feature for 6G wireless networks, the idea of sensing-communication integration requires the system not only to complete reliable multi-user communication but also to achieve accurate environment sensing.
no code implementations • 16 Aug 2021 • Xinyue Wei, HaoZhi Huang, Yujin Shi, Hongliang Yuan, Li Shen, Jue Wang
We show in this work that Monte Carlo path tracing can be further accelerated by joint super-resolution and denoising (SRD) in post-processing.
no code implementations • 24 Jun 2021 • Anoop Cherian, Jue Wang
One-class learning is the classic problem of fitting a model to the data for which annotations are available only for a single class.
no code implementations • CVPR 2022 • Jue Wang, Gedas Bertasius, Du Tran, Lorenzo Torresani
Our approach, named Long-Short Temporal Contrastive Learning (LSTCL), enables video transformers to learn an effective clip-level representation by predicting temporal context captured from a longer temporal extent.
1 code implementation • AAAI 2021 • Jue Wang, Ke Chen, Lidan Shou, Sai Wu, Gang Chen
By using some particular weakly-labeled data, namely the plain phrases included in sentences, we propose a weaklysupervised slot filling approach.
no code implementations • 13 Apr 2021 • Anoop Cherian, Panagiotis Stanitsas, Jue Wang, Mehrtash Harandi, Vassilios Morellas, Nikolaos Papanikolopoulos
There exist several similarity measures for comparing SPD matrices with documented benefits.
no code implementations • 3 Feb 2021 • Ru Li, Chuan Wang, Jue Wang, Guanghui Liu, Heng-Yu Zhang, Bing Zeng, Shuaicheng Liu
The ground truth images play a leading role in generating reasonable HDR images.
no code implementations • CVPR 2021 • Xudong Lin, Gedas Bertasius, Jue Wang, Shih-Fu Chang, Devi Parikh, Lorenzo Torresani
We present \textsc{Vx2Text}, a framework for text generation from multimodal inputs consisting of video plus text, speech, or audio.
1 code implementation • 27 Jan 2021 • Haipeng Li, Shuaicheng Liu, Jue Wang
In this work, we propose a deep network that compensates the motions caused by the OIS, such that the gyroscopes can be used for image alignment on the OIS cameras.
1 code implementation • 8 Dec 2020 • Qihang Fang, Yingda Yin, Qingnan Fan, Fei Xia, Siyan Dong, Sheng Wang, Jue Wang, Leonidas Guibas, Baoquan Chen
These approaches localize the camera in the discrete pose space and are agnostic to the localization-driven scene property, which restricts the camera pose accuracy in the coarse scale.
2 code implementations • CVPR 2021 • Kunming Luo, Chuan Wang, Shuaicheng Liu, Haoqiang Fan, Jue Wang, Jian Sun
By integrating these two components together, our method achieves the best performance for unsupervised optical flow learning on multiple leading benchmarks, including MPI-SIntel, KITTI 2012 and KITTI 2015.
Ranked #1 on Optical Flow Estimation on Sintel Final unsupervised
1 code implementation • ECCV 2020 • Yuzhi Wang, Haibin Huang, Qin Xu, Jiaming Liu, Yiqun Liu, Jue Wang
Deep learning-based image denoising approaches have been extensively studied in recent years, prevailing in many public benchmark datasets.
2 code implementations • 12 Oct 2020 • Linchao Bao, Xiangkai Lin, Yajing Chen, Haoxian Zhang, Sheng Wang, Xuefei Zhe, Di Kang, HaoZhi Huang, Xinwei Jiang, Jue Wang, Dong Yu, Zhengyou Zhang
We present a fully automatic system that can produce high-fidelity, photo-realistic 3D digital human heads with a consumer RGB-D selfie camera.
2 code implementations • EMNLP 2020 • Jue Wang, Wei Lu
In this work, we propose the novel {\em table-sequence encoders} where two different encoders -- a table encoder and a sequence encoder are designed to help each other in the representation learning process.
Ranked #2 on Zero-shot Relation Triplet Extraction on Wiki-ZSL
Joint Entity and Relation Extraction named-entity-recognition +5
no code implementations • 29 Sep 2020 • Zhuojun Tian, Zhaoyang Zhang, Jue Wang, Xiaoming Chen, Wei Wang, Huaiyu Dai
In this paper, we propose a novel distributed alternating direction method of multipliers (ADMM) algorithm with synergetic communication and computation, called SCCD-ADMM, to reduce the total communication and computation cost of the system.
2 code implementations • ECCV 2020 • Xuewen Yang, Heming Zhang, Di Jin, Yingru Liu, Chi-Hao Wu, Jianchao Tan, Dongliang Xie, Jue Wang, Xin Wang
The goal of this work is to develop a novel learning framework for accurate and expressive fashion captioning.
no code implementations • 5 Jul 2020 • Heming Zhang, Xuewen Yang, Jianchao Tan, Chi-Hao Wu, Jue Wang, C. -C. Jay Kuo
Color compatibility is important for evaluating the compatibility of a fashion outfit, yet it was neglected in previous studies.
2 code implementations • ACL 2020 • Jue Wang, Lidan Shou, Ke Chen, Gang Chen
Its hidden state at layer l represents an l-gram in the input text, which is labeled only if its corresponding text region represents a complete entity mention.
Ranked #1 on Nested Named Entity Recognition on NNE
no code implementations • 30 Jun 2020 • Kunming Luo, Chuan Wang, Nianjin Ye, Shuaicheng Liu, Jue Wang
Occlusion is an inevitable and critical problem in unsupervised optical flow learning.
no code implementations • 1 Jun 2020 • Xiangling Li, Wei Feng, Jue Wang, Yunfei Chen, Ning Ge, Cheng-Xiang Wang
In this paper, we study the integration of UAVs with existing MCNs, and investigate the potential gains of hybrid satellite-UAV-terrestrial networks for maritime coverage.
no code implementations • 17 Jan 2020 • Anoop Cherian, Jue Wang, Chiori Hori, Tim K. Marks
To this end, we propose a Spatio-Temporal and Temporo-Spatial (STaTS) attention model which, conditioned on the language state, hierarchically combines spatial and temporal attention to videos in two different orders: (i) a spatio-temporal (ST) sub-model, which first attends to regions that have temporal evolution, then temporally pools the features from these regions; and (ii) a temporo-spatial (TS) sub-model, which first decides a single frame to attend to, then applies spatial attention within that frame.
no code implementations • 11 Dec 2019 • Nianjin Ye, Chuan Wang, Shuaicheng Liu, Lanpeng Jia, Jue Wang, Yongqing Cui
Deep homography methods, on the other hand, are free from such problem by learning deep features for robust performance.
no code implementations • ICCV 2019 • Jue Wang, Shaoli Huang, Xinchao Wang, Dacheng Tao
We model parts with higher DOFs like the elbows, as dependent components of the corresponding parts with lower DOFs like the torso, of which the 3D locations can be more reliably estimated.
1 code implementation • ECCV 2020 • Jirong Zhang, Chuan Wang, Shuaicheng Liu, Lanpeng Jia, Nianjin Ye, Jue Wang, Ji Zhou, Jian Sun
Homography estimation is a basic image alignment method in many applications.
Ranked #5 on Homography Estimation on S-COCO
no code implementations • ICCV 2019 • Shaofan Cai, Xiaoshuai Zhang, Haoqiang Fan, Haibin Huang, Jiangyu Liu, Jiaming Liu, Jiaying Liu, Jue Wang, Jian Sun
Most previous image matting methods require a roughly-specificed trimap as input, and estimate fractional alpha values for all pixels that are in the unknown region of the trimap.
no code implementations • 5 Sep 2019 • Jue Wang, Anoop Cherian
With the features from the video as a positive bag and the irrelevant features as the negative bag, we cast an objective to learn a (nonlinear) hyperplane that separates the unknown useful features from the rest in a multiple instance learning formulation within a support vector machine setup.
no code implementations • ICCV 2019 • Jue Wang, Anoop Cherian
One-class learning is the classic problem of fitting a model to data for which annotations are available only for a single class.
no code implementations • ICCV 2019 • Yi He, Jiayuan Shi, Chuan Wang, Haibin Huang, Jiaming Liu, Guanbin Li, Risheng Liu, Jue Wang
In this paper we present a new data-driven method for robust skin detection from a single human portrait image.
no code implementations • 20 May 2019 • Jue Wang, Shaoli Huang, Xinchao Wang, DaCheng Tao
We model parts with higher DOFs like the elbows, as dependent components of the corresponding parts with lower DOFs like the torso, of which the 3D locations can be more reliably estimated.
no code implementations • 8 May 2019 • Yifan Ding, Chuan Wang, Haibin Huang, Jiaming Liu, Jue Wang, Liqiang Wang
Compared with image inpainting, performing this task on video presents new challenges such as how to preserving temporal consistency and spatial details, as well as how to handle arbitrary input video size and length fast and efficiently.
1 code implementation • 29 Apr 2019 • Jiaming Liu, Chi-Hao Wu, Yuzhi Wang, Qin Xu, Yuqian Zhou, Haibin Huang, Chuan Wang, Shaofan Cai, Yifan Ding, Haoqiang Fan, Jue Wang
In this paper, we present new data pre-processing and augmentation techniques for DNN-based raw image denoising.
no code implementations • 8 Apr 2019 • Jue Wang, Ke Chen, Lidan Shou, Sai Wu, Sharad Mehrotra
In this paper, we redefine the problem as question-answer extraction, and present SAMIE: Self-Asking Model for Information Ixtraction, a semi-supervised model which dually learns to ask and to answer questions by itself.
2 code implementations • 6 Apr 2019 • Yuqian Zhou, Jianbo Jiao, Haibin Huang, Yang Wang, Jue Wang, Honghui Shi, Thomas Huang
In this paper, we propose a novel approach to boost the performance of a real image denoiser which is trained only with synthetic pixel-independent noise data dominated by AWGN.
Ranked #2 on Denoising on Darmstadt Noise Dataset
2 code implementations • 25 Jan 2019 • Huda Alamri, Vincent Cartillier, Abhishek Das, Jue Wang, Anoop Cherian, Irfan Essa, Dhruv Batra, Tim K. Marks, Chiori Hori, Peter Anderson, Stefan Lee, Devi Parikh
We introduce the task of scene-aware dialog.
no code implementations • CVPR 2019 • Yang Wang, Haibin Huang, Chuan Wang, Tong He, Jue Wang, Minh Hoai
In this paper, we propose GIF2Video, the first learning-based method for enhancing the visual quality of GIFs in the wild.
no code implementations • CVPR 2019 • Tong He, Haibin Huang, Li Yi, Yuqian Zhou, Chi-Hao Wu, Jue Wang, Stefano Soatto
Surface-based geodesic topology provides strong cues for object semantic analysis and geometric modeling.
no code implementations • ECCV 2018 • Jue Wang, Anoop Cherian
As the perturbed features belong to data classes that are likely to be confused with the original features, the discriminative subspace will characterize parts of the feature space that are more representative of the original data, and thus may provide robust video representations.
no code implementations • 23 Aug 2018 • Xian Wu, Rui-Long Li, Fang-Lue Zhang, Jian-Cheng Liu, Jue Wang, Ariel Shamir, Shi-Min Hu
We evaluate our method on public portrait image datasets, and show that it outperforms other state-of-art general image completion methods.
Graphics
no code implementations • ECCV 2018 • Jue Wang, Anoop Cherian
In this paper, we propose to use such perturbations within a novel contrastive learning setup to build negative samples, which are then used to produce improved video representations.
Ranked #42 on Action Recognition on HMDB-51
no code implementations • 22 Jun 2018 • Chuan Wang, Haibin Huang, Xiaoguang Han, Jue Wang
We present a new data-driven video inpainting method for recovering missing regions of video frames.
2 code implementations • 21 Jun 2018 • Chiori Hori, Huda Alamri, Jue Wang, Gordon Wichern, Takaaki Hori, Anoop Cherian, Tim K. Marks, Vincent Cartillier, Raphael Gontijo Lopes, Abhishek Das, Irfan Essa, Dhruv Batra, Devi Parikh
We introduce a new dataset of dialogs about videos of human behaviors.
1 code implementation • CVPR 2018 • Ke Ma, Zhixin Shu, Xue Bai, Jue Wang, Dimitris Samaras
The network is trained on this dataset with various data augmentations to improve its generalization ability.
Ranked #4 on SSIM on DocUNet (using extra training data)
4 code implementations • 1 Jun 2018 • Huda Alamri, Vincent Cartillier, Raphael Gontijo Lopes, Abhishek Das, Jue Wang, Irfan Essa, Dhruv Batra, Devi Parikh, Anoop Cherian, Tim K. Marks, Chiori Hori
Scene-aware dialog systems will be able to have conversations with users about the objects and events around them.
no code implementations • CVPR 2018 • Jue Wang, Anoop Cherian, Fatih Porikli, Stephen Gould
In an attempt to tackle this problem, we propose discriminative pooling, based on the notion that among the deep features generated on all short clips, there is at least one that characterizes the action.
4 code implementations • CVPR 2018 • Xin Tao, Hongyun Gao, Yi Wang, Xiaoyong Shen, Jue Wang, Jiaya Jia
In single image deblurring, the "coarse-to-fine" scheme, i. e. gradually restoring the sharp image on different resolutions in a pyramid, is very successful in both traditional optimization-based methods and recent neural-network-based approaches.
Ranked #2 on Image Deblurring on GoPro (Params (M) metric, using extra training data)
no code implementations • 19 Sep 2017 • Tengda Han, Jue Wang, Anoop Cherian, Stephen Gould
For effective human-robot interaction, it is important that a robotic assistant can forecast the next action a human will consider in a given task.
1 code implementation • CVPR 2017 • Shuochen Su, Mauricio Delbracio, Jue Wang, Guillermo Sapiro, Wolfgang Heidrich, Oliver Wang
We show that the features learned from this dataset extend to deblurring motion blur that arises due to camera shake in a wide range of videos, and compare the quality of results to a number of other baselines.
1 code implementation • ICCV 2017 • Xin Tao, Chao Zhou, Xiaoyong Shen, Jue Wang, Jiaya Jia
In this paper, we study an unconventional but practically meaningful reversibility problem of commonly used image filters.
1 code implementation • ICCV 2017 • Xin Tao, Hongyun Gao, Renjie Liao, Jue Wang, Jiaya Jia
In this paper, we show that proper frame alignment and motion compensation is crucial for achieving high quality results.
Ranked #11 on Video Super-Resolution on Vid4 - 4x upscaling
no code implementations • 6 Apr 2017 • Jue Wang, Anoop Cherian, Fatih Porikli, Stephen Gould
Applying multiple instance learning in an SVM setup, we use the parameters of this separating hyperplane as a descriptor for the video.
2 code implementations • ACCV 2017 • Steve Bako, Soheil Darabi, Eli Shechtman, Jue Wang, Kalyan Sunkavalli, Pradeep Sen
In this work, we automatically detect and remove distracting shadows from photographs of documents and other text-based items.
no code implementations • 12 Jan 2017 • Jue Wang, Anoop Cherian, Fatih Porikli
Training of Convolutional Neural Networks (CNNs) on long video sequences is computationally expensive due to the substantial memory requirements and the massive number of parameters that deep architectures demand.
1 code implementation • 25 Nov 2016 • Shuochen Su, Mauricio Delbracio, Jue Wang, Guillermo Sapiro, Wolfgang Heidrich, Oliver Wang
We show that the features learned from this dataset extend to deblurring motion blur that arises due to camera shake in a wide range of videos, and compare the quality of results to a number of other baselines.
no code implementations • CVPR 2016 • Yao Lu, Xue Bai, Linda Shapiro, Jue Wang
Interactive video segmentation systems aim at producing sub-pixel-level object boundaries for visual effect applications.
no code implementations • CVPR 2016 • Renjiao Yi, Jue Wang, Ping Tan
We present a fully automatic approach to detect and segment fence-like occluders from a video clip.
no code implementations • 21 Mar 2016 • Liqian Ma, Jue Wang, Eli Shechtman, Kalyan Sunkavalli, Shi-Min Hu
In this work we propose a fully automatic shadow region harmonization approach that improves the appearance compatibility of the de-shadowed region as typically produced by previous methods.
no code implementations • 16 Feb 2016 • Junyan Wang, Sai-Kit Yeung, Jue Wang, Kun Zhou
Comprehensive experiments on both RGB and RGB-D data demonstrate that our simple and effective method significantly outperforms the segmentation propagation methods adopted in the state-of-the-art video cutout systems, and the results also suggest the potential usefulness of our method in image cutout system.
no code implementations • CVPR 2015 • Tao Yue, Jinli Suo, Jue Wang, Xun Cao, Qionghai Dai
Furthermore, by investigating the visual artifacts of aberration degenerated images captured by consumer-level cameras, the non-uniform distribution of sharpness across color channels and the image lattice is exploited as visual priors, resulting in a novel strategy to utilize the guidance from the sharpest channel and local image regions to improve the overall performance and robustness.
no code implementations • CVPR 2014 • Ketan Tang, Jianchao Yang, Jue Wang
Haze is one of the major factors that degrade outdoor images.
no code implementations • CVPR 2014 • Zhe Hu, Sunghyun Cho, Jue Wang, Ming-Hsuan Yang
Images taken in low-light conditions with handheld cameras are often blurry due to the required long exposure time.
Ranked #11 on Deblurring on RealBlur-R (trained on GoPro)
no code implementations • CVPR 2013 • Shulin Yang, Jue Wang, Linda Shapiro
This paper proposes a new supervised semantic edge and gradient extraction approach, which allows the user to roughly scribble over the desired region to extract semantically-dominant and coherent edges in it.
no code implementations • CVPR 2013 • Lin Zhong, Sunghyun Cho, Dimitris Metaxas, Sylvain Paris, Jue Wang
Based on this observation, our method applies a series of directional filters at different orientations to the input image, and estimates an accurate Radon transform of the blur kernel from each filtered image.
no code implementations • NeurIPS 2012 • Shulin Yang, Liefeng Bo, Jue Wang, Linda G. Shapiro
It differs from recognition of basic categories, such as humans, tables, and computers, in that there are global similarities in shape or structure shared within a category, and the differences are in the details of the object parts.
no code implementations • NeurIPS 2010 • Yanjun Han, Qing Tao, Jue Wang
In multi-instance learning, there are two kinds of prediction failure, i. e., false negative and false positive.