no code implementations • 3 Jun 2024 • Haoran Que, Jiaheng Liu, Ge Zhang, Chenchen Zhang, Xingwei Qu, Yinghao Ma, Feiyu Duan, Zhiqi Bai, Jiakai Wang, Yuanxing Zhang, Xu Tan, Jie Fu, Wenbo Su, Jiamang Wang, Lin Qu, Bo Zheng
To address the limitations of existing methods, inspired by the Scaling Law for performance prediction, we propose to investigate the Scaling Law of the Domain-specific Continual Pre-Training (D-CPT Law) to decide the optimal mixture ratio with acceptable training costs for LLMs of different sizes.
no code implementations • 3 Jun 2024 • Ken Deng, Jiaheng Liu, He Zhu, Congnan Liu, Jingxin Li, Jiakai Wang, Peng Zhao, Chenchen Zhang, Yanan Wu, Xueqiao Yin, Yuanxing Zhang, Wenbo Su, Bangyu Xiang, Tiezheng Ge, Bo Zheng
Code completion models have made significant progress in recent years.
1 code implementation • 26 May 2024 • Dongchen Han, Ziyi Wang, Zhuofan Xia, Yizeng Han, Yifan Pu, Chunjiang Ge, Jun Song, Shiji Song, Bo Zheng, Gao Huang
By exploring the similarities and disparities between the effective Mamba and subpar linear attention Transformer, we provide comprehensive analyses to demystify the key factors behind Mamba's success.
no code implementations • 25 May 2024 • Jiayan Guo, Yusen Huo, Zhilin Zhang, Tianyu Wang, Chuan Yu, Jian Xu, Yan Zhang, Bo Zheng
Auto-bidding plays a crucial role in facilitating online advertising by automatically providing bids for advertisers.
1 code implementation • 24 May 2024 • Chunjiang Ge, Sijie Cheng, ZiMing Wang, Jiale Yuan, Yuan Gao, Jun Song, Shiji Song, Gao Huang, Bo Zheng
To enhance the capabilities of ConvLLaVA, we propose two critical optimizations.
Ranked #30 on Visual Question Answering on MM-Vet
no code implementations • 22 May 2024 • Zhendong Liu, Yuanbi Nie, Yingshui Tan, Xiangyu Yue, Qiushi Cui, Chongjun Wang, Xiaoyong Zhu, Bo Zheng
To address this issue, we enhance the existing VLMs' visual modality safety alignment by adding safety modules, including a safety projector, safety tokens, and a safety head, through a two-stage training process, effectively improving the model's defense against risky images.
no code implementations • 23 Apr 2024 • Hongyu Chen, Yiqi Gao, Min Zhou, Peng Wang, Xubin Li, Tiezheng Ge, Bo Zheng
Meanwhile, a network, dubbed as Masked ControlNet, is designed to utilize these object masks for object generation in the misaligned visual control region.
no code implementations • 22 Apr 2024 • Chen Xu, Tianhui Song, Weixin Feng, Xubin Li, Tiezheng Ge, Bo Zheng, LiMin Wang
Diffusion models have significantly advanced the state of the art in image, audio, and video generation tasks.
no code implementations • 22 Apr 2024 • Chengrui Wang, PengFei Liu, Min Zhou, Ming Zeng, Xubin Li, Tiezheng Ge, Bo Zheng
The style guidance is a hand image, e. g., the malformed hand itself, and is employed to furnish the style reference for hand refining.
no code implementations • 26 Mar 2024 • Yusuke Takimoto, Hikari Takehara, Hiroyuki Sato, Zihao Zhu, Bo Zheng
In the film and gaming industries, achieving a realistic hair appearance typically involves the use of strands originating from the scalp.
no code implementations • 11 Mar 2024 • Wenhao Wu, Jialiang Zhou, Ailong He, Shuguang Han, Jufeng Chen, Bo Zheng
Due to limited user interactions for each product (i. e. item), the corresponding item embedding in the CTR model may not easily converge.
no code implementations • 5 Mar 2024 • Zhen Gong, Lvyin Niu, Yang Zhao, Miao Xu, Zhenzhe Zheng, Haoqi Zhang, Zhilin Zhang, Fan Wu, Rongquan Bai, Chuan Yu, Jian Xu, Bo Zheng
Through extensive offline and online experiments, we demonstrate the effectiveness and efficiency of our method, and we obtain a 7. 01% lift in Gross Merchandise Volume, a 7. 42% lift in Return on Investment, and a 3. 26% lift in ad buy count.
no code implementations • 5 Mar 2024 • Weijie Li, Litong Gong, Yiran Zhu, Fanda Fan, Biao Wang, Tiezheng Ge, Bo Zheng
The experimental results demonstrate the effectiveness of our approach in improving the fidelity of generated videos.
no code implementations • 4 Mar 2024 • Litong Gong, Yiran Zhu, Weijie Li, Xiaoyang Kang, Biao Wang, Tiezheng Ge, Bo Zheng
Recently, video generation has achieved significant rapid development based on superior text-to-image generation techniques.
1 code implementation • 4 Mar 2024 • Jiahuan Yan, Bo Zheng, Hongxia Xu, Yiheng Zhu, Danny Z. Chen, Jimeng Sun, Jian Wu, Jintai Chen
Condensing knowledge from diverse domains, language models (LMs) possess the capability to comprehend feature names from various tables, potentially serving as versatile learners in transferring knowledge across distinct tables and diverse prediction tasks, but their discrete text representation space is inherently incompatible with numerical feature values in tables.
no code implementations • 3 Mar 2024 • Jiahuan Yan, Jintai Chen, Chaowen Hu, Bo Zheng, Yaojun Hu, Jimeng Sun, Jian Wu
Recent development of large language models (LLMs) has exhibited impressive zero-shot proficiency on generic and common sense questions.
1 code implementation • 22 Feb 2024 • Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Tiezheng Ge, Bo Zheng, Wanli Ouyang
By conducting a detailed analysis of real multi-turn dialogue data, we construct a three-tier hierarchical ability taxonomy comprising 4208 turns across 1388 multi-turn dialogues in 13 distinct tasks.
1 code implementation • 22 Feb 2024 • Yanan Wu, Jie Liu, Xingyuan Bu, Jiaheng Liu, Zhanhui Zhou, Yuanxing Zhang, Chenchen Zhang, Zhiqi Bai, Haibin Chen, Tiezheng Ge, Wanli Ouyang, Wenbo Su, Bo Zheng
This paper introduces ConceptMath, a bilingual (English and Chinese), fine-grained benchmark that evaluates concept-wise mathematical reasoning of Large Language Models (LLMs).
no code implementations • 19 Feb 2024 • Zhijian Duan, Haoran Sun, Yichong Xia, Siqiang Wang, Zhilin Zhang, Chuan Yu, Jian Xu, Bo Zheng, Xiaotie Deng
Subsequently, we propose a novel optimization method that combines both zeroth-order and first-order techniques to optimize the VVCA parameters.
no code implementations • 13 Jan 2024 • Jiaheng Liu, Zhiqi Bai, Yuanxing Zhang, Chenchen Zhang, Yu Zhang, Ge Zhang, Jiakai Wang, Haoran Que, Yukang Chen, Wenbo Su, Tiezheng Ge, Jie Fu, Wenhu Chen, Bo Zheng
Typically, training LLMs with long context sizes is computationally expensive, requiring extensive training hours and GPU resources.
2 code implementations • 31 Dec 2023 • Haiyang Liu, Zihao Zhu, Giorgio Becherini, Yichen Peng, Mingyang Su, You Zhou, Xuefei Zhe, Naoya Iwamoto, Bo Zheng, Michael J. Black
We propose EMAGE, a framework to generate full-body human gestures from audio and masked gestures, encompassing facial, local body, hands, and global movements.
Ranked #1 on 3D Face Animation on BEAT2
no code implementations • 14 Dec 2023 • Xiaoqiang Gui, Yueyao Cheng, Xiang-Rong Sheng, Yunfeng Zhao, Guoxian Yu, Shuguang Han, Yuning Jiang, Jian Xu, Bo Zheng
A typical practice is privileged features distillation (PFD): train a teacher model using all features (including privileged ones) and then distill the knowledge from the teacher model using a student model (excluding the privileged features), which is then employed for online serving.
1 code implementation • NeurIPS 2023 • Zhanke Zhou, Jiangchao Yao, Jiaxu Liu, Xiawei Guo, Quanming Yao, Li He, Liang Wang, Bo Zheng, Bo Han
To address this dilemma, we propose an information-theory-guided principle, Robust Graph Information Bottleneck (RGIB), to extract reliable supervision signals and avoid representation collapse.
no code implementations • 24 Oct 2023 • Minfang Lu, Yuchen Jiang, Huihui Dong, Qi Li, Ziru Xu, Yuanlin Liu, Lixia Wu, Haoyuan Hu, Han Zhu, Yuning Jiang, Jian Xu, Bo Zheng
The robust representation learning employs domain adversarial learning and multi-view wasserstein distribution learning to learn robust representations.
no code implementations • 24 Sep 2023 • Yixuan Wu, Bo Zheng, Jintai Chen, Danny Z. Chen, Jian Wu
As deep learning methods continue to improve medical image segmentation performance, data annotation is still a big bottleneck due to the labor-intensive and time-consuming burden on medical experts, especially for 3D images.
no code implementations • 14 Sep 2023 • Maonian Wu, Bang Chen, Shaojun Zhu, Bo Zheng, Wei Peng, Mingyi Zhang
A recommendation system assists users in finding items that are relevant to them.
no code implementations • 9 Aug 2023 • Yunfeng Zhao, Xu Yan, Xiaoqiang Gui, Shuguang Han, Xiang-Rong Sheng, Guoxian Yu, Jufeng Chen, Zhao Xu, Bo Zheng
Furthermore, there is delayed feedback in both conversion and refund events and they are sequentially dependent, named cascade delayed feedback (CDF), which significantly harms data freshness for model training.
1 code implementation • 11 Jul 2023 • Bang Chen, Wei Peng, Maonian Wu, Bo Zheng, Shaojun Zhu
Some researchers use user behavior for logic reasoning to achieve recommendation prediction from the perspective of cognitive reasoning, but this kind of reasoning is a local one and ignores implicit information on a global scale.
no code implementations • 29 Jun 2023 • Yu Tian, Bofang Li, Si Chen, Xubin Li, Hongbo Deng, Jian Xu, Bo Zheng, Qian Wang, Chenliang Li
Recently, Multi-Scenario Learning (MSL) is widely used in recommendation and retrieval systems in the industry because it facilitates transfer learning from different scenarios, mitigating data sparsity and reducing maintenance cost.
no code implementations • 12 Jun 2023 • Haozhe Wang, Chao Du, Panyan Fang, Li He, Liang Wang, Bo Zheng
In this regard, we explore the problem of constrained bidding in adversarial bidding environments, which assumes no knowledge about the adversarial factors.
no code implementations • 6 Jun 2023 • Jingyue Gao, Shuguang Han, Han Zhu, Siran Yang, Yuning Jiang, Jian Xu, Bo Zheng
Another line of work relies on costly uniform data that is inadequate to train industrial models.
no code implementations • 6 Jun 2023 • Zhishan Zhao, Jingyue Gao, Yu Zhang, Shuguang Han, Siyuan Lou, Xiang-Rong Sheng, Zhe Wang, Han Zhu, Yuning Jiang, Jian Xu, Bo Zheng
In this architecture, the pre-ranking model is expected to be a lightweight approximation of the ranking model, which handles more candidates with strict latency requirements.
no code implementations • 23 May 2023 • Wenhao Zhu, Tianyu Wen, Guojie Song, Liang Wang, Bo Zheng
Graph Transformer has recently received wide attention in the research community with its outstanding performance, yet its structural expressive power has not been well analyzed.
1 code implementation • 22 May 2023 • Zhangming Chan, Yu Zhang, Shuguang Han, Yong Bai, Xiang-Rong Sheng, Siyuan Lou, Jiacen Hu, Baolin Liu, Yuning Jiang, Jian Xu, Bo Zheng
However, we observe that a well-trained CVR prediction model often performs sub-optimally during sales promotions.
no code implementations • 15 May 2023 • Penghui Wei, Hongjian Dou, Shaoguo Liu, Rongjun Tang, Li Liu, Liang Wang, Bo Zheng
We introduce FedAds, the first benchmark for CVR estimation with vFL, to facilitate standardized and systematical evaluations for vFL algorithms.
no code implementations • 14 Mar 2023 • Jinchao Li, Kaitao Song, Junan Li, Bo Zheng, Dongsheng Li, Xixin Wu, Xunying Liu, Helen Meng
This paper presents several efficient methods to extract better AD-related cues from high-level acoustic and linguistic features.
no code implementations • 10 Mar 2023 • Xuanhua Yang, Jianxin Zhao, Shaoguo Liu, Liang Wang, Bo Zheng
Multi-task learning (MTL) has been widely applied in online advertising and recommender systems.
no code implementations • 6 Feb 2023 • Shanlei Mu, Penghui Wei, Wayne Xin Zhao, Shaoguo Liu, Liang Wang, Bo Zheng
In this paper, we propose a Hybrid Contrastive Constrained approach (HC^2) for multi-scenario ad ranking.
no code implementations • 6 Feb 2023 • Penghui Wei, Yongqiang Chen, Shaoguo Liu, Liang Wang, Bo Zheng
In a whole delivery period, advertisers usually desire a certain impression count for the ads, and they also expect that the delivery performance is as good as possible (e. g., obtaining high click-through rate).
no code implementations • 5 Jan 2023 • Bo Zheng, Zhouyang Li, Fuxuan Wei, Qiguang Chen, Libo Qin, Wanxiang Che
Multilingual spoken language understanding (SLU) consists of two sub-tasks, namely intent detection and slot filling.
1 code implementation • 28 Dec 2022 • Shiyu Wang, Fan Zhou, Yinbo Sun, Lintao Ma, James Zhang, Yangfei Zheng, Bo Zheng, Lei Lei, Yun Hu
Multivariate time series forecasting with hierarchical structure is pervasive in real-world applications, demanding not only predicting each level of the hierarchy, but also reconciling all forecasts to ensure coherency, i. e., the forecasts should satisfy the hierarchical aggregation constraints.
no code implementations • 21 Nov 2022 • Zixuan Xu, Penghui Wei, Shaoguo Liu, Weimin Zhang, Liang Wang, Bo Zheng
Conventional graph neural network based methods usually deal with each domain separately, or train a shared model to serve all domains.
1 code implementation • 29 Oct 2022 • Yi Wang, Jiajun Deng, Tianzi Wang, Bo Zheng, Shoukang Hu, Xunying Liu, Helen Meng
Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care and to delay further progression.
1 code implementation • 13 Oct 2022 • Zhiyu Mou, Yusen Huo, Rongquan Bai, Mingzhou Xie, Chuan Yu, Jian Xu, Bo Zheng
Due to safety concerns, it was believed that the RL training process can only be carried out in an offline virtual advertising system (VAS) that is built based on the historical data generated in the RAS.
1 code implementation • 4 Sep 2022 • Zhao-Yu Zhang, Xiang-Rong Sheng, Yujing Zhang, Biye Jiang, Shuguang Han, Hongbo Deng, Bo Zheng
However, far less attention has been paid to the overfitting problem of models in recommendation systems, which, on the contrary, is recognized as a critical issue for deep neural networks.
no code implementations • 29 Aug 2022 • Zihan Lin, Xuanhua Yang, Xiaoyu Peng, Wayne Xin Zhao, Shaoguo Liu, Liang Wang, Bo Zheng
For this purpose, we build a relatedness prediction network, so that it can predict the contrast strength for inter-task representations of an instance.
no code implementations • 22 Aug 2022 • Yujing Zhang, Zhangming Chan, Shuhao Xu, Weijie Bian, Shuguang Han, Hongbo Deng, Bo Zheng
To alleviate this issue, we propose to extract knowledge from the \textit{super-domain} that contains web-scale and long-time impression data, and further assist the online recommendation task (downstream task).
no code implementations • 12 Aug 2022 • Xiang-Rong Sheng, Jingyue Gao, Yueyao Cheng, Siran Yang, Shuguang Han, Hongbo Deng, Yuning Jiang, Jian Xu, Bo Zheng
It can be attributed to the calibration ability of the pointwise loss since the prediction can be viewed as the click probability.
no code implementations • 27 Jun 2022 • Xuanhua Yang, Xiaoyu Peng, Penghui Wei, Shaoguo Liu, Liang Wang, Bo Zheng
Click-through rate (CTR) prediction is a fundamental technique in recommendation and advertising systems.
no code implementations • 20 Jun 2022 • Yuchen Jiang, Qi Li, Han Zhu, Jinbei Yu, Jin Li, Ziru Xu, Huihui Dong, Bo Zheng
Industrial recommender systems usually hold data from multiple business scenarios and are expected to provide recommendation services for these scenarios simultaneously.
1 code implementation • 10 Jun 2022 • Haozhe Wang, Chao Du, Panyan Fang, Shuo Yuan, Xuming He, Liang Wang, Bo Zheng
Real-Time Bidding (RTB) is an important mechanism in modern online advertising systems.
no code implementations • 2 Jun 2022 • Mingyuan Cheng, Xinru Liao, Quan Liu, Bin Ma, Jian Xu, Bo Zheng
Learning individual-level treatment effect is a fundamental problem in causal inference and has received increasing attention in many areas, especially in the user growth area which concerns many internet companies.
no code implementations • 31 May 2022 • Dagui Chen, Qi Yan, Chunjie Chen, Zhenzhe Zheng, Yangsu Liu, Zhenjia Ma, Chuan Yu, Jian Xu, Bo Zheng
To this end, adaptive ad exposure has become an appealing strategy to boost the overall performance of the feed.
no code implementations • 30 May 2022 • Penghui Wei, Shaoguo Liu, Xuanhua Yang, Liang Wang, Bo Zheng
Current bundle generation studies focus on generating a combination of items to improve user experience.
no code implementations • 23 May 2022 • Wenbo Su, Yuanxing Zhang, Yufeng Cai, Kaixu Ren, Pengjie Wang, Huimin Yi, Yue Song, Jing Chen, Hongbo Deng, Jian Xu, Lin Qu, Bo Zheng
High-concurrency asynchronous training upon parameter server (PS) architecture and high-performance synchronous training upon all-reduce (AR) architecture are the most commonly deployed distributed training modes for recommendation models.
no code implementations • NAACL (ACL) 2022 • Penghui Wei, Xuanhua Yang, Shaoguo Liu, Liang Wang, Bo Zheng
This paper focuses on automatically generating the text of an ad, and the goal is that the generated text can capture user interest for achieving higher click-through rate (CTR).
no code implementations • 15 May 2022 • Penghui Wei, Weimin Zhang, Ruijie Hou, Jinquan Liu, Shaoguo Liu, Liang Wang, Bo Zheng
Calibration techniques aim to post-process model predictions to posterior probabilities.
no code implementations • 9 May 2022 • Si Chen, Chen Lin, Wanxian Guan, Jiayi Wei, Xingyuan Bu, He guo, Hui Li, Xubin Li, Jian Xu, Bo Zheng
In this paper, we present a visual encoding framework for CTR prediction to overcome these problems.
1 code implementation • ACL 2022 • Damai Dai, Li Dong, Shuming Ma, Bo Zheng, Zhifang Sui, Baobao Chang, Furu Wei
We point out that existing learning-to-route MoE methods suffer from the routing fluctuation issue, i. e., the target expert of the same input may change along with training, but only one expert will be activated for the input during inference.
1 code implementation • 11 Apr 2022 • Yuanxing Zhang, Langshi Chen, Siran Yang, Man Yuan, Huimin Yi, Jie Zhang, Jiamang Wang, Jianbo Dong, Yunlong Xu, Yue Song, Yong Li, Di Zhang, Wei Lin, Lin Qu, Bo Zheng
However, we observe that GPU devices in training recommender systems are underutilized, and they cannot attain an expected throughput improvement as what it has achieved in CV and NLP areas.
no code implementations • 4 Apr 2022 • Yusuke Takimoto, Hiroyuki Sato, Hikari Takehara, Keishiro Uragaki, Takehiro Tawara, Xiao Liang, Kentaro Oku, Wataru Kishimoto, Bo Zheng
HardSoftRas, our novel rendering process, is designed for inverse rendering with a graphics pipeline.
1 code implementation • 30 Mar 2022 • Bencheng Yan, Pengjie Wang, Kai Zhang, Feng Li, Hongbo Deng, Jian Xu, Bo Zheng
In many web applications, deep learning-based CTR prediction models (deep CTR models for short) are widely adopted.
no code implementations • 28 Mar 2022 • Zhirong Xu, Shiyang Wen, Junshan Wang, Guojun Liu, Liang Wang, Zhi Yang, Lei Ding, Yan Zhang, Di Zhang, Jian Xu, Bo Zheng
Moreover, to deploy AMCAD in Taobao, one of the largest ecommerce platforms with hundreds of million users, we design an efficient two-layer online retrieval framework for the task of graph based advertisement retrieval.
2 code implementations • 10 Mar 2022 • Haiyang Liu, Zihao Zhu, Naoya Iwamoto, Yichen Peng, Zhengqing Li, You Zhou, Elif Bozkurt, Bo Zheng
Achieving realistic, vivid, and human-like synthesized conversational gestures conditioned on multi-modal data is still an unsolved problem due to the lack of available datasets, models and standard evaluation metrics.
Ranked #1 on Gesture Generation on BEAT
no code implementations • 14 Feb 2022 • Rihan Chen, Bin Liu, Han Zhu, Yaoxuan Wang, Qi Li, Buting Ma, Qingbo Hua, Jun Jiang, Yunlong Xu, Hongbo Deng, Bo Zheng
In this paper, we propose a novel method to extend ANN search to arbitrary matching functions, e. g., a deep neural network.
1 code implementation • 14 Feb 2022 • Yu Chen, Jiaqi Jin, Hui Zhao, Pengjie Wang, Guojun Liu, Jian Xu, Bo Zheng
Moreover, to estimate CVR upon the freshly observed but biased distribution with fake negatives, the importance sampling is widely used to reduce the distribution bias.
1 code implementation • 9 Feb 2022 • Siguang Huang, Yunli Wang, Lili Mou, Huayue Zhang, Han Zhu, Chuan Yu, Bo Zheng
In previous work, researchers have developed several calibration methods to post-process the outputs of a predictor to obtain calibrated values, such as binning and scaling methods.
no code implementations • 20 Jan 2022 • Zixuan Xu, Penghui Wei, Weimin Zhang, Shaoguo Liu, Liang Wang, Bo Zheng
Then a student model is trained on both clicked and unclicked ads with knowledge distillation, performing uncertainty modeling to alleviate the inherent noise in pseudo-labels.
no code implementations • 18 Jan 2022 • Qianqian Zhang, Xinru Liao, Quan Liu, Jian Xu, Bo Zheng
Advertisers play an essential role in many e-commerce platforms like Taobao and Amazon.
no code implementations • 21 Dec 2021 • Kailun Wu, Zhangming Chan, Weijie Bian, Lejian Ren, Shiming Xiang, Shuguang Han, Hongbo Deng, Bo Zheng
We further show that such a process is equivalent to adding an adversarial perturbation to the model input, and thereby name our proposed approach as an the Adversarial Gradient Driven Exploration (AGE).
no code implementations • 8 Nov 2021 • Haibin Wu, Bo Zheng, Xu Li, Xixin Wu, Hung-Yi Lee, Helen Meng
As the paradigm of the self-supervised learning upstream model followed by downstream tasks arouses more attention in the speech community, characterizing the adversarial robustness of such paradigm is of high priority.
no code implementations • 1 Nov 2021 • Zongtao Liu, Bin Ma, Quan Liu, Jian Xu, Bo Zheng
When speaking of sponsored search, bid keyword recommendation is the fundamental service.
2 code implementations • EMNLP 2021 • Bo Zheng, Li Dong, Shaohan Huang, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei
We find that many languages are under-represented in recent cross-lingual language models due to the limited vocabulary capacity.
no code implementations • 24 Aug 2021 • Bencheng Yan, Pengjie Wang, Kai Zhang, Wei Lin, Kuang-Chih Lee, Jian Xu, Bo Zheng
Each feature value is mapped to an embedding vector via an embedding learning process.
no code implementations • 24 Aug 2021 • Bencheng Yan, Pengjie Wang, Jinquan Liu, Wei Lin, Kuang-Chih Lee, Jian Xu, Bo Zheng
In these applications, embedding learning of categorical features is crucial to the success of deep learning models.
3 code implementations • ACL 2022 • Zewen Chi, Shaohan Huang, Li Dong, Shuming Ma, Bo Zheng, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, Heyan Huang, Furu Wei
In this paper, we introduce ELECTRA-style tasks to cross-lingual language model pre-training.
Ranked #1 on Zero-Shot Cross-Lingual Transfer on XTREME
1 code implementation • ECCV 2020 • Zhihang Zhong, Ye Gao, Yinqiang Zheng, Bo Zheng, Imari Sato
Real-world video deblurring in real time still remains a challenging task due to the complexity of spatially and temporally varying blur itself and the requirement of low computational cost.
Ranked #35 on Image Deblurring on GoPro (using extra training data)
1 code implementation • ACL 2021 • Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei
Fine-tuning pre-trained cross-lingual language models can transfer task-specific supervision from one language to the others.
1 code implementation • ACL 2021 • Zewen Chi, Li Dong, Bo Zheng, Shaohan Huang, Xian-Ling Mao, Heyan Huang, Furu Wei
The cross-lingual language models are typically pretrained with masked language modeling on multilingual text or parallel sentences.
1 code implementation • 11 Jun 2021 • Chao Wen, Miao Xu, Zhilin Zhang, Zhenzhe Zheng, Yuhui Wang, Xiangyu Liu, Yu Rong, Dong Xie, Xiaoyang Tan, Chuan Yu, Jian Xu, Fan Wu, Guihai Chen, Xiaoqiang Zhu, Bo Zheng
Third, to deploy MAAB in the large-scale advertising system with millions of advertisers, we propose a mean-field approach.
no code implementations • 8 Jun 2021 • Ziyu Guan, Hongchang Wu, Qingyu Cao, Hao liu, Wei Zhao, Sheng Li, Cai Xu, Guang Qiu, Jian Xu, Bo Zheng
Although a few studies use multi-agent reinforcement learning to set up a cooperative game, they still suffer the following drawbacks: (1) They fail to avoid collusion solutions where all the advertisers involved in an auction collude to bid an extremely low price on purpose.
no code implementations • 17 May 2021 • Xu Ma, Pengjie Wang, Hui Zhao, Shaoguo Liu, Chuhan Zhao, Wei Lin, Kuang-Chih Lee, Jian Xu, Bo Zheng
In real-world search, recommendation, and advertising systems, the multi-stage ranking architecture is commonly adopted.
no code implementations • 17 May 2021 • Feng Li, Bencheng Yan, Qingqing Long, Pengjie Wang, Wei Lin, Jian Xu, Bo Zheng
Most of the existing methods adopt a DNN-based model to capture the cross features in an implicit manner.
1 code implementation • ICCV 2021 • Yang Li, Hikari Takehara, Takafumi Taketomi, Bo Zheng, Matthias Nießner
Tracking non-rigidly deforming scenes using range sensors has numerous applications including computer vision, AR/VR, and robotics.
no code implementations • 11 Mar 2021 • Guannan Geng, Qingyang Xiao, Shigan Liu, Xiaodong Liu, Jing Cheng, Yixuan Zheng, Dan Tong, Bo Zheng, Yiran Peng, Xiaomeng Huang, Kebin He, Qiang Zhang
Accordingly, a full-coverage high-resolution air pollutant dataset with timely updates and historical long-term records is essential to support both research and environmental management.
no code implementations • 3 Mar 2021 • Zhu Liu, Zhu Deng, Philippe Ciais, Jianguang Tan, Biqing Zhu, Steven J. Davis, Robbie Andrew, Olivier Boucher, Simon Ben Arous, Pep Canadel, Xinyu Dou, Pierre Friedlingstein, Pierre Gentine, Rui Guo, Chaopeng Hong, Robert B. Jackson, Daniel M. Kammen, Piyu Ke, Corinne Le Quere, Crippa Monica, Greet Janssens-Maenhout, Glen Peters, Katsumasa Tanaka, Yilong Wang, Bo Zheng, Haiwang Zhong, Taochun Sun, Hans Joachim Schellnhuber
That even substantial world-wide lockdowns of activity led to a one-time decline in global CO$_2$ emissions of only 5. 4% in one year highlights the significant challenges for climate change mitigation that we face in the post-COVID era.
Atmospheric and Oceanic Physics General Economics Economics
1 code implementation • ACL 2020 • Bo Zheng, Haoyang Wen, Yaobo Liang, Nan Duan, Wanxiang Che, Daxin Jiang, Ming Zhou, Ting Liu
Natural Questions is a new challenging machine reading comprehension benchmark with two-grained answers, which are a long answer (typically a paragraph) and a short answer (one or more entities inside the long answer).
no code implementations • 7 Mar 2020 • Xiang Li, Chao Wang, Jiwei Tan, Xiaoyi Zeng, Dan Ou, Bo Zheng
Finally, we achieve the multimodal item representations by combining both modality-specific and modality-invariant representations.
no code implementations • 28 Oct 2019 • Xiuying Chen, Daorui Xiao, Shen Gao, Guojun Liu, Wei. Lin, Bo Zheng, Dongyan Zhao, Rui Yan
Sponsored search optimizes revenue and relevance, which is estimated by Revenue Per Mille (RPM).
no code implementations • 25 Sep 2019 • Yu He, Shiyang Wen, Wenjin Wu, Yan Zhang, Siran Yang, Yuan Wei, Di Zhang, Guojie Song, Wei Lin, Liang Wang, Bo Zheng
The Graph Convolutional Network (GCN) and its variants are powerful models for graph representation learning and have recently achieved great success on many graph-based applications.
no code implementations • 1 Jul 2019 • Bo wang, Minghui Qiu, Xisen Wang, Yaliang Li, Yu Gong, Xiaoyi Zeng, Jung Huang, Bo Zheng, Deng Cai, Jingren Zhou
To the best of our knowledge, this is the first to build a minimax game based model for selective transfer learning.
no code implementations • 24 Feb 2019 • Ryuichi Takanobu, Tao Zhuang, Minlie Huang, Jun Feng, Haihong Tang, Bo Zheng
In this paper, we investigate the task of aggregating search results from heterogeneous sources in an E-commerce environment.
Hierarchical Reinforcement Learning reinforcement-learning +3
1 code implementation • EMNLP 2018 • Yijia Liu, Wanxiang Che, Bo Zheng, Bing Qin, Ting Liu
In this paper, we propose a new rich resource enhanced AMR aligner which produces multiple alignments and a new transition system for AMR parsing along with its oracle parser.
Ranked #2 on AMR Parsing on LDC2014T12:
1 code implementation • CONLL 2018 • Wanxiang Che, Yijia Liu, Yuxuan Wang, Bo Zheng, Ting Liu
This paper describes our system (HIT-SCIR) submitted to the CoNLL 2018 shared task on Multilingual Parsing from Raw Text to Universal Dependencies.
Ranked #3 on Dependency Parsing on Universal Dependencies
no code implementations • CONLL 2017 • Wanxiang Che, Jiang Guo, Yuxuan Wang, Bo Zheng, Huaipeng Zhao, Yang Liu, Dechuan Teng, Ting Liu
Our system includes three pipelined components: \textit{tokenization}, \textit{Part-of-Speech} (POS) \textit{tagging} and \textit{dependency parsing}.
no code implementations • WS 2016 • Bo Zheng, Wanxiang Che, Jiang Guo, Ting Liu
This paper introduces our Chinese Grammatical Error Diagnosis (CGED) system in the NLP-TEA-3 shared task for CGED.
no code implementations • CVPR 2014 • Xiangqi Huang, Bo Zheng, Takeshi Masuda, Katsushi Ikeuchi
Our feature description is designed as two steps: 1) we normalize the detected local regions to canonical shapes for robust matching; 2) we encode each key point with multiple vectors at different Morse function values.
no code implementations • CVPR 2013 • Bo Zheng, Yibiao Zhao, Joey C. Yu, Katsushi Ikeuchi, Song-Chun Zhu
In this paper, we present an approach for scene understanding by reasoning physical stability of objects from point cloud.