no code implementations • 22 May 2024 • YanRong Li, Tengxiang Zhang, Xin Zeng, Yuntao Wang, Haotian Zhang, Yiqiang Chen
Recent advancements in augmented reality (AR) have enabled the use of various sensors on smart glasses for applications like facial reconstruction, which is vital to improve AR experiences for virtual social activities.
no code implementations • 3 May 2024 • Yingshuang Zou, Yikang Ding, Xi Qiu, Haoqian Wang, Haotian Zhang
This paper presents a novel self-supervised two-frame multi-camera metric depth estimation network, termed M${^2}$Depth, which is designed to predict reliable scale-aware surrounding depth in autonomous driving.
1 code implementation • 29 Apr 2024 • Chenyang Liu, Keyan Chen, Bowen Chen, Haotian Zhang, Zhengxia Zou, Zhenwei Shi
In terms of efficient temporal modeling, considering the potential correlation between the temporal scanning characteristics of Mamba and the temporality of the RSICC, we propose the Temporal-Traversing SSM (TT-SSM), which scans bi-temporal features in a temporal cross-wise manner, enhancing the model's temporal understanding and information interaction.
no code implementations • 11 Apr 2024 • Haotian Zhang, Haoxuan You, Philipp Dufter, BoWen Zhang, Chen Chen, Hong-You Chen, Tsu-Jui Fu, William Yang Wang, Shih-Fu Chang, Zhe Gan, Yinfei Yang
While Ferret seamlessly integrates regional understanding into the Large Language Model (LLM) to facilitate its referring and grounding capability, it poses certain limitations: constrained by the pre-trained fixed visual encoder and failed to perform well on broader tasks.
Ranked #61 on Visual Question Answering on MM-Vet
no code implementations • 8 Apr 2024 • Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Swearngin, Jeffrey Nichols, Yinfei Yang, Zhe Gan
For model evaluation, we establish a comprehensive benchmark encompassing all the aforementioned tasks.
1 code implementation • 28 Mar 2024 • Chenyang Liu, Keyan Chen, Haotian Zhang, Zipeng Qi, Zhengxia Zou, Zhenwei Shi
The Change-Agent integrates a multi-level change interpretation (MCI) model as the eyes and a large language model (LLM) as the brain.
no code implementations • 16 Mar 2024 • Jizhe Dou, Haotian Zhang, Guodong Sun
To address this issue, we present a hybrid-action deep reinforcement learning framework, called HaDMC, which uses a standard policy learning algorithm to generate latent continuous actions.
no code implementations • 14 Mar 2024 • Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier, Sam Dodge, BoWen Zhang, Philipp Dufter, Dhruti Shah, Xianzhi Du, Futang Peng, Floris Weers, Anton Belyi, Haotian Zhang, Karanjeet Singh, Doug Kang, Ankur Jain, Hongyu Hè, Max Schwarzer, Tom Gunter, Xiang Kong, Aonan Zhang, Jianyu Wang, Chong Wang, Nan Du, Tao Lei, Sam Wiseman, Guoli Yin, Mark Lee, ZiRui Wang, Ruoming Pang, Peter Grasch, Alexander Toshev, Yinfei Yang
Through careful and comprehensive ablations of the image encoder, the vision language connector, and various pre-training data choices, we identified several crucial design lessons.
Ranked #21 on Visual Question Answering on MM-Vet
no code implementations • 9 Mar 2024 • Cunhui Dong, Haichuan Ma, Haotian Zhang, Changsheng Gao, Li Li, Dong Liu
Neural network-based image coding has been developing rapidly since its birth.
1 code implementation • 20 Feb 2024 • Yusu Qian, Haotian Zhang, Yinfei Yang, Zhe Gan
The remarkable advancements in Multimodal Large Language Models (MLLMs) have not rendered them immune to challenges, particularly in the context of handling deceptive information in prompts, thus producing hallucinated responses under such conditions.
no code implementations • 23 Dec 2023 • Chenyang Liu, Keyan Chen, Zipeng Qi, Haotian Zhang, Zhengxia Zou, Zhenwei Shi
The existing methods for Remote Sensing Image Change Captioning (RSICC) perform well in simple scenes but exhibit poorer performance in complex scenes.
2 code implementations • 23 Dec 2023 • Keyan Chen, Chengyang Liu, Wenyuan Li, Zili Liu, Hao Chen, Haotian Zhang, Zhengxia Zou, Zhenwei Shi
Change detection, a prominent research area in remote sensing, is pivotal in observing and analyzing surface transformations.
Ranked #6 on Change Detection on LEVIR-CD
no code implementations • 11 Dec 2023 • Haotian Zhang, Semujju Stuart Dereck, Zhicheng Wang, Xianwei Lv, Kang Xu, Liang Wu, Ye Jia, Jing Wu, Zhuo Long, Wensheng Liang, X. G. Ma, Ruiyan Zhuang
Although the applications of artificial intelligence especially deep learning had greatly improved various aspects of intelligent manufacturing, they still face challenges for wide employment due to the poor generalization ability, difficulties to establish high-quality training datasets, and unsatisfactory performance of deep learning methods.
1 code implementation • 11 Oct 2023 • Haoxuan You, Haotian Zhang, Zhe Gan, Xianzhi Du, BoWen Zhang, ZiRui Wang, Liangliang Cao, Shih-Fu Chang, Yinfei Yang
We introduce Ferret, a new Multimodal Large Language Model (MLLM) capable of understanding spatial referring of any shape or granularity within an image and accurately grounding open-vocabulary descriptions.
1 code implementation • 11 Oct 2023 • Zhengfeng Lai, Haotian Zhang, BoWen Zhang, Wentao Wu, Haoping Bai, Aleksei Timofeev, Xianzhi Du, Zhe Gan, Jiulong Shan, Chen-Nee Chuah, Yinfei Yang, Meng Cao
For example, VeCLIP achieves up to +25. 2% gain in COCO and Flickr30k retrieval tasks under the 12M setting.
no code implementations • 4 Oct 2023 • Haotian Zhang, Shijian Gao, Xiang Cheng, Liuqing Yang
The future of vehicular communication networks relies on mmWave massive multi-input-multi-output antenna arrays for intensive data transfer and massive vehicle access.
no code implementations • 29 Sep 2023 • Haotian Zhang, Li Li, Dong Liu
In principle, we find two factors crucial: one is the discrepancy between the surrogate and rounding, leading to train-test mismatch; the other is gradient estimation risk due to the surrogate, which consists of bias and variance of the gradient estimation.
no code implementations • 4 Aug 2023 • Haotian Zhang, Huifeng Zhao, Xujun Zhang, Qun Su, Hongyan Du, Chao Shen, Zhe Wang, Dan Li, Peichen Pan, Guangyong Chen, Yu Kang, Chang-Yu Hsieh, Tingjun Hou
Drug discovery is a highly complicated process, and it is unfeasible to fully commit it to the recently developed molecular generation methods.
no code implementations • 11 Jul 2023 • Chuanbo Tang, Xihua Sheng, Zhuoyuan Li, Haotian Zhang, Li Li, Dong Liu
In the offline stage, we fine-tune a trained optical flow estimation network with the motion information provided by a traditional (non-deep) video compression scheme, e. g. H. 266/VVC, as we believe the motion information of H. 266/VVC achieves a better rate-distortion trade-off.
1 code implementation • 28 Jun 2023 • Keyan Chen, Chenyang Liu, Hao Chen, Haotian Zhang, Wenyuan Li, Zhengxia Zou, Zhenwei Shi
We also propose several ongoing derivatives for instance segmentation tasks, drawing on recent advancements within the SAM community, and compare their performance with RSPrompter.
no code implementations • 25 Jun 2023 • Xiang Cheng, Haotian Zhang, Jianan Zhang, Shijian Gao, Sijiang Li, Ziwei Huang, Lu Bai, Zonghui Yang, Xinhu Zheng, Liuqing Yang
Currently, some research efforts have been devoted to exploring multi-modal sensing-communication integration but still lack a comprehensive review.
no code implementations • 25 Jun 2023 • Xiang Cheng, Ziwei Huang, Lu Bai, Haotian Zhang, Mingran Sun, Boxun Liu, Sijiang Li, Jianan Zhang, Minson Lee
A comprehensive dataset is a prerequisite for 6G integrated sensing-communication research.
no code implementations • 24 May 2023 • Zhuokai Zhao, Yang Yang, Wenyu Wang, Chihuang Liu, Yu Shi, Wenjie Hu, Haotian Zhang, Shuang Yang
A key puzzle in search, ads, and recommendation is that the ranking model can only utilize a small portion of the vastly available user interaction data.
1 code implementation • 24 May 2023 • Hao Chen, Haotian Zhang, Keyan Chen, Chenyao Zhou, Song Chen, Zhengxia Zou, Zhenwei Shi
Toward continuous cross-resolution CD, we propose scale-invariant learning to enforce the model consistently predicting HR results given synthesized samples of varying resolution differences.
no code implementations • 8 Mar 2023 • Haotian Zhang, Jintu Zhang, Huifeng Zhao, Dejun Jiang, Yafeng Deng
Recently, with the development of geometric neural networks, the data-driven schemes have been successfully applied in this field, both for molecular conformation generation (in vacuum) and binding pose generation (in protein pocket).
no code implementations • 12 Nov 2022 • Haotian Zhang, Linxiaoyi Wan
To design a drug given a biological molecule by using deep learning methods, there are many successful models published recently.
no code implementations • 5 Oct 2022 • Haotian Zhang, Chenyang Bu, Fei Liu, Shuochen Liu, Yuhong Zhang, Xuegang Hu
If a student wants to answer a question correctly, the student should not only master the set of skills involved in the question but also think and obtain the associative path on the skills graph.
1 code implementation • 21 Jul 2022 • Yikang Ding, Qingtian Zhu, Xiangyue Liu, Wentao Yuan, Haotian Zhang, Chi Zhang
Supervised multi-view stereo (MVS) methods have achieved remarkable progress in terms of reconstruction quality, but suffer from the challenge of collecting large-scale ground-truth depth.
1 code implementation • 21 Jul 2022 • Wentao Yuan, Qingtian Zhu, Xiangyue Liu, Yikang Ding, Haotian Zhang, Chi Zhang
Recently, Implicit Neural Representations (INRs) parameterized by neural networks have emerged as a powerful and promising tool to represent different kinds of signals due to its continuous, differentiable properties, showing superiorities to classical discretized representations.
2 code implementations • 20 Jul 2022 • James Hong, Haotian Zhang, Michaël Gharbi, Matthew Fisher, Kayvon Fatahalian
We introduce the task of spotting temporally precise, fine-grained events in video (detecting the precise moment in time events occur).
Ranked #6 on Action Spotting on SoccerNet-v2
no code implementations • 15 Jul 2022 • Haiyi Mao, Minxue Jia, Jason Xiaotian Dou, Haotian Zhang, Panayiotis V. Benos
Metacells are disjoint and homogeneous groups of single-cell profiles, representing discrete and highly granular cell states.
1 code implementation • 12 Jun 2022 • Haotian Zhang, Pengchuan Zhang, Xiaowei Hu, Yen-Chun Chen, Liunian Harold Li, Xiyang Dai, Lijuan Wang, Lu Yuan, Jenq-Neng Hwang, Jianfeng Gao
We present GLIPv2, a grounded VL understanding model, that serves both localization tasks (e. g., object detection, instance segmentation) and Vision-Language (VL) understanding tasks (e. g., VQA, image captioning).
Ranked #1 on Phrase Grounding on Flickr30k Entities Test (using extra training data)
2 code implementations • CVPR 2022 • Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, Jianfeng Gao
The unification brings two benefits: 1) it allows GLIP to learn from both detection and grounding data to improve both tasks and bootstrap a good grounding model; 2) GLIP can leverage massive image-text pairs by generating grounding boxes in a self-training fashion, making the learned representation semantic-rich.
Ranked #1 on 2D Object Detection on RF100
no code implementations • 29 Nov 2021 • Weihan Li, Haotian Zhang, Bruis van Vlijmen, Philipp Dechent, Dirk Uwe Sauer
In this paper, we propose a data-driven prognostics framework to predict both capacity and power fade simultaneously with multi-task learning.
1 code implementation • CVPR 2022 • Yikang Ding, Wentao Yuan, Qingtian Zhu, Haotian Zhang, Xiangyue Liu, Yuanjiang Wang, Xiao Liu
We analogize MVS back to its nature of a feature matching task and therefore propose a powerful Feature Matching Transformer (FMT) to leverage intra- (self-) and inter- (cross-) attention to aggregate long-range context information within and across images.
Ranked #8 on 3D Reconstruction on DTU
no code implementations • 25 Nov 2021 • Shuxue Peng, Zihang He, Haotian Zhang, Ran Yan, Chuting Wang, Qingtian Zhu, Xiao Liu
In this paper, we present a visual localization pipeline, namely MegLoc, for robust and accurate 6-DoF pose estimation under varying scenarios, including indoor and outdoor scenes, different time across a day, different seasons across a year, and even across years.
no code implementations • 10 Aug 2021 • Xiaopeng Bi, Yu Chen, Xinyang Liu, Dehao Zhang, Ran Yan, Zheng Chai, Haotian Zhang, Xiao Liu
This report describes Megvii-3D team's approach towards CVPR 2021 Image Matching Workshop.
no code implementations • 10 Aug 2021 • Xiaopeng Bi, Ran Yan, Zheng Chai, Haotian Zhang, Xiao Liu
This report describes Megvii-3D team's approach towards SimLocMatch Challenge @ CVPR 2021 Image Matching Workshop.
no code implementations • ICCV 2021 • Haotian Zhang, Yicheng Luo, Fangbo Qin, Yijia He, Xiao Liu
The line description ability of ELSD also outperforms the previous works on the line matching task.
Ranked #1 on Line Segment Detection on wireframe dataset
1 code implementation • 3 Nov 2020 • Haotian Zhang, Yuhao Wang, Jianyong Sun, Zongben Xu
Efficient exploration is one of the most important issues in deep reinforcement learning.
1 code implementation • Findings of the Association for Computational Linguistics 2020 • Ning Shi, Ziheng Zeng, Haotian Zhang, Yichen Gong
In neural text editing, prevalent sequence-to-sequence based approaches directly map the unedited text either to the edited text or the editing operations, in which the performance is degraded by the limited source text encoding and long, varying decoding steps.
no code implementations • 24 Jun 2020 • Jiarui Cai, Yizhou Wang, Haotian Zhang, Hung-Min Hsu, Chengqian Ma, Jenq-Neng Hwang
Meanwhile, the spatial attention, which focuses on the foreground within the bounding boxes, is generated from the given instance masks and applied to the extracted embedding features.
no code implementations • 10 Mar 2020 • Haotian Zhang, Jianyong Sun, Zongben Xu
This paper proposes to learn a two-phase (including a minimization phase and an escaping phase) global optimization algorithm for smooth non-convex functions.
no code implementations • 4 Mar 2020 • Haotian Zhang, Jianyong Sun, Zongben Xu
This paper proposes the first-ever algorithmic framework for tuning hyper-parameters of stochastic optimization algorithm based on reinforcement learning.
no code implementations • 2 Mar 2020 • Haotian Zhang, Jianyong Sun, Zongben Xu
Tuning hyper-parameters for evolutionary algorithms is an important issue in computational intelligence.
no code implementations • ICLR 2020 • Haotian Zhang, Jian Sun, Zongben Xu
Bayesian optimization is an effective tool to optimize black-box functions and popular for hyper-parameter tuning in machine learning.
no code implementations • IJCNLP 2019 • Zeynep Akkalyoncu Yilmaz, Wei Yang, Haotian Zhang, Jimmy Lin
This paper applies BERT to ad hoc document retrieval on news articles, which requires addressing two challenges: relevance judgments in existing test collections are typically provided only at the document level, and documents often exceed the length that BERT was designed to handle.
no code implementations • IJCNLP 2019 • Zeynep Akkalyoncu Yilmaz, Shengjin Wang, Wei Yang, Haotian Zhang, Jimmy Lin
We present Birch, a system that applies BERT to document retrieval via integration with the open-source Anserini information retrieval toolkit to demonstrate end-to-end search over large document collections.
no code implementations • 18 Oct 2019 • Haotian Zhang, Gaoang Wang, Zhichao Lei, Jenq-Neng Hwang
Drones, or general UAVs, equipped with a single camera have been widely deployed to a broad range of applications, such as aerial photography, fast goods delivery and most importantly, surveillance.
no code implementations • 8 Oct 2019 • Henry H. Yu, Jiang Liu, Hao Sun, Ziwen Wang, Haotian Zhang
Image pairing is an important research task in the field of computer vision.
1 code implementation • 7 Oct 2019 • Daniel Y. Fu, Will Crichton, James Hong, Xinwei Yao, Haotian Zhang, Anh Truong, Avanika Narayan, Maneesh Agrawala, Christopher Ré, Kayvon Fatahalian
Many real-world video analysis applications require the ability to identify domain-specific events in video, such as interviews and commercials in TV news broadcasts, or action sequences in film.
1 code implementation • ICCV 2019 • Haotian Zhang, Long Mai, Ning Xu, Zhaowen Wang, John Collomosse, Hailin Jin
We propose a novel video inpainting algorithm that simultaneously hallucinates missing appearance and motion (optical flow) information, building upon the recent 'Deep Image Prior' (DIP) that exploits convolutional network architectures to enforce plausible texture in static images.
2 code implementations • 26 Mar 2019 • Wei Yang, Haotian Zhang, Jimmy Lin
Following recent successes in applying BERT to question answering, we explore simple applications to ad hoc document retrieval.
Ranked #2 on Ad-Hoc Information Retrieval on TREC Robust04 (MAP metric)
1 code implementation • CVPR 2019 • Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkhouser, Matthias Nießner, Leonidas Guibas
We introduce, TextureNet, a neural network architecture designed to extract features from high-resolution signals associated with 3D surface meshes (e. g., color texture maps).
Ranked #22 on Semantic Segmentation on ScanNet
1 code implementation • 18 Nov 2018 • Gaoang Wang, Yizhou Wang, Haotian Zhang, Renshu Gu, Jenq-Neng Hwang
Multi-object tracking (MOT) is an important and practical task related to both surveillance systems and moving camera applications, such as autonomous driving and robotic vision.
Ranked #19 on Multi-Object Tracking on MOT16
no code implementations • 23 Mar 2018 • Haotian Zhang, Gordon V. Cormack, Maura R. Grossman, Mark D. Smucker
This study uses a novel simulation framework to evaluate whether the time and effort necessary to achieve high recall using active learning is reduced by presenting the reviewer with isolated sentences, as opposed to full documents, for relevance feedback.
no code implementations • 25 Jul 2017 • Royal Sequiera, Gaurav Baruah, Zhucheng Tu, Salman Mohammed, Jinfeng Rao, Haotian Zhang, Jimmy Lin
Most work on natural language question answering today focuses on answer selection: given a candidate list of sentences, determine which contains the answer.
no code implementations • 25 Jul 2017 • Jinfeng Rao, Hua He, Haotian Zhang, Ferhan Ture, Royal Sequiera, Salman Mohammed, Jimmy Lin
To our knowledge, we are the first to integrate lexical and temporal signals in an end-to-end neural network architecture, in which existing neural ranking models are used to generate query-document similarity vectors that feed into a bidirectional LSTM layer for temporal modeling.