2 code implementations • 8 Apr 2024 • Yurong You, Cheng Perng Phoo, Carlos Andres Diaz-Ruiz, Katie Z Luo, Wei-Lun Chao, Mark Campbell, Bharath Hariharan, Kilian Q Weinberger
Accurate 3D object detection is crucial to autonomous driving.
no code implementations • 6 Feb 2024 • Jihyung Kil, Chan Hee Song, Boyuan Zheng, Xiang Deng, Yu Su, Wei-Lun Chao
Automatic web navigation aims to build a web agent that can follow language instructions to execute complex and diverse tasks on real-world websites.
no code implementations • 31 Dec 2023 • Vardaan Pahuja, Weidi Luo, Yu Gu, Cheng-Hao Tu, Hong-You Chen, Tanya Berger-Wolf, Charles Stewart, Song Gao, Wei-Lun Chao, Yu Su
In this work, we leverage the structured context associated with the camera trap images to improve out-of-distribution generalization for the task of species identification in camera traps.
1 code implementation • 30 Nov 2023 • Samuel Stevens, Jiaman Wu, Matthew J Thompson, Elizabeth G Campolongo, Chan Hee Song, David Edward Carlyn, Li Dong, Wasila M Dahdul, Charles Stewart, Tanya Berger-Wolf, Wei-Lun Chao, Yu Su
We then develop BioCLIP, a foundation model for the tree of life, leveraging the unique properties of biology captured by TreeOfLife-10M, namely the abundance and variety of images of plants, animals, and fungi, together with the availability of rich structured biological knowledge.
1 code implementation • 7 Nov 2023 • Dipanjyoti Paul, Arpita Chowdhury, Xinqi Xiong, Feng-Ju Chang, David Carlyn, Samuel Stevens, Kaiya L. Provost, Anuj Karpatne, Bryan Carstens, Daniel Rubenstein, Charles Stewart, Tanya Berger-Wolf, Yu Su, Wei-Lun Chao
We present a novel usage of Transformers to make image classification interpretable.
1 code implementation • 23 Oct 2023 • Tai-Yu Pan, Chenyang Ma, Tianle Chen, Cheng Perng Phoo, Katie Z Luo, Yurong You, Mark Campbell, Kilian Q. Weinberger, Bharath Hariharan, Wei-Lun Chao
Accurate 3D object detection and understanding for self-driving cars heavily relies on LiDAR point clouds, necessitating large amounts of labeled data to train.
1 code implementation • 20 Oct 2023 • Yuxiao Qu, Jinmeng Rao, Song Gao, Qianheng Zhang, Wei-Lun Chao, Yu Su, Michelle Miller, Alfonso Morales, Patrick Huber
This paper proposes FLEE-GNN, a novel Federated Learning System for Edge-Enhanced Graph Neural Network, designed to overcome these challenges and enhance the analysis of geospatial resilience of multicommodity food flow network, which is one type of spatial networks.
no code implementations • 21 Sep 2023 • Travis Zhang, Katie Luo, Cheng Perng Phoo, Yurong You, Wei-Lun Chao, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
Additionally, we leverage the statistics for a novel self-training process to stabilize the training.
1 code implementation • 5 Jun 2023 • Mohannad Elhamod, Mridul Khurana, Harish Babu Manogaran, Josef C. Uyeda, Meghan A. Balk, Wasila Dahdul, Yasin Bakış, Henry L. Bart Jr., Paula M. Mabee, Hilmar Lapp, James P. Balhoff, Caleb Charpentier, David Carlyn, Wei-Lun Chao, Charles V. Stewart, Daniel I. Rubenstein, Tanya Berger-Wolf, Anuj Karpatne
Discovering evolutionary traits that are heritable across species on the tree of life (also referred to as a phylogenetic tree) is of great interest to biologists to understand how organisms diversify and evolve.
1 code implementation • CVPR 2023 • Tai-Yu Pan, Qing Liu, Wei-Lun Chao, Brian Price
Second, we introduce a novel approach to improve part segmentation on unseen objects, inspired by an interesting finding -- for unseen objects, the pixel-wise features extracted by the model often reveal high-quality part segments.
1 code implementation • 9 May 2023 • Tianle Chen, Zheda Mai, Ruiwen Li, Wei-Lun Chao
Weakly supervised semantic segmentation (WSSS) aims to bypass the need for laborious pixel-level annotation by using only image-level annotation.
no code implementations • 16 Apr 2023 • Hong-You Chen, Jike Zhong, Mingda Zhang, Xuhui Jia, Hang Qi, Boqing Gong, Wei-Lun Chao, Li Zhang
FedBasis learns a set of few shareable ``basis'' models, which can be linearly combined to form personalized models for clients.
no code implementations • ICCV 2023 • Reza Averly, Wei-Lun Chao
We show that this framework unifies the detection of OOD examples caused by semantic shift and covariate shift, and closely addresses the concern of applying a machine learning model to uncontrolled environments.
Out-of-Distribution Detection Out of Distribution (OOD) Detection
1 code implementation • 27 Mar 2023 • Yurong You, Cheng Perng Phoo, Katie Z Luo, Travis Zhang, Wei-Lun Chao, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
For a self-driving car to operate reliably, its perceptual system must generalize to the end-user's environment -- ideally without additional annotation efforts.
1 code implementation • 14 Mar 2023 • Cheng-Hao Tu, Hong-You Chen, David Carlyn, Wei-Lun Chao
Fractals are geometric shapes that can display complex and self-similar patterns found in nature (e. g., clouds and plants).
no code implementations • 12 Mar 2023 • Jike Zhong, Hong-You Chen, Wei-Lun Chao
We reinvestigate factors that are believed to cause this problem, including the mismatch of BN statistics across clients and the deviation of gradients during local training.
no code implementations • CVPR 2023 • Hong-You Chen, Yandong Li, Yin Cui, Mingda Zhang, Wei-Lun Chao, Li Zhang
We study the problem of how to train a "personalization-friendly" model such that given only the task descriptions, the model can be adapted to different end-users' needs, e. g., for accurately classifying different subsets of objects.
no code implementations • ICCV 2023 • Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M. Sadler, Wei-Lun Chao, Yu Su
In this work, we propose a novel method, LLM-Planner, that harnesses the power of large language models to do few-shot planning for embodied agents.
1 code implementation • CVPR 2023 • Cheng-Hao Tu, Zheda Mai, Wei-Lun Chao
Through introducing a handful of learnable ``query'' tokens to each layer, VQT leverages the inner workings of Transformers to ``summarize'' rich intermediate features of each layer, which can then be used to train the prediction heads of downstream tasks.
no code implementations • 23 Sep 2022 • Youya Xia, Josephine Monica, Wei-Lun Chao, Bharath Hariharan, Kilian Q Weinberger, Mark Campbell
In this paper, we investigate the idea of turning sensor inputs (i. e., images) captured in an adverse condition into a benign one (i. e., sunny), upon which the downstream tasks (e. g., semantic segmentation) can attain high accuracy.
no code implementations • ICCV 2023 • Jihyung Kil, Soravit Changpinyo, Xi Chen, Hexiang Hu, Sebastian Goodman, Wei-Lun Chao, Radu Soricut
The ability to recognize and reason about text embedded in visual inputs is often lacking in vision-and-language (V&L) models, perhaps because V&L pre-training methods have often failed to include such an ability in their training objective.
no code implementations • CVPR 2022 • Carlos A. Diaz-Ruiz, Youya Xia, Yurong You, Jose Nino, Junan Chen, Josephine Monica, Xiangyu Chen, Katie Luo, Yan Wang, Marc Emond, Wei-Lun Chao, Bharath Hariharan, Kilian Q. Weinberger, Mark Campbell
Advances in perception for self-driving cars have accelerated in recent years due to the availability of large-scale datasets, typically collected at specific locations and under nice weather conditions.
1 code implementation • NeurIPS 2021 • Hong-You Chen, Wei-Lun Chao
This coarse domain sequence then undergoes a fine indexing step via a novel cycle-consistency loss, which encourages the next intermediate domain to preserve sufficient discriminative knowledge of the current intermediate domain.
1 code implementation • 23 Jun 2022 • Hong-You Chen, Cheng-Hao Tu, Ziwei Li, Han-Wei Shen, Wei-Lun Chao
To make our findings applicable to situations where pre-trained models are not directly available, we explore pre-training with synthetic data or even with clients' data in a decentralized manner, and found that they can already improve FL notably.
2 code implementations • CVPR 2022 • Yurong You, Katie Z Luo, Cheng Perng Phoo, Wei-Lun Chao, Wen Sun, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
Current 3D object detectors for autonomous driving are almost entirely trained on human-annotated data.
1 code implementation • ICLR 2022 • Yurong You, Katie Z Luo, Xiangyu Chen, Junan Chen, Wei-Lun Chao, Wen Sun, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
Self-driving cars must detect vehicles, pedestrians, and other traffic participants accurately to operate safely.
no code implementations • 22 Feb 2022 • Cheng Zhang, Tai-Yu Pan, Tianle Chen, Jike Zhong, WenJin Fu, Wei-Lun Chao
One fundamental challenge in building an instance segmentation model for a large number of classes in complex scenes is the lack of training examples, especially for rare objects.
1 code implementation • CVPR 2022 • Chan Hee Song, Jihyung Kil, Tai-Yu Pan, Brian M. Sadler, Wei-Lun Chao, Yu Su
We study the problem of developing autonomous agents that can follow human instructions to infer and perform a sequence of actions to complete the underlying task.
no code implementations • 24 Sep 2021 • Tai-Hsien Wu, Chunfeng Lian, Sanghee Lee, Matthew Pastewait, Christian Piers, Jie Liu, Fang Wang, Li Wang, Chiung-Ying Chiu, Wenchi Wang, Christina Jackson, Wei-Lun Chao, Dinggang Shen, Ching-Chang Ko
Our TS-MDL first adopts an end-to-end \emph{i}MeshSegNet method (i. e., a variant of the existing MeshSegNet with both improved accuracy and efficiency) to label each tooth on the downsampled scan.
1 code implementation • EMNLP 2021 • Jihyung Kil, Cheng Zhang, Dong Xuan, Wei-Lun Chao
We found that many of the "unknowns" to the learned VQA model are indeed "known" in the dataset implicitly.
1 code implementation • NeurIPS 2021 • Tai-Yu Pan, Cheng Zhang, Yandong Li, Hexiang Hu, Dong Xuan, Soravit Changpinyo, Boqing Gong, Wei-Lun Chao
We propose NorCal, Normalized Calibration for long-tailed object detection and instance segmentation, a simple and straightforward recipe that reweighs the predicted scores of each class by its training sample size.
3 code implementations • ICLR 2022 • Hong-You Chen, Wei-Lun Chao
On the one hand, we introduce a family of losses that are robust to non-identical class distributions, enabling clients to train a generic predictor with a consistent objective across them.
1 code implementation • 1 Jul 2021 • Han-Jia Ye, Lu Ming, De-Chuan Zhan, Wei-Lun Chao
Many existing works take the meta-learning approach, constructing a few-shot learner that can learn from few-shot examples to generate a classifier.
1 code implementation • ICLR 2022 • Han-Jia Ye, Wei-Lun Chao
We find that these permutations lead to a huge variance of accuracy, making MAML unstable in few-shot classification.
1 code implementation • NAACL 2021 • Jihyung Kil, Wei-Lun Chao
Zero-shot learning aims to recognize unseen objects using their semantic representations.
no code implementations • ICCV 2021 • Han-Jia Ye, De-Chuan Zhan, Wei-Lun Chao
To correct these wrong predictions, the neural network then must focus on pushing features of minor class data across the decision boundaries between major and minor classes, leading to much larger gradients for features of minor classes.
no code implementations • 26 Mar 2021 • Yurong You, Carlos Andres Diaz-Ruiz, Yan Wang, Wei-Lun Chao, Bharath Hariharan, Mark Campbell, Kilian Q Weinberger
Self-driving cars must detect other vehicles and pedestrians in 3D to plan safe routes and avoid collisions.
1 code implementation • ICCV 2021 • Cheng Zhang, Tai-Yu Pan, Yandong Li, Hexiang Hu, Dong Xuan, Soravit Changpinyo, Boqing Gong, Wei-Lun Chao
Many objects do not appear frequently enough in complex scenes (e. g., certain handbags in living rooms) for training an accurate object detector, but are often found frequently by themselves (e. g., in product images).
2 code implementations • ICLR 2021 • Hong-You Chen, Wei-Lun Chao
Federated learning aims to collaboratively train a strong global model by accessing users' locally trained models but not their own data.
1 code implementation • ICCV 2021 • Luyu Yang, Yan Wang, Mingfei Gao, Abhinav Shrivastava, Kilian Q. Weinberger, Wei-Lun Chao, Ser-Nam Lim
To integrate the strengths of the two classifiers, we apply the well-established co-training framework, in which the two classifiers exchange their high confident predictions to iteratively "teach each other" so that both classifiers can excel in the target domain.
Semi-supervised Domain Adaptation Unsupervised Domain Adaptation
1 code implementation • NeurIPS 2020 • Divyansh Garg, Yan Wang, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger, Wei-Lun Chao
Existing approaches to depth or disparity estimation output a distribution over a set of pre-defined discrete values.
Ranked #2 on Stereo Depth Estimation on KITTI2015 (three pixel error metric)
3D Object Detection From Stereo Images Autonomous Driving +5
1 code implementation • CVPR 2020 • Yan Wang, Xiangyu Chen, Yurong You, Li Erran, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger, Wei-Lun Chao
In the domain of autonomous driving, deep learning has substantially improved the 3D object detection accuracy for LiDAR and stereo camera data alike.
1 code implementation • CVPR 2020 • Rui Qian, Divyansh Garg, Yan Wang, Yurong You, Serge Belongie, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger, Wei-Lun Chao
Reliable and accurate 3D object detection is a necessity for safe autonomous driving.
1 code implementation • 19 Feb 2020 • Vikram Shree, Wei-Lun Chao, Mark Campbell
In this work, we consider the problem of searching people in an unconstrained environment, with natural language descriptions.
no code implementations • 3 Feb 2020 • Wei-Lun Chao, Han-Jia Ye, De-Chuan Zhan, Mark Campbell, Kilian Q. Weinberger
Recent years have witnessed an abundance of new publications and approaches on meta-learning.
1 code implementation • 25 Jan 2020 • Vikram Shree, Wei-Lun Chao, Mark Campbell
Person re-identification aims to identify a person from an image collection, given one image of that person as the query.
no code implementations • 10 Jan 2020 • Shih-Han Chou, Wei-Lun Chao, Wei-Sheng Lai, Min Sun, Ming-Hsuan Yang
We then study two different VQA models on VQA 360, including one conventional model that takes an equirectangular image (with intrinsic distortion) as input and one dedicated model that first projects a 360 image onto cubemaps and subsequently aggregates the information from multiple spatial resolutions.
1 code implementation • 6 Jan 2020 • Han-Jia Ye, Hong-You Chen, De-Chuan Zhan, Wei-Lun Chao
Classifiers trained with class-imbalanced data are known to perform poorly on test data of the "minor" classes, of which we have insufficient training data.
6 code implementations • 12 Nov 2019 • Yan Wang, Wei-Lun Chao, Kilian Q. Weinberger, Laurens van der Maaten
Few-shot learners aim to recognize new object classes based on a small number of labeled training examples.
1 code implementation • 30 Oct 2019 • Brian H. Wang, Wei-Lun Chao, Yan Wang, Bharath Hariharan, Kilian Q. Weinberger, Mark Campbell
We obtain 2-D segmentation predictions by applying Mask-RCNN to the RGB image, and then link this image to a 3-D lidar point cloud by building a graph of connections among 3-D points and 2-D pixels.
1 code implementation • NeurIPS 2019 • Tao Yu, Shengyuan Hu, Chuan Guo, Wei-Lun Chao, Kilian Q. Weinberger
Natural images are virtually surrounded by low-density misclassified regions that can be efficiently discovered by gradient-guided search --- enabling the generation of adversarial images.
no code implementations • 28 Jul 2019 • Cheng Zhang, Wei-Lun Chao, Dong Xuan
Specifically, we investigate the use of scene graphs derived from images for Visual QA: an image is abstractly represented by a graph with nodes corresponding to object entities and edges to object relationships.
1 code implementation • ICLR 2020 • Yurong You, Yan Wang, Wei-Lun Chao, Divyansh Garg, Geoff Pleiss, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
In this paper we provide substantial advances to the pseudo-LiDAR framework through improvements in stereo depth estimation.
3D Object Detection From Stereo Images Autonomous Driving +2
2 code implementations • CVPR 2019 • Yan Wang, Wei-Lun Chao, Divyansh Garg, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
However, in this paper we argue that it is not the quality of the data but its representation that accounts for the majority of the difference.
3D Object Detection From Stereo Images Autonomous Driving +2
1 code implementation • 16 Dec 2018 • Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, Fei Sha
Zero-shot learning (ZSL) enables solving a task without the need to see its examples.
no code implementations • CVPR 2018 • Wei-Lun Chao, Hexiang Hu, Fei Sha
Analogous to domain adaptation for visual recognition, this setting is appealing when the target dataset does not have a sufficient amount of labeled data to learn an "in-domain" model.
no code implementations • CVPR 2018 • Hexiang Hu, Wei-Lun Chao, Fei Sha
These properties make the approach particularly appealing for transfer learning for open-ended Visual QA, where the source dataset on which the model is learned has limited overlapping with the target dataset in the space of answers.
no code implementations • NAACL 2018 • Wei-Lun Chao, Hexiang Hu, Fei Sha
We apply the procedures to re-construct decoy answers for two popular Visual QA datasets as well as to create a new Visual QA dataset from the Visual Genome project, resulting in the largest dataset for this task.
1 code implementation • 26 May 2016 • Ke Zhang, Wei-Lun Chao, Fei Sha, Kristen Grauman
We propose a novel supervised learning technique for summarizing videos by automatically selecting keyframes or key subshots.
no code implementations • ICCV 2017 • Soravit Changpinyo, Wei-Lun Chao, Fei Sha
Leveraging class semantic descriptions and examples of known objects, zero-shot learning makes it possible to train a recognition model for an object class whose examples are not available.
1 code implementation • 13 May 2016 • Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, Fei Sha
Zero-shot learning (ZSL) methods have been studied in the unrealistic setting where test data are assumed to come from unseen classes only.
no code implementations • CVPR 2016 • Ke Zhang, Wei-Lun Chao, Fei Sha, Kristen Grauman
Video summarization has unprecedented importance to help us digest, browse, and search today's ever-growing video collections.
2 code implementations • CVPR 2016 • Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, Fei Sha
Given semantic descriptions of object classes, zero-shot learning aims to accurately recognize objects of the unseen classes, from which no examples are available at the training stage, by associating them to the seen classes, from which labeled examples are provided.
Ranked #1 on Few-Shot Image Classification on AWA - 0-Shot
no code implementations • NeurIPS 2014 • Boqing Gong, Wei-Lun Chao, Kristen Grauman, Fei Sha
Video summarization is a challenging problem with great application potential.
no code implementations • 6 Nov 2014 • Boqing Gong, Wei-Lun Chao, Kristen Grauman, Fei Sha
Extensive empirical studies validate our contributions, including applications on challenging document and video summarization, where flexibility in modeling the kernel matrix and balancing different errors is indispensable.