no code implementations • 16 May 2024 • Zheng Gu, Shiyuan Yang, Jing Liao, Jing Huo, Yang Gao
For visual prompting, we propose a self-attention cloning (SAC) method to guide the fine-grained structural-level analogy between image examples.
no code implementations • 16 May 2024 • Peiying Zhang, Nanxuan Zhao, Jing Liao
By optimizing the combination of neural paths, we can incorporate geometric constraints while preserving expressivity in generated SVGs.
no code implementations • 19 Apr 2024 • Yixiang Zhuang, Baoping Cheng, Yao Cheng, Yuntao Jin, Renshuai Liu, Chengyang Li, Xuan Cheng, Jing Liao, Juncong Lin
Speech-driven facial animation methods usually contain two main classes, 3D and 2D talking face, both of which attract considerable research attention in recent years.
no code implementations • 18 Apr 2024 • Ronghuan Wu, Wanchao Su, Kede Ma, Jing Liao
To generate cartoon-style and smooth motion, we first define B\'{e}zier curves over keypoints of the clipart image as a form of motion regularization.
no code implementations • 4 Apr 2024 • Zhongkai Wu, Ziyu Wan, Jing Zhang, Jing Liao, Dong Xu
Instead of reconstructing a blurred NeRF by averaging inconsistencies, we introduce a novel approach using Generative Adversarial Networks (GANs) for NeRF generation to better accommodate the geometric and appearance inconsistencies present in the multi-view images.
no code implementations • 5 Feb 2024 • Shiyuan Yang, Liang Hou, Haibin Huang, Chongyang Ma, Pengfei Wan, Di Zhang, Xiaodong Chen, Jing Liao
In practice, users often desire the ability to control object motion and camera movement independently for customized video creation.
no code implementations • 31 Jan 2024 • Xiaoyu Li, Qi Zhang, Di Kang, Weihao Cheng, Yiming Gao, Jingbo Zhang, Zhihao Liang, Jing Liao, Yan-Pei Cao, Ying Shan
In this survey, we aim to introduce the fundamental methodologies of 3D generation methods and establish a structured roadmap, encompassing 3D representation, generation methods, datasets, and corresponding applications.
no code implementations • 22 Dec 2023 • Wanchao Su, Can Wang, Chen Liu, Hangzhou Han, Hongbo Fu, Jing Liao
To address such issues, we present StyleRetoucher, a novel automatic portrait image retouching framework, leveraging StyleGAN's generation and generalization ability to improve an input portrait image's skin condition while preserving its facial details.
no code implementations • 12 Dec 2023 • Hongyu Liu, Xuan Wang, Ziyu Wan, Yujun Shen, Yibing Song, Jing Liao, Qifeng Chen
The noisy image, landmarks, and text condition are then fed into the frozen ControlNet twice for noise prediction.
no code implementations • 11 Dec 2023 • Ziyu Wan, Despoina Paschalidou, IAn Huang, Hongyu Liu, Bokui Shen, Xiaoyu Xiang, Jing Liao, Leonidas Guibas
The increased demand for 3D data in AR/VR, robotics and gaming applications, gave rise to powerful generative pipelines capable of synthesizing high-quality 3D objects.
no code implementations • 4 Dec 2023 • Can Wang, Mingming He, Menglei Chai, Dongdong Chen, Jing Liao
We first introduce a differentiable method using marching tetrahedra for polygonal mesh extraction from the neural implicit field and then design a differentiable color extractor to assign colors obtained from the volume renderings to this extracted mesh.
no code implementations • 28 Nov 2023 • Jingbo Zhang, Xiaoyu Li, Qi Zhang, YanPei Cao, Ying Shan, Jing Liao
Optimization-based methods that lift text-to-image diffusion models to 3D generation often fail to preserve the texture details of the reference image, resulting in inconsistent appearances in different views.
no code implementations • 18 Oct 2023 • Hongliang Zhong, Jingbo Zhang, Jing Liao
By discretizing the materials, our model can reduce noise in the decomposition process and generate a segmentation map of discrete materials.
1 code implementation • ICCV 2023 • Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Weiming Zhang, Gang Hua, Nenghai Yu
Even though they can enable very fine-grained local control, such interaction modes are inefficient for the editing conditions that can be easily specified by language descriptions or reference images.
1 code implementation • 11 Oct 2023 • Shiyuan Yang, Xiaodong Chen, Jing Liao
Recently, text-to-image denoising diffusion probabilistic models (DDPMs) have demonstrated impressive image generation capabilities and have also been successfully applied to image inpainting.
no code implementations • 21 Sep 2023 • Peiying Zhang, Nanxuan Zhao, Jing Liao
In this paper, we propose a novel pipeline that generates high-quality customized vector graphics based on textual prompts while preserving the properties and layer-wise information of a given exemplar SVG.
1 code implementation • ICCV 2023 • Qihua Dong, Hao Du, Ying Song, Yan Xu, Jing Liao
Our approach balances image similarity and volume preservation in different regions, i. e., normal and tumor regions, by using soft tumor masks to adjust the imposition of volume-preserving loss on each one.
no code implementations • 22 Jun 2023 • Zhiyuan Zhang, Zhitong Huang, Jing Liao
However, none of these methods have been able to edit the layout of single existing images.
1 code implementation • 19 May 2023 • Jingbo Zhang, Xiaoyu Li, Ziyu Wan, Can Wang, Jing Liao
Extensive experiments demonstrate that our Text2NeRF outperforms existing methods in producing photo-realistic, multi-view consistent, and diverse 3D scenes from a variety of natural language prompts.
no code implementations • 27 Apr 2023 • Ronghuan Wu, Wanchao Su, Kede Ma, Jing Liao
More importantly, we demonstrate the flexibility of IconShop with multiple novel icon synthesis tasks, including icon editing, icon interpolation, icon semantic combination, and icon design auto-suggestion.
no code implementations • CVPR 2023 • Ziyu Wan, Christian Richardt, Aljaž Božič, Chao Li, Vijay Rengarajan, Seonghyeon Nam, Xiaoyu Xiang, Tuotuo Li, Bo Zhu, Rakesh Ranjan, Jing Liao
Neural radiance fields (NeRFs) enable novel view synthesis with unprecedented visual quality.
1 code implementation • ICCV 2023 • Ruixiang Jiang, Can Wang, Jingbo Zhang, Menglei Chai, Mingming He, Dongdong Chen, Jing Liao
Neural implicit fields are powerful for representing 3D scenes and generating high-quality novel views, but it remains challenging to use such implicit representations for creating a 3D human avatar with a specific identity and artistic style that can be easily animated.
no code implementations • CVPR 2023 • Li Ma, Xiaoyu Li, Jing Liao, Pedro V. Sander
Looping videos are short video clips that can be looped endlessly without visible seams or artifacts.
no code implementations • 4 Feb 2023 • Hao Du, Qihua Dong, Yan Xu, Jing Liao
Furthermore, we propose contrastive similarity to encourage organ pixels to gather around in the contrastive embedding space, which helps better distinguish low-contrast tissues.
1 code implementation • 15 Dec 2022 • Can Wang, Ruixiang Jiang, Menglei Chai, Mingming He, Dongdong Chen, Jing Liao
As a powerful representation of 3D scenes, the neural radiance field (NeRF) enables high-quality novel view synthesis from multi-view images.
1 code implementation • 6 Dec 2022 • Shuquan Ye, Dongdong Chen, Songfang Han, Jing Liao
To handle boundary-level label noise, we also propose a variant ``PNAL-boundary " with a progressive boundary label cleaning strategy.
1 code implementation • CVPR 2023 • Shuquan Ye, Yujia Xie, Dongdong Chen, Yichong Xu, Lu Yuan, Chenguang Zhu, Jing Liao
Through our analysis, we find one important reason is that existing large-scale VL datasets do not contain much commonsense knowledge, which motivates us to improve the commonsense of VL-models from the data perspective.
no code implementations • 5 Oct 2022 • Ryusuke Sugimoto, Mingming He, Jing Liao, Pedro V. Sander
We propose an approach to simulate and render realistic water animation from a single still input photograph.
no code implementations • 22 Sep 2022 • Zhitong Huang, Nanxuan Zhao, Jing Liao
In the first stage, multi-modal conditions are converted into a common representation of hint points.
1 code implementation • 19 Sep 2022 • Chufeng Xiao, Wanchao Su, Jing Liao, Zhouhui Lian, Yi-Zhe Song, Hongbo Fu
We invited 70 novice users and 38 expert users to sketch 136 3D objects, which were presented as 362 images rendered from multiple views.
no code implementations • 15 Aug 2022 • Jingbo Zhang, Ziyu Wan, Jing Liao
Due to inevitable noises introduced during scanning and quantization, 3D reconstruction via RGB-D sensors suffers from errors both in geometry and texture, leading to artifacts such as camera drifting, mesh distortion, texture ghosting, and blurriness.
1 code implementation • 11 Aug 2022 • Jingbo Zhang, Xiaoyu Li, Ziyu Wan, Can Wang, Jing Liao
Unlike existing dynamic NeRFs that require dense images as input and can only be modeled for a single identity, our method enables face reconstruction across different persons with few-shot inputs.
no code implementations • 1 Jul 2022 • Li Ma, Xiaoyu Li, Jing Liao, Xuan Wang, Qi Zhang, Jue Wang, Pedro Sander
Implicit radiance functions emerged as a powerful scene representation for reconstructing and rendering photo-realistic views of a 3D scene.
1 code implementation • CVPR 2022 • Ziyu Wan, Bo Zhang, Dongdong Chen, Jing Liao
We present a learning-based framework, recurrent transformer network (RTN), to restore heavily degraded old films.
Ranked #6 on Analog Video Restoration on TAPE
no code implementations • 15 Dec 2021 • Shuquan Ye, Dongdong Chen, Songfang Han, Jing Liao
To this end, we propose a novel transformer-based 3DQA framework "3DQA-TR", which consists of two encoders for exploiting the appearance and geometry information, respectively.
2 code implementations • CVPR 2022 • Can Wang, Menglei Chai, Mingming He, Dongdong Chen, Jing Liao
Furthermore, we propose an inverse optimization method that accurately projects an input image to the latent codes for manipulation to enable editing on real images.
1 code implementation • CVPR 2022 • Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Zhentao Tan, Lu Yuan, Weiming Zhang, Nenghai Yu
Hair editing is an interesting and challenging problem in computer vision and graphics.
1 code implementation • CVPR 2022 • Li Ma, Xiaoyu Li, Jing Liao, Qi Zhang, Xuan Wang, Jue Wang, Pedro V. Sander
We demonstrate that our method can be used on both camera motion blur and defocus blur: the two most common types of blur in real scenes.
1 code implementation • 5 Aug 2021 • Jie Zhang, Dongdong Chen, Qidong Huang, Jing Liao, Weiming Zhang, Huamin Feng, Gang Hua, Nenghai Yu
As the image structure can keep its semantic meaning during the data transformation, such trigger pattern is inherently robust to data transformations.
no code implementations • 5 Aug 2021 • Jie Zhang, Dongdong Chen, Jing Liao, Han Fang, Zehua Ma, Weiming Zhang, Gang Hua, Nenghai Yu
However, little attention has been devoted to the protection of DNNs in image processing tasks.
no code implementations • 4 Aug 2021 • Fangzhou Han, Can Wang, Hao Du, Jing Liao
To address this, we present a novel deep learning framework for portrait lighting enhancement based on 3D facial guidance.
1 code implementation • ICCV 2021 • Shuquan Ye, Dongdong Chen, Songfang Han, Jing Liao
Point cloud segmentation is a fundamental task in 3D.
1 code implementation • CVPR 2021 • Hongyu Liu, Ziyu Wan, Wei Huang, Yibing Song, Xintong Han, Jing Liao
To this end, we propose spatially probabilistic diversity normalization (SPDNorm) inside the modulation to model the probability of generating a pixel conditioned on the context information.
no code implementations • 29 Apr 2021 • Fangzhou Han, Shuquan Ye, Mingming He, Menglei Chai, Jing Liao
In the second texture style transfer stage, we focus on performing style transfer on the canonical texture by adopting a differentiable renderer to optimize the texture in a multi-view framework.
1 code implementation • 22 Apr 2021 • Can Wang, Menglei Chai, Mingming He, Dongdong Chen, Jing Liao
Face image manipulation via three-dimensional guidance has been widely applied in various interactive scenarios due to its semantically-meaningful understanding and user-friendly controllability.
no code implementations • ICCV 2021 • Xiaoyu Li, Bo Zhang, Jing Liao, Pedro V. Sander
This new dataset and our novel framework lead to our method that is able to address different contaminants and outperforms competitive restoration approaches both qualitatively and quantitatively.
2 code implementations • 15 Apr 2021 • Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Weiming Zhang, Lu Yuan, Gang Hua, Nenghai Yu
This paper studies the problem of StyleGAN inversion, which plays an essential role in enabling the pretrained StyleGAN to be used for real image editing tasks.
4 code implementations • ICCV 2021 • Ziyu Wan, Jingbo Zhang, Dongdong Chen, Jing Liao
Image completion has made tremendous progress with convolutional neural networks (CNNs), because of their powerful texture modeling capacity.
Ranked #6 on Image Inpainting on CelebA-HQ
1 code implementation • CVPR 2021 • Hongyu Liu, Ziyu Wan, Wei Huang, Yibing Song, Xintong Han, Jing Liao, Bing Jiang, Wei Liu
While existing methods combine an input image and these low-level controls for CNN inputs, the corresponding feature representations are not sufficient to convey user intentions, leading to unfaithfully generated content.
1 code implementation • CVPR 2021 • Zhentao Tan, Menglei Chai, Dongdong Chen, Jing Liao, Qi Chu, Bin Liu, Gang Hua, Nenghai Yu
In this paper, we propose a novel diverse semantic image synthesis framework from the perspective of semantic class distributions, which naturally supports diverse generation at semantic or even instance level.
Ranked #1 on Image-to-Image Translation on ADE20K Labels-to-Photos (LPIPS metric)
1 code implementation • 8 Mar 2021 • Jie Zhang, Dongdong Chen, Jing Liao, Weiming Zhang, Huamin Feng, Gang Hua, Nenghai Yu
By jointly training the target model and watermark embedding, the extra barrier can even be absorbed into the target model.
1 code implementation • 8 Feb 2021 • Shuquan Ye, Dongdong Chen, Songfang Han, Ziyu Wan, Jing Liao
Thus, Meta-PU even outperforms the existing methods trained for a specific scale factor only.
Graphics
no code implementations • CVPR 2021 • Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Hanqing Zhao, Weiming Zhang, Nenghai Yu
Image matting is a fundamental and challenging problem in computer vision and graphics.
1 code implementation • 8 Dec 2020 • Zhentao Tan, Dongdong Chen, Qi Chu, Menglei Chai, Jing Liao, Mingming He, Lu Yuan, Gang Hua, Nenghai Yu
Spatially-adaptive normalization (SPADE) is remarkably successful recently in conditional semantic image synthesis \cite{park2019semantic}, which modulates the normalized activation with spatially-varying transformations learned from semantic layouts, to prevent the semantic information from being washed away.
no code implementations • 1 Nov 2020 • Hang Zhou, Dongdong Chen, Jing Liao, Weiming Zhang, Kejiang Chen, Xiaoyi Dong, Kunlin Liu, Gang Hua, Nenghai Yu
To overcome these shortcomings, this paper proposes a novel label guided adversarial network (LG-GAN) for real-time flexible targeted point cloud attack.
1 code implementation • 30 Oct 2020 • Zhentao Tan, Menglei Chai, Dongdong Chen, Jing Liao, Qi Chu, Lu Yuan, Sergey Tulyakov, Nenghai Yu
In this paper, we present MichiGAN (Multi-Input-Conditioned Hair Image GAN), a novel conditional image generation method for interactive portrait hair manipulation.
2 code implementations • NeurIPS 2020 • Houwen Peng, Hao Du, Hongyuan Yu, Qi Li, Jing Liao, Jianlong Fu
The experiments on ImageNet verify such path distillation method can improve the convergence ratio and performance of the hypernetwork, as well as boosting the training of subnetworks.
1 code implementation • NeurIPS 2020 • Jie Zhang, Dongdong Chen, Jing Liao, Weiming Zhang, Gang Hua, Nenghai Yu
Only when the model IP is suspected to be stolen by someone, the private passport-aware branch is added back for ownership verification.
1 code implementation • 12 Oct 2020 • Jialu Huang, Jing Liao, Sam Kwong
We proposed a new I2I translation method that generates a new model in the target domain via a series of model transformations on a pre-trained StyleGAN2 model in the source domain.
8 code implementations • 14 Sep 2020 • Zi-Yu Wan, Bo Zhang, Dong-Dong Chen, Pan Zhang, Dong Chen, Jing Liao, Fang Wen
Unlike conventional restoration tasks that can be solved through supervised learning, the degradation in real photos is complex and the domain gap between synthetic images and real old photos makes the network fail to generalize.
1 code implementation • 10 Aug 2020 • Xiaoyu Li, Bo Zhang, Jing Liao, Pedro V. Sander
The key idea of the proposed approach is to estimate the dense cross-domain correspondence between the sketch and cartoon video frames, and employ a blending module with occlusion estimation to synthesize the middle frame guided by the sketch.
no code implementations • 18 Jun 2020 • Jialu Huang, Jing Liao, Zhifeng Tan, Sam Kwong
Sketch-to-image (S2I) translation plays an important role in image synthesis and manipulation tasks, such as photo editing and colorization.
7 code implementations • CVPR 2020 • Zi-Yu Wan, Bo Zhang, Dong-Dong Chen, Pan Zhang, Dong Chen, Jing Liao, Fang Wen
Unlike conventional restoration tasks that can be solved through supervised learning, the degradation in real photos is complex and the domain gap between synthetic images and real old photos makes the network fail to generalize.
no code implementations • 6 Apr 2020 • Zhentao Tan, Dongdong Chen, Qi Chu, Menglei Chai, Jing Liao, Mingming He, Lu Yuan, Nenghai Yu
Despite its impressive performance, a more thorough understanding of the true advantages inside the box is still highly demanded, to help reduce the significant computation and parameter overheads introduced by these new structures.
no code implementations • 1 Apr 2020 • Rui Nie, Huan Yang, Hejuan Peng, Wenbin Luo, Weiya Fan, Jie Zhang, Jing Liao, Fang Huang, Yufeng Xiao
Small intestinal capsule endoscopy is the mainstream method for inspecting small intestinal lesions, but a single small intestinal capsule endoscopy will produce 60, 000 - 120, 000 images, the majority of which are similar and have no diagnostic value.
1 code implementation • 25 Feb 2020 • Jie Zhang, Dong-Dong Chen, Jing Liao, Han Fang, Weiming Zhang, Wenbo Zhou, HAO CUI, Nenghai Yu
In this way, when the attacker trains one surrogate model by using the input-output pairs of the target model, the hidden watermark will be learned and extracted afterward.
1 code implementation • 29 Oct 2019 • Zixuan Huang, Jinghuai Zhang, Jing Liao
Recent neural style transfer frameworks have obtained astonishing visual quality and flexibility in Single-style Transfer (SST), but little attention has been paid to Multi-style Transfer (MST) which refers to simultaneously transferring multiple styles to the same image.
no code implementations • 28 Sep 2019 • Jialu Huang, Jing Liao, Tak Wu Sam Kwong
Given a reference image and an input in another domain, a semantic matching is first performed between the two visual contents and generates the auxiliary image, which is explicitly encouraged to preserve semantic characteristics of the reference.
1 code implementation • 20 Sep 2019 • Xiaoyu Li, Bo Zhang, Jing Liao, Pedro V. Sander
We propose a novel learning method to rectify document images with various distortion types from a single input image.
1 code implementation • CVPR 2019 • Xiaoyu Li, Bo Zhang, Pedro V. Sander, Jing Liao
We propose the first general framework to automatically correct different types of geometric distortion in a single input image.
1 code implementation • CVPR 2019 • Bo Zhang, Mingming He, Jing Liao, Pedro V. Sander, Lu Yuan, Amine Bermak, Dong Chen
This paper presents the first end-to-end network for exemplar-based video colorization.
1 code implementation • NeurIPS 2019 • Zi-Yu Wan, Dong-Dong Chen, Yan Li, Xingguang Yan, Junge Zhang, Yizhou Yu, Jing Liao
Based on the observation that visual features of test instances can be separated into different clusters, we propose a new visual structure constraint on class centers for transductive ZSL, to improve the generality of the projection function (i. e. alleviate the above domain shift problem).
1 code implementation • 21 Nov 2018 • Dongdong Chen, Mingming He, Qingnan Fan, Jing Liao, Liheng Zhang, Dongdong Hou, Lu Yuan, Gang Hua
Image dehazing aims to recover the uncorrupted content from a hazy image.
Ranked #1 on Rain Removal on DID-MDN
no code implementations • 1 Nov 2018 • Kaidi Cao, Jing Liao, Lu Yuan
Facial caricature is an art form of drawing faces in an exaggerated way to convey humor or sarcasm.
no code implementations • 21 Aug 2018 • Kfir Aberman, Mingyi Shi, Jing Liao, Dani Lischinski, Baoquan Chen, Daniel Cohen-Or
After training a deep generative network using a reference video capturing the appearance and dynamics of a target actor, we are able to generate videos where this actor reenacts other performances.
1 code implementation • 17 Jul 2018 • Mingming He, Dong-Dong Chen, Jing Liao, Pedro V. Sander, Lu Yuan
More importantly, as opposed to other learning-based colorization methods, our network allows the user to achieve customizable results by simply feeding different references.
1 code implementation • CVPR 2018 • Shuyang Gu, Congliang Chen, Jing Liao, Lu Yuan
We theoretically prove that our new style loss based on reshuffle connects both global and local style losses respectively used by most parametric and non-parametric neural style transfer methods.
2 code implementations • 10 May 2018 • Kfir Aberman, Jing Liao, Mingyi Shi, Dani Lischinski, Baoquan Chen, Daniel Cohen-Or
Correspondence between images is a fundamental problem in computer vision, with a variety of graphics applications.
no code implementations • CVPR 2018 • Dongdong Chen, Lu Yuan, Jing Liao, Nenghai Yu, Gang Hua
This paper presents the first attempt at stereoscopic neural style transfer, which responds to the emerging demand for 3D movies or AR/VR.
3 code implementations • 2 Oct 2017 • Mingming He, Jing Liao, Dong-Dong Chen, Lu Yuan, Pedro V. Sander
The proposed method can be successfully extended from one-to-one to one-to-many color transfer.
5 code implementations • 2 May 2017 • Jing Liao, Yuan YAO, Lu Yuan, Gang Hua, Sing Bing Kang
We propose a new technique for visual attribute transfer across images that may have very different appearance but have perceptually similar semantic structure.
1 code implementation • CVPR 2017 • Dongdong Chen, Lu Yuan, Jing Liao, Nenghai Yu, Gang Hua
It also enables us to conduct incremental learning to add a new image style by learning a new filter bank while holding the auto-encoder fixed.
no code implementations • ICCV 2017 • Dongdong Chen, Jing Liao, Lu Yuan, Nenghai Yu, Gang Hua
Training a feed-forward network for fast neural style transfer of images is proven to be successful.