Xinhang Liu

I am a third-year PhD student at HKUST, advised by Chi-Keung Tang and Yu-Wing Tai. I graduated from ShanghaiTech University with Bachelor's degree in 2022, where I was lucky to join the group of Jingyi Yu. Outside of academics, I am a basketball enthusiast.

Email  /  Resume  /  GitHub  /  Google Scholar  

profile photo

Research
project image

ChatCam: Empowering Camera Control through Conversational AI


Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang
NeurIPS, 2024
project page / arXiv / code

We introduce ChatCam, a system that navigates camera movements through conversations with users, mimicking a professional cinematographer's workflow.

project image

Deceptive-NeRF/3DGS: Diffusion-Generated Pseudo-Observations for High-Quality Sparse-View Reconstruction


Xinhang Liu, Jiaben Chen, Shiu-hong Kao, Yu-Wing Tai, Chi-Keung Tang
ECCV, 2024 (Also presented at ECCV'24 Wild3D Workshop)
project page / arXiv / code

We enhance sparse-view reconstruction by leveraging a diffusion model pre-trained from multiview datasets to synthesize pseudo-observations.

project image

CryoFormer: Continuous Heterogeneous Cryo-EM Reconstruction using Transformer-based Neural Representations


Xinhang Liu*, Yan Zeng*, Yifan Qin, Hao Li, Jiakai Zhang, Lan Xu, Jingyi Yu
ECCV'24 NFBCC Workshop (Highlight)
project page / arXiv

With hundreds of thousands of noisy images as inputs, our method can reconstruct the heterogeneous structures of proteins and highlight the local, flexible motions.

project image

Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling


Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang, Pedro Miraldo, Suhas Lohit, Moitreya Chatterjee
CVPR, 2024 (Highlight, 2.8% of 11532)
project page / arXiv / video / code

We learn a 4D semantic embedding and introduce the concept of gears for stratified modeling of dynamic regions achieving photo-realistic dynamic novel view synthesis and free-viewpoint tracking.

project image

Unsupervised Multi-View Object Segmentation Using Radiance Field Propagation


Xinhang Liu, Jiaben Chen, Huai Yu, Yu-Wing Tai, Chi-Keung Tang
NeurIPS, 2022
project page / arXiv / code / data

We study segmenting objects in 3D during reconstruction given only unlabeled multi-view images of a scene, from an information-theoretic perspective.

project image

Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time


Liao Wang*, Jiakai Zhang*, Xinhang Liu, Fuqiang Zhao, Yanshun Zhang, Yingliang Zhang, Minye Wu Lan Xu, Jingyi Yu
CVPR, 2022
project page / arXiv

A novel technique to tackle efficient neural modeling and real-time rendering of dynamic scenes captured under the free-view video (FVV) setting.

project image

Editable Free-Viewpoint Video using a Layered Neural Representation


Jiakai Zhang, Xinhang Liu, Xinyi Ye, Fuqiang Zhao, Yanshun Zhang, Minye Wu, Yingliang Zhang, Lan Xu, Jingyi Yu
SIGRRAPH, 2021
project page / arXiv / code / Two Minute Papers

The first approach for editable photo-realistic free-viewpoint video generation for large-scale dynamic scenes using only sparse 16 cameras.

Experience

  • Mitsubishi Electric Research Laboratories (MERL), Cambridge, MA: Research Intern (2023 summer).
  • Carnegie Mellon University, Pittsburgh, PA: Robotics Institute Summer Scholar (2021 summer).
    		•

    Service & Teaching

  • Conference Reviewer of CVPR, ICCV, ECCV, ICML, NeurIPS and ICLR.
  • COMP2611 - Computer Organization (24S), Teaching Assistant.
  • COMP5421 - Computer Vision (23S), Teaching Assistant.
  • CS182 - Introduction to Machine Learning (21F), Teaching Assistant.
  • SI100B - Introduction to Information Science and Technology (21F), Teaching Assistant.
  • CS276 - Computational Photography (21F), Guest Lecture in 'NeRF'.
    		•

    Design and source code modified based on Jon Barron's website

    Last update: Feb, 2024