Fangfu Liu | 刘芳甫

I'm a second-year PhD student in the Department of Electronic Engineering at Tsinghua University , advised by Prof. Yueqi Duan. In 2023, I obtained my B.Eng. in the Department of Electronic Engineering, Tsinghua University.

My research interest lies in Machine Learning and 3D Computer Vision (e.g., 3D AIGC). I aim to build spatially intelligent AI that can model the world and reason about objects, places, and interactions in 3D space and time.

If you are interested in working with us (in person or remotely) as an intern at Tsinghua University, please feel free to drop me an email.

Email  /  CV  /  Google Scholar  /  Github  /  Twitter

profile photo
News

  • 2024-09: Two papers on 3D Vision are accepted by NeurIPS 2024.
  • 2024-07: Three papers on 3D AIGC are accepted by ECCV 2024.
  • 2024-02: One paper on 3D AIGC is accepted by CVPR 2024.
  • 2023-05: One paper on Structure Learning is accepted by KDD 2023.
  • 2023-02: One paper on NeRF is accepted by CVPR 2023.
  • 2023-01: One paper on Causal Discovery is accepted by ICLR 2023.
  • Preprints

    *Equal contribution    Project leader

    dise DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion
    Wenqiang Sun* , Shuo Chen* , Fangfu Liu*, Zilong Chen , Yueqi Duan , Jun Zhang , Yikai Wang
    Arxiv, 2024
    [arXiv] [Code] [Project Page]

    In this paper, we introduce DimensionX, a framework designed to generate photorealistic 3D and 4D scenes from just a single image with video diffusion. We believe that our research provides a promising direction to create a dynamic and interactive environment with video diffusion models.

    dise ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model
    Fangfu Liu*, Wenqiang Sun* , Hanyang Wang* , Yikai Wang , Haowen Sun , Junliang Ye ,
    Jun Zhang , Yueqi Duan
    Arxiv, 2024
    [arXiv] [Code] [Project Page]

    In this paper, we propose ReconX, a novel 3D scene reconstruction paradigm that reframes the ambiguous reconstruction challenge as a temporal generation task. The key insight is to unleash the strong generative prior of large pre-trained video diffusion models for sparse-view reconstruction.

    dise DreamCinema: Cinematic Transfer with Free Camera and 3D Character
    Weiliang Chen, Fangfu Liu, Diankun Wu, Haowen Sun, Haixu Song, Yueqi Duan
    Arxiv, 2024
    [arXiv] [Code] [Project Page]

    In this paper, we propose DreamCinema, a novel cinematic transfer framework that pioneers generative AI into the film production paradigm, aiming at facilitating user-friendly film creation.

    dise Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion
    Fangfu Liu*, Hanyang Wang* Shunyu Yao , Shengjun Zhang, Jie Zhou, Yueqi Duan
    Arxiv, 2024
    [arXiv] [Code] [Project Page]

    In this paper, we propose Physics3D, a novel method for learning various physical properties of 3D objects through a video diffusion model. Our approach involves designing a highly generalizable physical simulation system based on a viscoelastic material model, which enables us to simulate a wide range of materials with high-fidelity capabilities.

    Selected Publications

    *Equal contribution    Project leader

    dise Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image
    Kailu Wu , Fangfu Liu, Zhihan Cai, Runjie Yan, Hanyang Wang, Yating Hu,
    Yueqi Duan , Kaisheng Ma
    Conference on Neural Information Processing Systems (NeurIPS), 2024
    [arXiv] [Code] [Project Page]

    In this work, we introduce Unique3D, a novel image-to-3D framework for efficiently generating high-quality 3D meshes from single-view images, featuring state-of-the-art generation fidelity and strong generalizability. Unique3D can generate a high-fidelity textured mesh from a single orthogonal RGB image of any object in under 30 seconds.

    dise DreamReward: Text-to-3D Generation with Human Preference
    Junliang Ye* , Fangfu Liu*, Qixiu Li, Zhengyi Wang , Yikai Wang ,
    Xinzhou Wang, Yueqi Duan , Jun Zhu
    European Conference on Computer Vision (ECCV), 2024
    [arXiv] [Code] [Project Page]

    In this work, We propose the first general-purpose human preference reward model for text-to-3D generation, named Reward3D. Then we further introduce a novel text-to-3D framework, coined DreamReward, which greatly boosts high-text alignment and high-quality 3D generation through human preference feedback.

    dise Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generation
    Fangfu Liu, Hanyang Wang, Weiliang Chen, Haowen Sun, Yueqi Duan
    European Conference on Computer Vision (ECCV), 2024
    [arXiv] [Code] [Project Page]

    We introduce a novel 3D customization method, dubbed Make-Your-3D that can personalize high-fidelity and consistent 3D content from only a single image of a subject with text description within 5 minutes.

    dise Sherpa3D: Boosting High-Fidelity Text-to-3D Generation via Coarse 3D Prior
    Fangfu Liu, Diankun Wu, Yi Wei , Yongming Rao , Yueqi Duan
    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
    [arXiv] [Code] [Project Page]

    We propose Sherpa3D, a new text-to-3D framework that achieves high-fidelity, generalizability, and geometric consistency simultaneously. Extensive experiments show the superiority of our Sherpa3D over the state-of-the-art text-to-3D methods in terms of quality and 3D consistency.

    dise Discovering Dynamic Causal Space for DAG Structure Learning
    Fangfu Liu, Wenchang Ma, An Zhang , Xiang Wang , Yueqi Duan , Tat-Seng Chua
    ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2023
    Oral Presentation
    [arXiv] [Code] [Project Page]

    we propose a dynamic causal space for DAG structure learning, coined CASPER, that integrates the graph structure into the score function as a new measure in the causal space to faithfully reflect the causal distance between estimated and groundtruth DAG.

    dise Semantic Ray: Learning a Generalizable Semantic Field with Cross-Reprojection Attention
    Fangfu Liu, Chubin Zhang, Yu Zheng, Yueqi Duan
    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
    [arXiv] [Code] [Project Page]

    We propose a neural semantic representation called Semantic-Ray (S-Ray) to build a generalizable semantic field, which is able to learn from multiple scenes and directly infer semantics on novel viewpoints across novel scenes.

    dise Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting
    An Zhang, Fangfu Liu, Wenchang Ma, Zhibo Cai, Xiang Wang , Tat-Seng Chua
    International Conference on Learning Representations (ICLR), 2023
    [arXiv] [Code] [Project Page]

    We propose ReScore, a simple-yet-effective model-agnostic optimzation framework that simultaneously eliminates spurious edge learning and generalizes to heterogeneous data by utilizing learnable adaptive weights.

    dise VL-Grasp: a 6-Dof Interactive Grasp Policy for Language-Oriented Objects in Cluttered Indoor Scenes
    Yuhao Lu, Yixuan Fan, Beixing Deng, Fangfu Liu, Yali Li, Shengjin Wang
    International Conference on Intelligent Robots and Systems (IROS), 2023
    [arXiv] [Code] [Project Page]

    The VL-Grasp is an interactive grasp policy combined with visual grounding and 6-dof grasp pose detection tasks. The robot can adapt to various observation views and more diverse indoor scenes to grasp the target according to a human's language command by applying the VL-Grasp. Meanwhile, we build a new visual grounding dataset specially designed for the robot interaction grasp task, called RoboRefIt.

    Honors and Awards

  • National Scholarship (Top 1 in 260+ in the 2019-2020 academic year)
  • Tsinghua University Comprehensive Excellent Award twice (Top 5% in 260+, 2020&2021)
  • Tsinghua Science and Technology Innovation Excellence Award (2022)
  • Four star Bauhinia volunteer of Tsinghua University (Volunteer hours up to 150, 2021)
  • Advanced individual award of Tsinghua University (2019)
  • Academic Services

  • Review for PRCV 2024, TCSVT, NeurIPS 2022, IROS 2023, ACMMM 2024, NeurIPS 2024, ICLR 2025.

  • Website Template


    © Fangfu Liu | Last updated: 15 Aug, 2024