1. Human-Robot Interaction, 09/2012 – 06/2014
    Lab: Intelligent Robot and Automation Lab
    Advisor: Prof. Li-Chen Fu, NTUCSIE
    We designed an inference model to deal with the uncertainty in multi-human social interactive patterns and task planning. Human’s social interactive patterns were inferred with sensor fusion and sociological understanding of spatial signals; the interactive patterns coupled with environmental context were considered in a Decision Network to plan interaction tasks; and then the robot took a series of actions, modeled by a task planner, to complete those tasks. The sensor fusion sources are a laser range finder detecting human legs, an ASUS Xtion Pro with OPENNI2 library for human torso detection and a webcam with OpenCV library to detect human faces. We implemented the Decision Network with GUI GeNIe and SMILE library. The study has resulted in a paper submitted to the IEEE Transactions on Robotics in 2014. For more information, please refer to the research paper.

    Social interactive pattern inference of two groups of people

  2. Image Restoration and Super-resolution for Electron Microscopic Imaging, 11/2013 – 05/2014
    Lab: Media IC and System Lab
    Advisor: Prof. Shao-Yi Chien, NTUEE
    Our team developed an example-based electron microscopic (EM) image reconstruction framework to denoise and super-resolve real-world 3D protein structures. By exploiting prior information from the Protein Data Bank, we masked the input data using TOMOBFLOW to subtract the background. We then matched the foreground patches with the prior images by using a Nearest-neighbor algorithm and replaced them with the corresponding prior images. Finally we obtained high-resolution denoised images. Our team won Second Place in the Signal Processing Cup at the IEEE 2014 International Conference on Acoustics, Speech, and Signal Processing, 2014 (ICASSP 2014). For more information, please refer to the research paper.project_SPCup_result_1

    (upper row) Input structures
    (lower row) Denoised and superresolved results

  3. Interactive Coaching System on Altera FPGA, 05/2013 – 11/2013
    Lab: Digital Circuit Lab
    Advisor: Prof. Shao-Yi Chien, NTUEE
    Teaming up with two classmates, I developed an FPGA-based dance coaching system, called iCoach, featuring a “virtual dance coach,” for multiple novice users to learn dancing with real-time feedback on their performance. This work earned us a Silver Medal Award and an Outstanding Paper Award in the 2013 Altera InnovateAsia FPGA Workshop and Design Contest. For more information, please refer to the research paper and demo video.

    User Interface of iCoach


  1. Scene Recognition and User Pose Estimation, 07/2014 – 08/2014
    Advanced Research Institute, Institute for Information Industry, Taiwan
    Dr. Grace Lin
    In this project, I was assigned to co-develop a mobile app Smart Tourism Taiwan. I implemented a GPS receiver and real-time image recognition to identify tourist scenes. I also developed photo depth and camera pose estimation algorithms and employed them along with GPS information to estimate user poses. This project earned me First Place at the final presentation among 22 graduate and undergraduate summer interns. My work was also documented in a research paper accepted by 17th International Conference on Human-Computer Interaction (HCI International 2015). For more information, please refer to the research paper and demo video.


Depth estimation and user-pose affine matrix


  1. Handwritten Formula Recognition, 04/2014 – 06/2014
    Probabilistic Machine Perception
    Prof. Chieh-Chih (Bob) Wang, NTUCSIE
    Teaming up with two classmates, I implemented a handwritten formula recognizer. Individual symbol recognition was achieved by extracting the HOG features, reducing data dimensionality using Principal Component Analysis and training symbol classification model with libSVM, and formula structure analysis was done by a baseline approach. The recognizer was able to recognize complex arithmetic equations, fractions and powers. For more information, please refer to the report.project_pmp_result

    Recognition result

  2. 3D Facial Model Construction, 09/2014 – 06/2014
    Network and Multimedia Lab
    Instructor: Prof. Tsungnan Lin, NTUEE
    Teaming up with two classmates, I developed a cross-platform 3D face modeling mobile application SUPER FACE. Developed with Unity3D, I implemented Image Warping based on corresponding keypoints to transform the photo of users and applied the transformed photo patches as textures to 3D face models from Maya. For more information, please refer to the presentation slides and demo video.project_nmlab_result_all

    Facial model of one of my teammates

  3. Projector Array 3D Display, 03/2014 – 06/2014
    Multimedia Processing and Communications Lab
    Prof. Homer H. Chen, NTUEE
    Inspired by USC’s work on 3D facial display, my teammate and I employed a pico-project array to achieve a “glasses-free 3D display”. Pico-projectors were used to project images from different view angles, and an anisotropic lenticular lens screen (with narrow horizontal diffusion and wide vertical diffusion) was setup to reflect the projection light, and thus the images were interlaced for observation.project_mpac_result_2_proj

    The images observed from different view angles (2 projectors)

  4. Simultaneous Localization and Mapping (SLAM), 09/2013 – 01/2014
    Robot Perception and Learning
    Prof. Chieh-Chih (Bob) Wang, NTUCSIE
    Teaming up with two classmates, constructed a 3D map from RGB-Depth data using the techniques of Perspective-n-Point camera pose estimation with SURF matching, and Iterative Closest Point. The robot was able to localize itself in an unknown environment and construct a rather satisfying map of the environment within reasonable time. The implementation was done in C++ with OpenCV and Point Cloud Library. For more information, please refer to the report and demo video.

    Snapshot of the map point cloud during SLAM

  5. Storage Robot, 09/2013 – 01/2014
    Course: Robotics
    Instructor: Prof. Li-Chen Fu, NTUCSIE
    Teaming up with two classmates, I built a clerk robot which helps customers deposit and withdraw their belongings. A Pioneer 3DX mobile robot, integrated with a UAL5D robot arm and Microsoft Kinect (with embedded microphone), was able to move and navigate, grasp objects and position them using inverse kinematics, and interact with humans by audio (speech recognition and text-to-speech) and vision (human posture tracking and object detection). My role in this project was controlling the whole storage scenario and implementing the vision and speech functions, which were by achieved by OpenNI2, OpenCV (vision), Google Translate and AT&T Text-to-Speech Demo (speech) respectively. For more information, please refer to the report and demo video.Storage_Robot

    Storage Robot handing a box to a user from the user’s point of view

  6. Real-Time Rugby Posture Classification, 02/2013 – 06/2013
    Course: Advanced Mobile Robotics
    Instructor: Prof. Chieh-Chih (Bob) Wang, NTUCSIE
    In this project, a classification model was trained by libSVM to recognize rugby postures using Microsoft Kinect. For more information, please refer to the presentation

    Collecting training data of the rugby posture “pass”

  7. Object Tracking of Mobile Robot, 09/2012 – 01/2013
    Course: Special Project
    Instructor: Prof. Li-Chen Fu, NTUCSIE
    Teaming up with two classmates, I programmed a Pioneer 3DX mobile robot with a Microsoft Kinect to track a user-specified moving target. We calibrated the Kinect sensor, used OpenCV library to detect and track the target, and ARIA library to control the Pioneer 3DX to approach the target. For more information, please refer to the report.p3dx_with_kinect

    Diagram of our mobile robot: a Microsoft Kinect mounted on a Pioneer 3DX ( image source )

  8. AI Poker Game Player, 09/2012 – 01/2013
    Course: Artificial Intelligence
    Instructor: Prof. Tian-Li Yu, NTUEE
    Teaming up with two classmates, I programmed an AI player for the poker game “Cheat” using Reinforcement Learning with winning rate over 70% against human players. I contributed to this project by designing multiple playing strategies thus different learning algorithms of our AI agents. For more information, please refer to the report and presentation slides.cheat

    User interface of poker game “Cheat”