Face2Face: Real-time Face Capture and Reenactment of RGB Videos (CVPR 2016 Oral)

Author: Matthias Niessner
3193419 View
6m 36s Lenght
18327 Rating

CVPR 2016 Paper Video (Oral) Project Page: http://www.graphics.stanford.edu/~niessner/thies2016face.html IMPORTANT NOTE: This demo video is purely research-focused and we would like to clarify the goals and intent of our work. Our aim is to demonstrate the capabilities of modern computer vision and graphics technology, and convey it in an approachable and fun way. We want to emphasize that computer-generated videos have been part in feature-film movies for over 30 years. Virtually every high-end movie production contains a significant percentage of synthetically-generated content (from Lord of the Rings to Benjamin Button). These results are hard to distinguish from reality and it often goes unnoticed that the content is not real. The novelty and contribution of our work is that we can edit pre-recorded videos in real-time on a commodity PC. Please also note that our efforts include the detection of edits in video footage in order to verify a clip’s authenticity. For additional information, we refer to our project website (see above). Hopefully, you enjoyed watching our video, and we hope to provide a positive takeaway :) Paper Abstract We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time.

Comments

  1. Jebus Fudging Cripes.
  2. This is fucking scary
  3. How do Iget this at home though guys?? SERIOUSLY. I just wanna mess around with the kids and have some fun. This is SOOOO damn cool
  4. This is scary
  5. This is scary, but how would they get people to say things they wanted?
  6. Mutahar brought me here
  7. This is probably the most important video on youtube
  8. can someone put this in laymans terms? i dont understand what it is
  9. Now just imagine applying this technology to porn.
  10. aaaaaaaaaand "talking heads" was never a more relevant phrase.
  11. so now you will find different types of truth. nothing is what is seems anymore
  12. This is my theory on how they made that Mr.Robot speech Obama gave on Mr.Robot.
  13. Please NO. You would ruins my world.
  14. inb4 FOX news.
  15. shit this is not good.
  16. FUUUUUUUUUUUUTTTTTTTTTTUUUUUUUUUUURRRRRRRRRRREEEEEEEEEEEEEEEE
  17. Now I can finally webcam as a beautiful woman in real time....
  18. We're going to have to see technology restricting laws sooner than assumed.

    First, needs to be automation so we don't kill the jobs market for the 7 BILLION people on this planet and lose it all to machines.
    Second, video evidence laws are going to have to come in place. Obviously we can no longer trust what people say on screen let alone on audio devices, jesus christ.
    Third, legalize sex robots. That should actually be priority.
  19. Whoa! This does not mean it can be done real time. People can use software to create these effects after the fact. Record a face making many expressions, post it, then apply it to a real video that occurs later. If you can, great, you are amazing, otherwise it is just as I thought... fake. It is too bad you won't answer me and the masses will follow you like sheep.
  20. Why do we need this, exactly? Just because you can... doesn't mean you should.