[06]image Composition And Matting

ICT Junior Camp 2007 @ Mahidol University Image Matting and Composition Dr. Rawesak Tanawongsuwan [email protected] 1

Outline • • • • •

Image Composition Matting Process Blue Screen Matting Environment/Shadow Matting Recent Matting Techniques (Knockout/Bayesian/Grabcut/Poisson) • Interactive Matting Techniques (Robust matting/Easy matting/Closed-form matting) • Flash Matting • Video Matting (Video cutout/Video defocus/Camera Array matting) 2

Digital Image Composition

3

Digital Image Composition

Matte

4

How does Superman fly?

5

Pulling a matte Problem Definition: • The separation of an image into • A foreground object image • A background image • An alpha matte

Hard problem 6

Matte creation • From digitized images – Blue-screen matting – Video or chroma-keying – Image processing

7

Blue Screen

8

Nothing special about blue

9

Digital Matting Equation • For each pixel C = αF + (1- α)B

• C: Rendered Color • F: Foreground color • B: Background color α: Amount of light passing through the foreground – called the Alpha-Channel 10

Challenges to Matting & Compositing • • • •

Extracting the Matte (alpha channel) from natural scenes Handling the occluding contour? Foreground has color of background • Refracted and reflected light • … 11

What about semi-transparent mattes?

12

We want a semi-transparent matte

13

Solution…

14

More example…

15

Problems with Matting • Images do not look realistic • Lack of Refracted Light • Lack of Reflected Light

16

Environment Matting and Composition

17

Process of environment matting • Learn how light is reflected and refracted from a backdrop and from various sidedrops

18

Process of environment matting

19

Results of environment matting

20

Results of environment matting

21

Movies of environment matting

22

Shadow matting and compositing

23

Shadow matting and compositing

24

Shadow matting • Show clip ShadowMattingDivXS.avi

25

Natural image matting • Beyasian approach

26

Natural image matting

27

Level of user interaction

Trimap GrabCut

28

Robust matting tool • Demo

29

Interactive matting • Easy matting

30

Interactive matting • Closed form matting

31

Interactive matting • Soft Scissors

32

Flash matting • Extract mattes using a pair of flash/no-flash images

33

Image Matting is Difficult

34

Image Matting is Easy with FLASH

35

Flash Matting

36

Flash matting • Show clip FlashMatting.wmv

37

Video • Let’s understand the nature of video clip • A sequence of image frames

• Show

videoCubism.avi

38

Video matting • Easier or more difficult? • Issues… • Show VideoCutout.avi

39

Special hardware/cameras • Need more than a single camera

• Defocus video matting • Multiple cameras matting

40

Defocus video : Capture Hardware

41

Capture Hardware

42

Preview: Input

Everything in Focus

IP

Actor in Focus

IF

Background in Focus

IB

43

Defocus video • Show clip DefocusMatting.avi

44

Multi-cameras system

45

Using a Camera Array

46

4500

2500

500

47

48

Multi-camera system • Automatic matte extraction • Toward real-time performance • Show clip CameraArrayMatting.mp4

49