Roberto Cipolla



Active Visual Inference
of Surface Shape




      Springer
                         Contents

Introduction                                                               1
1.1 M otivation                                                            1
     1.1.1 D epth cues from stereo and s tructure from motion . . . .      1
     1.1.2 Shortcomings                                                    5
1.2 Approach                                                               7
     1.2.1 Visual motion and differential geometry                         7
     1.2.2 Active vision                                                   7
     1.2.3 Shape representation                                            8
     1.2.4 Task oriented vision                                            9
1.3 T hemes and c ontributions                                             9
     1.3.1 Curved surfaces                                                 9
     1.3.2 Robustness                                                     10
1.4 O utline of book                                                      11

Surface Shape from the Deformation of Apparent Contours                   13
2.1 Introduction                                                          13
2.2 Theoretical framework                                                 15
    2.2.1 The apparent contour and its contour generator                  15
    2.2.2 Surface geometry                                                17
    2.2.3 Imaging model                                                   20
    2.2.4 Viewer and reference co-ordinate systems                        21
2.3 Geometric properties of the contour generator and its projection      21
    2.3.1 Tangency                                                        22
    2.3.2 Conjugate direction relationship of ray and contour generator   22
2.4 Static properties of apparent contours                                23
    2.4.1 Surface normal                                                  26
    2.4.2 Sign of normal curvature along the contour generator . .        26
    2.4.3 Sign of Gaussian curvature                                      28
2.5 The dynamic analysis of apparent contours                             29
    2.5.1 Spatio-temporal parameterisation                                29
                                                                        C ontents



      2.5.2   Epipolar parameterisation                                        30
2.6   Dynamic properties of a pparent contours                                 33
      2.6.1 Recovery of d epth from image velocities                           33
      2.6.2   Surface curvature from deformation of the a pparent contour 3 3
      2.6.3 Sidedness of a pparent contour a nd c ontour generator . . .       35
      2.6.4 Gaussian a nd m ean curvature                                      36
      2.6.5 Degenerate cases of the e pipolar parameterisation . . . .         36
2.7   Motion parallax a nd the r obust estimation of s urface curvature .      37
      2.7.1   Motion parallax                                                  41
      2.7.2   R a t e of p arallax                                             42
      2.7.3   Degradation of s ensitivity with separation of p oints . . . .   44
      2.7.4 Qualitative shape                                                  45
2.8   Summary                                                                  45

Deformation o f A pparent Contours — I mplementation                            47
3.1 Introduction                                                               47
3.2 Tracking image contours with B-spline snakes                               48
    3.2.1 Active contours -snakes                                              50
    3.2.2 T he B-spline snake                                                  51
3.3 T he epipolar parameterisation'                                            57
    3.3.1 Epipolar plane image analysis                                        58
    3.3.2 Discrete viewpoint analysis                                          64
3.4 Error a nd sensitivity analysis . r                                        68
3.5 Detecting extremal boundaries a nd recovering surface shape . . .          71
    3.5.1 Discriminating between fixed a nd e xtremal boundaries . .           71
    3.5.2 Reconstruction of surfaces                                           75
3.6 Real-time experiments exploiting visually derived shape information        78
    3.6.1 Visual navigation around curved objects                              78
    3.6.2 Manipulation of curved objects                                       79

Qualitative Shape from Images of Surface Curves                                81
4.1 Introduction                                                               81
4.2 The perspective projection of space curves                                 84
    4.2.1 Review of space curve geometry                                       84
    4.2.2 Spherical camera notation                                            86
    4.2.3 Relating image and space curve geometry                              88
4.3 Deformation due to viewer movements                                        90
    4.3.1 Depth from image velocities                                          92
    4.3.2 Curve tangent from rate of change of orientation of image
           tangent                 •                                           93
    4.3.3 Curvature and curve normal                                           94
Contents                                                                  XI


    4.4   Surface geometry                          -                     95
          4.4.1 Visibility constraint                                     95
          4.4.2 Tangency constraint                                       97
          4.4.3 Sign of normal curvature at inflections                   97
          4.4.4 Surface curvature at curve intersections                 107
    4.5   Ego-motion from the image motion of curves                     109
    4.6   Summary                                                        114

5   Orientation and Time to Contact from Image Divergence and
    Deformation                                                           117
    5.1 Introduction                                                      117
    5.2 Structure from motion                                             118
        5.2.1 Background                                                  118
        5.2.2 Problems with this approach                                 119
        5.2.3 The advantages of partial solutions                         120
    5.3 Differential invariants of the image velocity field               121
        5.3.1 Review                                                      121
        5.3.2 Relation to 3D shape and viewer ego-motion                  125
        5.3.3 Applications                                                131
        5.3.4 Extraction of differential invariants                       133
    5.4 Recovery of differential invariants from closed contours          136
    5.5 Implementation and experimental results                           139
        5.5.1 Tracking closed loop contours                               139
        5.5.2 Recovery of time to contact and surface orientation . . . . 140

6   Conclusions                                                          1 51
    6.1 Summary                                                          151
    6.2 Future work                                                      152

A Bibliographical Notes                                                  155
    A.I Stereo vision                                                     155
    A.2 Surface reconstruction                                            157
    A.3 Structure from motion                                             159
    A.4 Measurement and analysis of visual motion                         160
        A.4.1 Difference techniques                                       160
        A.4.2 Spatio-temporal gradient techniques                         160
        A.4.3 Token matching                                        . . . 161
        A.4.4 Kalman filtering                                            164
        A.4.5 Detection of independent motion                             164
        A.4.6 Visual attention                                            165
    A.5 Monocular shape cues                                              166
XII                                                         Contents


          A.5.1 Shape from shading                              166
          A.5.2 Interpreting line drawings                      167
          A.5.3 Shape from contour                              168
          A.5.4 Shape from texture                              169
      A.6 Curved surfaces                                       169
          A.6.1 Aspect graph and singularity theory             169
          A.6.2 Shape from specularities                        170

B Orthographic projection and planar motion                     172

C Determining Su-n from t he spatio-temporal image q(s,t)       175

D Correction for parallax based measurements when image points
  are not coincident                                         177

Bibliography                                                    179