Active Sensor Planning for Multiview Vision Tasks

   Contents




   Preface                                                                                                  V

   Chapter 1 Introduction      ......... ..........               ..............    .............. ........... 1
   1.1 Motivations                                                                                            1
       1.1.1 The Tasks                                                                                        1
       1.1.2 From a Biological View                                                                           3
       1.1.3 The Problems and Goals                                                                           4
       1.1.4 Significance and Applications                                                                   6
   1.2 Objectives and Solutions                                                                               7
   1.3 Book Structure                                                                                         8

   Chapter 2 Active Vision Sensors ...................................................................     11
   2.1 3D Visual Sensing by Machine Vision                                                                 11
       2.1.1 Passive Visual Sensing                                                                        11
       2.1.2 Active Visual Sensing                                                                         14
   2.2 3D Sensing by Stereo Vision Sensors                                                                 19
       2.2.1 Setup with Two Cameras                                                                        19
       2.2.2 Projection Geometry                                                                           20
       2.2.3 3D Measurement Principle                                                                      21
   2.3 3D Sensing by Stripe Light Vision Sensors                                                           23
       2.3.1 Setup with a Switchable Line Projector                                                        23
       2.3.2 Coding Method                                                                                 24
       2.3.3 Measurement Principle                                                                         25
   2.4 3D Sensor Reconfiguration and Recalibration                                                         27
       2.4.1 The Motivation for Sensor Reconfiguration and Recalibration                                   28
       2.4.2 Setup of a Reconfigurable System                                                              29
       2.4.3 Geometrical Constraint                                                                        33
       2.4.4 Rectification of Stripe Locations                                                             34
       2.4.5 Solution Using the Geometrical Cue                                                            35
   2.5 Summary                                                                                             38

   Chapter 3 Active Sensor Planning - the State-of-the-Art..........._..............™.39
   3.1 The Problem                                                                    39
   3.2 Overview of the Recent Development                                             40
   3.3 Fundamentals of Sensor Modeling and Planning                                   43
   3.4 Planning for Dimensional Inspection                                            48
   3.5 Planning for Recognition and Search                                            51




                                                                                         gescannt durch
Vffl    Contents


3.6  Planning for Exploration, Navigation, and Tracking                                 54
3.7  Planning for Assembly and Disassembly                                              59
3.8  Planning with Illumination                                                         60
3.9  Other Planning Tasks                                                               63
     3.9.1 Interactive Sensor Planning                                                  63
     3.9.2 Placement for Virtual Reality                                                64
     3.9.3 Robot Localization                                                           64
     3.9.4 Attention and Gaze                                                           65
3.10 Summary                                                                            66

Chapter 4 Sensing Constraints and Evaluation .... .......        .........   .............67
4.1 Representation of Vision Sensors                                                      67
4.2 Placement Constraints                                                                 68
    4.2.1 Visibility                                                                      68
    4.2.2 Viewing Angle                                                                   69
    4.2.3 Field of View                                                                   69
    4.2.4 Resolution                                                                      70
    4.2.5 In Focus and Viewing Distance                                                   71
    4.2.6 Overlap                                                                    .....72
    4.2.7 Occlusion                                                                       72
    4.2.8 Image Contrast                                                                  73
    4.2.9 Robot Environment Constraints                                                   73
4.3 Common Approaches to Viewpoint Evaluation                                             75
4.4 Criterion of Lowest Operation Cost                                                    77
4.5 Summary                                                                               80

Chapter 5 Model-Based Sensor Planning... ........................—......      ...........81
5.1 Overview of the Method                                                               81
5.2 Sensor Placement Graph                                                               82
    5.2.1 HGA Representation                                                             82
    5.2.2 Min-Max Objective and Fitness Evaluation                                       83
    5.2.3 Evolutionary Computing                                                         84
5.3 The Shortest Path                                                                    85
    5.3.1 The Viewpoint Distance                                                         85
    5.3.2 Determination of a Shortest Path                                               86
5.4 Practical Considerations                                                             87
    5.4.1 Geometry Scripts                                        '.                  ...87
    5.4.2 Inspection Features                                                            87
    5.4.3 Sensor Structure                                                               88
    5.4.4 Constraint Satisfaction                                                        89
    5.4.5 Viewpoint Initialization                                                       90
5.5 Implementation                                                                       92
    5.5.1 The Viewpoint Planner                                                     .....92
    5.5.2 Examples of Planning Results                                              .....92
                                                                            Contents       DC
                                                                                             \.
    5.5.3 Viewpoint Observation                                                        .....97
    5.5.4 Experiments with a Real System                                                    98
5.6 Summary                                                                               100

Chapter 6 Planning for Freeform Surface Measurement..................                    101
6.1 The Problem                                                                          101
6.2 B-Spline Model Representation                                                        104
    6.2.1 B-Spline Representation                                                        104
    6.2.2 Model Selection                          „                                     105
6.3 Uncertainty Analysis                                                                 108
6.4 Sensing Strategy for Optimizing Measurement                                          110
    6.4.1 Determining the Number of Measurement Data                                     110
    6.4.2 Optimizing the Locations of Measurement Data                                   110
6.5 Experiments                                                                          112
6.6 Summary                                                                              118

Chapter 7 Sensor Planning for Object Modeling............                      ............119
7.1 Planning Approaches to Model Construction                                              119
    7.1.1 Model Construction from Multiple Views                                           119
    7.1.2 Previous Planning Approaches for Modeling                                        122
7.2 The Procedure for Model Construction                                                   124
7.3 Self-Termination Criteria                                                              127
    7.3.1 The Principle                                                                    127
    7.3.2 Termination Judgment                                                             128
7.4 Experiments                                                                            131
7.5 Summary                                                                                144

Chapter 8 Information Entropy Based Planning............................................ 147
8.1 Overview                                                                             147
8.2 Model Representation                                                                 148
    8.2.1 Curve Approximation                                                            149
    8.2.2 Improved BIC Criterion                                                         150
8.3 Expected Error                                                                       154
    8.3.1 Information Entropy of a B-Spline Model                                        155
    8.3.2 Information Gain                                                               156
8.4 View Planning                                                                        157
8.5 Experiments                                                                          159
    8.5.1 Setup                                                                          159
    8.5.2 Model Selection                                                                160
    8.5.3 Determining the NBV                                                            163
    8.5.4 Another Example                                                                172
8.6 Summary                                                                              175

Chapter 9 Model Prediction and Sensor Planning ...~™._......~...™.~...««._.177
9.1 Surface Trend and Target Prediction                                    177
    9.1.1 Surface Trend                                                    177
X     Contents

      9.1.2 Determination of the Exploration Direction                                                 179
      9.1.3 Surface Prediction                                                                         182
9.2   Determination of the Next Viewpoint                                                              183
9.3   Simulation                                                                                       186
      9.3.1 Practical Considerations                                                                   186
      9.3.2 Numerical Simulation                                                                       187
9.4   Practical Implementation                                                                      ...191
9.5   Discussion and Conclusion                                                                        203
      9.5.1 Discussion ....„                                                                           203
      9.5.2 Conclusion                                                                                 205

Chapter 10 Integrating Planning with Active Illumination                           ......................207
10.1 Introduction                                                                                        207
10.2 From Human Vision to Machine Vision                                                                 209
10.3 Evaluation of Illumination Conditions                                                               210
      10.3.1 SNR                                                                                         210
      10.3.2 Dynamic Range                                                                               210
      10.3.3 Linearity                                                                                   211
      10.3.4 Contrast                                                                                    211
      10.3.5 Feature Enhancement                                                                         211
10.4 Controllable Things                                                                                 212
      10.4.1 Brightness                                                                                  212
      10.4.2 Color Temperature and Color Rendering Index                                                 212
      10.4.3 Glare                                                                                       213
      10.4.4 Uniform Intensity                                                                           213
10.5 Glare Avoidance                                                                                     214
      10.5.1 Disability Glare                                                                            214
      10.5.2 Discomfort Glare                                                                            215
10.6 Intensity Estimation                                                                               .216
      10.6.1 Sensor Sensitivity                                                                          216
      10.6.2 Estimation of Image Irradiance                                                         .....217
10.7 Intensity Control                                                                                   221
      10.7.1 TheSetpoint                                                                                 221
      10.7.2 System Design                                                                              .223
10.8 Simulation                                                                                          224
10.9 Implementation                                                                                      227
      10.9.1 Design for Active Illumination                                                              227
      10.9.2 Experimental Robots                                                                         228
10.10 Summary                                                                                            231

Rihiinpranhv        .     ........................................ .......................................233
A                                                                                                        .233
B                                                                                                         234
C.                                                                                                       .235
D,E                                                                                                       237
                                                    Contents X I

F,G                                                         238
H                                                           240
IJ
 ,                                                          242
K                                                           243
L                                                           244
M                                                           246
N,O,P,Q                                                     248
R                                                           250
S                                                           251
T                                                           254
U,V,W                                                       255
X,Y,Z                                                       257

Index.........   .......... . .. , . ......... .........
                  ..........   ..     ........ ........    261