Contents




       Part I Introduction

            Computer Science and Fine Arts                                           3
            1.1 Why Use Computers for Arts?                                          3
                  1.1.1 Computer as an Art Tool                                      3
                 1.1.2 Computer as an Exceptional A rt Tool                          5
                 1.1.3 Computers as Mind-talkers                                     5
            1.2 Digital Arts                                                         7
                 1.2.1 What Are Digital Arts?                                        7
                 1.2.2 Manual or Automatic Art Creation                              7
                 1.2.3 Three Elements of Digital Arts                                8
                 1.2.4 Classification of the Book Chapters                           9
            1.3 Examples of Digital Arts                                             9
                 1.3.1 Digital Film                                                 10
                 1.3.2 Digital Painting                                             10
                 1.3.3 Computer Music                                               10
                 1.3.4 Digital Sculpture                                            11
                 1.3.5 Computer Dance                                               12
                 1.3.6 Computer Puppetry                                            13
                 1.3.7 Computer Calligraphy                                         14
            1.4 Why Digital Arts Are Computationally Challenging?                   15
                 1.4.1 Lack of Semantic Understanding                               15
                 1.4.2 T he Versatile Nature of Art                                 15
                 1.4.3 Aesthetic Evaluation and Feedback                            16
                 1.4.4 Inhomogeneity between the Two Types of Intelligence.         16
            References                                                              17




XIV     Contents


Part II Computer Science in Painting: A Brief Survey

2     Computer Science in Paintings or Drawings                           23
      2.1 Introduction                                                    23
      2.2 Automatic Generation of Paintings and Drawings from
           Photographs                                                    23
           2.2.1 Early Pioneering Work                                    23
           2.2.2 Representative Recent Work                               25
           2.2.3 Generating Paintings via Human-computer Interaction      29
      2.3 Automatic Generation of Painterly Rendering Animation
           from Videos                                                    30
      2.4 Interactive Generation of Painterly Rendering Images            31
      2.5 Automatic Generation of Painterly Rendering from 3D Models      32
           2.5.1 Automatic Generation of Illustrations and Line
                 Drawings from 3D Models                                  32
           2.5.2 Generating Painterly Rendering Animations from 3D
                 Models                                                   35
           2.5.3 Domain Specific Special-purpose Painterly Rendition
                 Generation                                               36
           2.5.4 Efficient Painterly Rendition Generation                 39
      2.6 Special Support for Digital Painting                            40
           2.6.1 Hardware Support for Digital Painting                    40
           2.6.2 Multiresolutional Painting                               40
      References                                                          42


Part III Interactive Digital Painting and Calligraphy

3     Introduction to Interactive Digital Chinese Painting and
      Calligraphy                                                         51
      3.1 Overview                                                        51
      3.2 Background                                                      51
           3.2.1 Previous Work                                            52
           3.2.2 Our Virtual Brush                                        55
      References                                                          56

4     Basic Algorithmic Framework of a Virtual Hairy
      Paintbrush System                                                   59
      4.1 Overview                                                        59
      4.2 Introduction                                                    59
          4.2.1 Overview of E-brush and Related Research                  60
          4.2.2 Our Work and Contributions                                62
      4.3 Writing Primitives                                              64
      4.4 The Model and the States                                        65
          4.4.1 The Parametric Model of the Virtual Hairy Brush . . . .   65
          4.4.2 The Parametric Model of a Writing Primitive               67
                                                      Contents      XV
     4.4.3 The Three States of a Brush                              70
4.5 Sampling of the Input Data                                      72
4.6 Dynamic Adjustments of the Brush                                75
     4.6.1 Estimating the Pysical Conditions of the Brush           75
     4.6.2 Dynamic Adjustment of the Middle Control Axis            76
     4.6.3 Dynamic Adjustment of the Middle Control Ellipse . . .   78
     4.6.4 Dynamic Adjustment of the Tip Control Line               79
     4.6.5 Splitting of the Virtual Hairy Brush                     80
     4.6.6 Ink Flowage between Writing Primitives                   80
4.7 The Writing Process                                             81
4.8 Customizing the Brush                                           83
     4.8.1 Quality Parameters                                       83
     4.8.2 Configuring the Brush with Machine Intelligence          84
4.9 System Implementation and Experiment Results                    87
4.10 Related Work                                                   89
     4.10.1 DAB                                                     90
     4.10.2 Virtual Brush by Wong and Ip                            93
     4.10.3 Other Virtual Brush Models                              94
4.11 Conclusion and Future Work                                     94
     4.11.1 Summary and Conclusion                                  94
     4.11.2 Future Work                                             95
References                                                          97

Performance Enhanced Virtual Hairy Paintbrush System .              103
5.1 Overview                                                        103
5.2 Introduction                                                    103
5.3 Modeling the Paintbrush's Geometry                              105
     5.3.1 Three-layer Hierarchical Modeling                        106
     5.3.2 Real-time Visual Display of the Brush                    108
5.4 Modeling the Paintbrush's Dynamic Behavior                      110
     5.4.1 Deformation due to Brush-paper Collision                 112
     5.4.2 Deformation due to Inner Stress                          114
     5.4.3 Calibrating the On-line Results                          117
5.5 E-painting System based on Realistic Virtual Brush Modeling     119
     5.5.1 Additional Components of Our New Painting System .       119
     5.5.2 The Running System                                       121
5.6 Related Work                                                    121
     5.6.1 Wong & Ip's System                                       121
     5.6.2 The DAB System                                           123
     5.6.3 Chu & Tai's System                                       124
5.7 Conclusion and Future Work                                      125
References                                                          126
XVI     Contents
6     Pigment Component of an Advanced Virtual Hairy
      Paintbrush System                                              129
      6.1 Overview                                                   129
      6.2 Introduction                                               129
           6.2.1 Main Ideas                                          130
           6.2.2 Pigment Model and the Brush                         131
           6.2.3 Organization of the Chapter                         132
      6.3 Previous Work                                              133
           6.3.1 Pigment Behavior Models                             133
           6.3.2 Comparison with Chu & Tai's Work                    134
      6.4 Pigment Sorption between the Brush and the Paper Surface . 135
      6.5 Pigment Diffusion on the Paper Surface                     137
      6.6 Pigment Diffusion at the Brush Tip                         139
      6.7 Evaporation                                                141
           6.7.1 At the Brush Tip Bündle                             141
           6.7.2 On the Paper Surface                                141
      6.8 Pigment Deposition on the Paper Fibers                     142
      6.9 Rendering the Simulation Results                           143
           6.9.1 Pigment Mixing with High Fidelity                   143
           6.9.2 Superimposing the Layers                            147
      6.10 Hardware-Accelerated Implementation                       148
      6.11 Experiment Results                                        148
      6.12 Conclusion and Future Work                                153
      References                                                     154

7     Rendering Component of an Advanced Virtual Hairy
      Paintbrush System                                                  159
      7.1 Motivation                                                     159
          7.1.1 Necessity and Importance of Brush Hair Rendering . . .   159
          7.1.2 Performance Requirements                                 160
          7.1.3 Brush Hair versus Human Hair                             160
      7.2 Overview                                                       161
      7.3 Introduction                                                   161
          7.3.1 Ideas and Contributions                                  161
          7.3.2 Organization of the Chapter                              162
      7.4 Related Work                                                   162
          7.4.1 Hair Rendering for Quality                               162
          7.4.2 Hair Rendering for Speed                                 163
          7.4.3 Image-Based Rendering                                    163
          7.4.4 Appearance Modeling                                      164
      7.5 Hair Modeling and Representation                               165
          7.5.1 Modeling Hair as Virtual Material                        165
          7.5.2 Four-level Hierarchy of Hair Modeling                    165
          7.5.3 Generalized Disk Structure for Representing Hair
                 Clusters                                                166
          7.5.4 Hair Density Field for Sector                            168
      7.6 HRIR-DB and Semantics-Aware Texture Function                   168
                                                       Contents    XVII
         7.6.1 SATF and Our Offline/Online Two-phased Rendering
                Algorithm                                           168
         7.6.2 Minimizing the Size of HRIR-DB                       170
    7.7 Constructing the Database of Hair Rendering Intermediate
         Results                                                    171
         7.7.1 Deriving an HRIR Record                              171
         7.7.2 Indexing an HRIR Record                              172
    7.8 Fast and High Quality Online Hair Rendering                 173
         7.8.1 Main Steps of Online Hair Rendering                  173
         7.8.2 SATF and re- and a-map Construction                  174
         7.8.3 Online Hair Lighting                                 177
         7.8.4 Online Hair Self-shadowing                           178
         7.8.5 Deriving Shading through Integrating All the
                Rendering Effects Together                          179
         7.8.6 Hardware Acceleration                                180
    7.9 Experiment Results                                          182
    7.10 Conclusion and Future Work                                 190
         7.10.1 Conclusion                                          190
         7.10.2 Discussion and Future Work                          190
    References                                                      195


Part IV Automatic Generation of Artistic Chinese Calligraphy

8   Principles of Automatic Generation of Artistic Chinese
    Calligraphy                                                    203
    8.1 Overview                                                   203
    8.2 Introduction                                               203
    8.3 Problem Formulation and Overall System Architecture        206
    8.4 Hierarchical and Parametric Representation                 208
         8.4.1 Hierarchical Representation                         208
         8.4.2 Six Levels of Parametric Representation             208
         8.4.3 Advantages of Our Representation                    210
    8.5 Calligraphic Shape Decomposition                           210
         8.5.1 Extracting Levels 0-1 Elements                      211
         8.5.2 Extracting Levels 2-3 Elements                      211
         8.5.3 Extracting Level 4 Elements                         212
    8.6 Calligraphic Model Creation from Examples                  212
         8.6.1 Principles of Calligraphic Model Creation           212
         8.6.2 Fusing Knowledge Sources in ARP                     213
         8.6.3 A Computational Psychology Perspective              214
    8.7 Generating Artistic Calligraphy                            214
         8.7.1 Extracting Aesthetic Constraints from Training
               Examples                                            214
         8.7.2 Past Results Reuse for Emcient Reasoning            215
    8.8 Experiment Results                                         216
    8.9 Possible Applications                                      220
XVIII Contents
    8.10 Conclusion and Future Work                                     222
         8.10.1 Conclusion                                              222
         8.10.2 Future Work                                             222
    References                                                          224

9   Two Perspectives on Automatic Generation of Artistic
    Chinese Calligraphy                                                 227
    9.1 Overview                                                        227
    9.2 A System Engineering Perspective on Automatic Generation
         of Artistic Chinese Calligraphy                                227
    9.3 Hierarchical and Parametric Representation                      228
         9.3.1 Hierarchical Representation                              228
         9.3.2 Six Levels of Parametric Representation                  229
         9.3.3 Deriving Parametric Representations for Constructive
                 Elements                                               230
    9.4 Facsimiling Existent Calligraphy                                233
         9.4.1 Extracting Levels 0-1 Elements                           233
         9.4.2 Extracting Levels 2-3 Elements                           234
         9.4.3 Extracting Level 4 Elements                              235
    9.5 Generating New Calligraphy                                      235
         9.5.1 Principle of New Calligraphy Generation                  235
         9.5.2 New Calligraphy Generation System                        236
    9.6 Generating Artistic Calligraphy                                 240
         9.6.1 Constraints on the Process                               240
         9.6.2 Extracting Aesthetic Constraints from Existent
                 Artwork                                                240
         9.6.3 Constraint Satisfaction for Calligraphy Generation....   241
         9.6.4 Relaxing the Aesthetic Constraints                       242
    9.7 An Artificial Intelligence's Perspective on Automatic
         Generation of Artistic Chinese Calligraphy                     242
    9.8 Background                                                      243
    9.9 The Synthesis Reasoning Model                                   243
         9.9.1 Features of the Model                                    244
         9.9.2 Key Concepts of the Model                                244
         9.9.3 The Computational Model of Synthesis Reasoning . . . .   245
    9.10 A Generic Methodology to Developing Synthesis
         Reasoning-based Intelligent Systems                            248
    References                                                          249

10 A Preliminary Attempt at Evaluating the Beauty of
   Chinese Calligraphy                                                  253
   10.1 Overview                                                        253
   10.2 Intrcduction                                                    254
        10.2.1 Motivation                                               254
        10.2.2 Chapter Organization                                     255
   10.3 Previous Work                                                   256
   10.4 Calligraphy Representation                                      257
                                                              Contents   XIX

    10.5 Extracting Calligraphy Representation through a
         Two-phased Method                                               258
         10.5.1 Best-effort Automatic Stroke Extraction                  258
         10.5.2 Intelligent User Interface for the Difficult Cases       265
    10.6 Calligraphy Aesthetics Evaluation                               267
         10.6.1 Evaluating Shapes of Individual Strokes                  268
         10.6.2 Evaluating Spatial Layout of Strokes                     271
         10.6.3 Evaluating Coherence of Stroke Styles                    274
         10.6.4 The Overall Evaluation                                   275
    10.7 Automatic Generation of Aesthetic Calligraphy                   276
    10.8 Intelligent Calligraphy Tutoring System                         278
    10.9 Conclusion and Future Work                                      280
         10.9.1 Conclusion                                               280
         10.9.2 Discussion and Future Work                               281
    References                                                           285


Part V Animating Chinese Paintings

11 Animating Chinese Paintings through Stroke-based
   Decomposition                                                    289
   11.1 Overview                                                    289
   11.2 Introduction                                                289
   11.3 Painting Decomposition Approach                             292
        11.3.1 Image Segmentation                                   295
        11.3.2 Stroke Extraction by Region Merging                  295
        11.3.3 Stroke Refinement and Appearance Capture             301
        11.3.4 Thin Brush Strokes                                   302
   11.4 Appearance Capture and Synthesis of Single Brush Strokes .. 303
        11.4.1 Single-stroke Appearance Model                       303
        11.4.2 Why Direct Texture Mapping is Inadequate             304
   11.5 Separating Overlapping Brush Strokes                        306
   11.6 Decomposition and Reconstruction Results                    309
   11.7 Animating Paintings                                         312
   11.8 Discussion                                                  313
   11.9 Conclusion and Future Work                                  319
        11.9.1 Conclusion                                           319
        11.9.2 Future Work                                          319
   References                                                       321
XX      Contents


Part VI Perspectives

12 Final Fantasies for Digital Painting and Calligraphy                327
   12.1 Perspectives on Digital Paintbrush Research                    327
        12.1.1 An Ideal Digital Paintbrush System                      327
        12.1.2 A Surreal Digital Paintbrush System                     331
   12.2 Perspectives on Intelligent Calligraphy Research               340
        12.2.1 An Ideal Intelligent Calligraphy System                 340
        12.2.2 Intelligent Calligraphy System for Font Applications .. 342
        12.2.3 Intelligent Calligraphy Study for Other Applications .. 345
   12.3 An Ideal Painting Animation System                             347
   References                                                          348

Index                                                                 353