Preface to the Second Edition Preface to the First Edition Introduction: Four Periods in the Research of the Learning Problem Rosenblatt's Perceptron (The 1960s) Construction of the Fundamentals of Learning Theory(The 1960s–1970s) Neural Networks (The 1980s) Returning to the Origin (The 1990s) Chapter 1 Setting of the Learning Problem 1.1 Function Estimation Model 1.2 The Problem of Risk Minimization 1.3 Three Main Learning Problems 1.3.1 Pattern Recognition 1.3.2 Regression Estimation 1.3.3 Density Estimation (Fisher–Wald Setting) 1.4 The General Setting of the Learning Problem 1.5 The Empirical Risk Minimization (ERM) Inductive Principle 1.6 The Four Parts of Learning Theory 1.7 The Classical Paradigm of Solving Learning Problems 1.7.1 Density Estimation Problem (MaximumLikelihood Method) 1.7.2 Pattern Recognition (Discriminant Analysis) Problem 1.7.3 Regression Estimation Model 1.7.4 Narrowness of the ML Method 1.8 Nonparametric Methods of Density Estimation 1.8.1 Parzen's Windows 1.8.2 The Problem of Density Estimation Is Ill-Posed 1.9 Main Principle for Solving Problems Using a Restricted Amount of Information 1.10 Model Minimization of the Risk Based on Empirical Data 1.10.1 Pattern Recognition 1.10.2 Regression Estimation 1.10.3 Density Estimation 1.11 Stochastic Approximation Inference Chapter 2 Consistency of Learning Processes 2.1 The Classical Definition of Consistency and the Concept of Nontrivial Consistency 2.2 The Key Theorem of Learning Theory 2.2.1 Remark on the ML Method 2.3 Necessary and Sufficient Conditions for Uniform Two-Sided Convergence 2.3.1 Remark on Law of Large Numbers and Its Generalization 2.3.2 Entropy of the Set of Indicator Functions 2.3.3 Entropy of the Set of Real Functions 2.3.4 Conditions for Uniform Two-Sided Convergence 2.4 Necessary and Sufficient Conditions for Uniform One-Sided Convergence 2.5 Theory of Nonfalsifiability 2.5.1 Kant's Problem of Demarcation and Popper's Theory of Nonfalsifiability 2.6 Theorems on Nonfalsifiability 2.6.1 Case of Complete (Popper's) Nonfalsifiability 2.6.2 Theorem on Partial Nonfalsifiability 2.6.3 Theorem on Potential Nonfalsifiability 2.7 Three Milestones in Learning Theory Informal Reasoning and Comments 2.8 The Basic Problems of Probability Theory and Statistics 2.8.1 Axioms of Probability Theory
2.9 Two Modes of Estimating a Probability Measure …… Chapter 3 Bounds on the Rate of Convergence ofLearning Processes Chapter 4 Controlling the Generalization Ability of Learning Processes Chapter 5 Methods of Pattern Recognition Chapter 6 Methods of Function Estimation Chapter 7 Direct Methods in Statistical Learning Theory Chapter 8 The Vicinal Risk Minimization Principle and the SVMs Chapter 9 Conclusion: What Is Important inLearning Theory? References Index
[