Probably Approximately Correct

Probably Approximately Correct Author : Leslie Valiant
Release : 2013-06-04
Publisher : Hachette UK
ISBN : 0465037909
File Size : 80.82 MB
Format : PDF, Docs
Download : 540
Read : 404

From a leading computer scientist, a unifying theory that will revolutionize our understanding of how life evolves and learns. How does life prosper in a complex and erratic world? While we know that nature follows patterns -- such as the law of gravity -- our everyday lives are beyond what known science can predict. We nevertheless muddle through even in the absence of theories of how to act. But how do we do it? In Probably Approximately Correct, computer scientist Leslie Valiant presents a masterful synthesis of learning and evolution to show how both individually and collectively we not only survive, but prosper in a world as complex as our own. The key is "probably approximately correct" algorithms, a concept Valiant developed to explain how effective behavior can be learned. The model shows that pragmatically coping with a problem can provide a satisfactory solution in the absence of any theory of the problem. After all, finding a mate does not require a theory of mating. Valiant's theory reveals the shared computational nature of evolution and learning, and sheds light on perennial questions such as nature versus nurture and the limits of artificial intelligence. Offering a powerful and elegant model that encompasses life's complexity, Probably Approximately Correct has profound implications for how we think about behavior, cognition, biological evolution, and the possibilities and limits of human and machine intelligence.

Probably Approximately Correct Learning

Probably Approximately Correct Learning Author : David Haussler
Release : 1990
Publisher :
ISBN :
File Size : 75.70 MB
Format : PDF, Kindle
Download : 893
Read : 909

Abstract: "This paper surveys some recent theoretical results on the efficiency of machine learning algorithms. The main tool described is the notion of Probably Approximately Correct (PAC) learning, introduced by Valiant. We define this learning model and then look at some of the results obtained in it. We then consider some criticisms of the PAC model and the extensions proposed to address these criticisms. Finally, we look briefly at other models recently proposed in computational learning theory."

Probably Approximately Correct (PAC) Exploration in Reinforcement Learning

Probably Approximately Correct (PAC) Exploration in Reinforcement Learning Author :
Release : 2007
Publisher :
ISBN :
File Size : 29.54 MB
Format : PDF, ePub
Download : 610
Read : 430

Reinforcement Learning (RL) in finite state and action Markov Decision Processes is studied with an emphasis on the well-studied exploration problem. We provide a general RL framework that applies to all results in this thesis and to other results in RL that generalize the finite MDP assumption. We present two new versions of the Model-Based Interval Estimation (MBIE) algorithm and prove that they are both PAC-MDP. These algorithms are provably more efficient any than previously studied RL algorithms. We prove that many model-based algorithms (including R-MAX and MBIE) can be modified so that their worst-case per-step computational complexity is vastly improved without sacrificing their attractive theoretical guarantees. We show that it is possible to obtain PAC-MDP bounds with a model-free algorithm called Delayed Q-learning.

Probably Approximately Correct Learnable Fuzzy System

Probably Approximately Correct Learnable Fuzzy System Author : Yan Wang
Release : 2019
Publisher :
ISBN :
File Size : 72.41 MB
Format : PDF, Kindle
Download : 256
Read : 509

This dissertation develops the probably approximately correct (PAC) learnable fuzzy system to predict clinical outcomes from a small number of survey questions (short form). There are five layers in the system: input, fuzzification, inference, defuzzification, and production. The major product in this dissertation is to derive the PAC learnable knowledge-driven machine learning algorithm by growing sample using Bootstrap samples with Gaussian distributed noise. The input layer is the procedure for preparing data input. In the fuzzification layer, sample size is significantly increased using bootstrap re-sampling with replacement. The fuzzy set with proposed membership function is generated by introducing Gaussian distributed noise to survey responses of the bootstrap samples to reflect uncertainty. This is a natural language extension from the point option in survey questions to region input with probabilities from survey design space. The inference layer includes both classification and prediction. Here we use machine learning techniques to derive the algorithms in this layer, e.g. Naive Bayesian method and eXtreme Gradient Boosting (XGBoost). The final predicted values require a defuzzification process in the next layer to remove noise in prediction. There are four types of input after fuzzification, original input, fuzzy input, input required interpolation and input required extrapolation. The defuzzification process is based on weighted means of related information. The last step of the system is the output layer with algorithms, final prediction and validation internally and externally. Lastly, we apply this fuzzy system to derive PAC learnable algorithms to predict oral health clinical outcomes. The input predictors include short forms and demographic information. The short forms, developed from Graded Response Models in Item Response Theory, have two versions (children and their parents). The clinical outcomes are referral for treatment needs (categorical) and children's oral health status index score (continuous). The prediction is evaluated internally and externally by sensitivity and specificity of a binary variable, correlation (between original value and predicted value) and root mean square error (RMSE) of a continuous variable. Both internal and external validation show the improvement of prediction when new information is added and generalizability as well as the stability of the algorithm. The best prediction (high sensitivity and relatively high specificity for categorical variables, low RMSE and high correlation) is reached when using child's self-reported short form, plus parent's proxy-reported short form, and demographic characteristics.

Existence of PAC Concept Classes of Incomparable Degrees

Existence of PAC Concept Classes of Incomparable Degrees Author : D. Gihanee M. Senadheera
Release : 2019
Publisher :
ISBN :
File Size : 81.79 MB
Format : PDF, Kindle
Download : 829
Read : 787

Probably Approximately Correct (PAC) Learning is one of the models used in machine learning. This model was proposed by Valiant in 1984. The work related to this abstract was inspired by the Post's problem. Post classified computably enumerable (c.e.) sets and their degrees and was interested in finding more than two c.e. degrees. This was known as the Post's problem. In 1957 Friedberg and Muchnik independently showed this is possible. In PAC learning model, there are concept classes which are learnable also there are concept classes which are hard to learn. Later mathematicians were able to postulate the notion of PAC reducibility. That is, if a concept class C0 is PAC learnable through an algorithm then the concept class C1 reduces to the concept class C0 means C1 can be learned through the existing algorithm for C0. The term PAC degree means degree of unsolvability of a PAC concept class. It is natural to ask the question whether there are incomparable PAC degrees. In order to prove that there are incomparable PAC degrees we use the method known as priority construction, which is used by Freidbuerg and Muchnik in their work. We construct two concept classes C0 and C1 such that C0 is not reducible to C1 and C1 is not reducible to C0.

An Introduction to Computational Learning Theory

An Introduction to Computational Learning Theory Author : Michael J. Kearns
Release : 1994
Publisher : MIT Press
ISBN : 0262111934
File Size : 54.44 MB
Format : PDF, Mobi
Download : 127
Read : 611

Emphasizing issues of computational efficiency, Michael Kearns and Umesh Vazirani introduce a number of central topics in computational learning theory for researchers and students in artificial intelligence, neural networks, theoretical computer science, and statistics. Emphasizing issues of computational efficiency, Michael Kearns and Umesh Vazirani introduce a number of central topics in computational learning theory for researchers and students in artificial intelligence, neural networks, theoretical computer science, and statistics. Computational learning theory is a new and rapidly expanding area of research that examines formal models of induction with the goals of discovering the common methods underlying efficient learning algorithms and identifying the computational impediments to learning. Each topic in the book has been chosen to elucidate a general principle, which is explored in a precise formal setting. Intuition has been emphasized in the presentation to make the material accessible to the nontheoretician while still providing precise arguments for the specialist. This balance is the result of new proofs of established theorems, and new presentations of the standard proofs. The topics covered include the motivation, definitions, and fundamental results, both positive and negative, for the widely studied L. G. Valiant model of Probably Approximately Correct Learning; Occam's Razor, which formalizes a relationship between learning and data compression; the Vapnik-Chervonenkis dimension; the equivalence of weak and strong learning; efficient learning in the presence of noise by the method of statistical queries; relationships between learning and cryptography, and the resulting computational limitations on efficient learning; reducibility between learning problems; and algorithms for learning finite automata from active experimentation.

Efficiency and Computational Limitations of Learning Algorithms

Efficiency and Computational Limitations of Learning Algorithms Author : Vitaly Feldman
Release : 2007
Publisher :
ISBN : 9781109894431
File Size : 67.33 MB
Format : PDF, ePub, Mobi
Download : 763
Read : 955

This thesis presents new positive and negative results concerning the learnability of several well-studied function classes in the Probably Approximately Correct (PAC) model of learning.

Noise Tolerant Algorithms for Learning and Searching

Noise Tolerant Algorithms for Learning and Searching Author : Javed A. Aslam
Release : 1995
Publisher :
ISBN :
File Size : 80.81 MB
Format : PDF, ePub
Download : 163
Read : 436

Abstract: "We consider the problem of developing robust algorithms which cope with noisy data. In the Probably Approximately Correct Model of machine learning, we develop a general technique which allows nearly all PAC learning algorithms to be converted into highly efficient PAC learning algorithms which tolerate noise. In the field of combinatorial algorithms, we develop techniques for constructing search algorithms which tolerate linearly bounded errors and probabilistic errors. In the field of machine learning, we derive general bounds on the complexity of learning in the recently introduced Statistical Query model and in the PAC model with noise. We do so by considering the problem of improving the accuracy of learning algorithms. In particular, we study the problem of 'boosting' the accuracy of 'weak' learning algorithms which fall within the Statistical Query model, and we show that it is possible to improve the accuracy of such learning algorithms to any arbitrary accuracy. We derive a number of interesting consequences from this result, and in particular, we show that nearly all PAC learning algorithms can be converted into highly efficient PAC learning algorithms which tolerate classification noise and malicious errors. We also investigate the longstanding problem of searching in the presence of errors. We consider the problem of determining an unknown quantity x by asking 'yes-no' questions, where some of the answers may be erroneous. We focus on two different models of error: the linearly bounded model, where for some known constant r

Design and Analysis of Efficient Reinforcement Learning Algorithms

Design and Analysis of Efficient Reinforcement Learning Algorithms Author : Claude-Nicolas Fiechter
Release : 1997
Publisher :
ISBN :
File Size : 28.56 MB
Format : PDF, ePub, Mobi
Download : 788
Read : 507

Reinforcement learning considers the problem of learning a task or behavior by interacting with one's environment. The learning agent is not explicitly told how the task is to be achieved and has to learn by trial-and-error, using only the rewards and punishments that it receives in response to the actions it takes. In the last ten years there has been a rapidly growing interest in reinforcement learning techniques as a base for intelligent control architectures. Many methods have been proposed and a number of very successful applications have been developed. This dissertation contributes to a theoretical foundation for the study of reinforcement learning by applying some of the methods and tools of computational learning theory to the problem. We propose a formal model of efficient reinforcement learning based on Valiant's Probably Approximately Correct (PAC) learning framework, and use it to design reinforcement learning algorithms and to analyze their performance. We describe the first polynomial-time PAC algorithm for the general finite-state reinforcement learning problem and show that an active and directed exploration of its environment by the learning agent is necessary and sufficient to obtain efficient learning for that problem. We consider the trade-off between exploration and exploitation in reinforcement learning algorithms and show how in general an off-line PAC algorithm can be converted into an on-line algorithm that efficiently balances exploration and exploitation. We also consider the problem of generalization in reinforcement learning and show how in some cases the underlying structure of the environment can be exploited to achieve faster learning. We describe a PAC algorithm for the associative reinforcement learning problem that uses a form of decision lists to represent the policies in a compact way and generalize across different inputs. In addition, we describe a PAC algorithm for a special case of reinforcement learning where the environment can be modeled by a linear system. This particular reinforcement learning problem corresponds to the so-called linear quadratic regulator which is extensively studied and used in automatic and adaptive control.

The Harmonic Sieve: a Novel Application of Fourier Analysis of Machine Learning Theory and Practice

The Harmonic Sieve: a Novel Application of Fourier Analysis of Machine Learning Theory and Practice Author : Carnegie-Mellon University. Computer Science Department
Release : 1995
Publisher :
ISBN :
File Size : 23.83 MB
Format : PDF
Download : 388
Read : 803

Abstract: "This thesis presents new positive results -- both theoretical and empirical -- in machine learning. The primary learning- theoretic contribution is the Harmonic Sieve, the first efficient algorithm for learning the well-studied class of Disjunctive Normal Form (DNF) expressions (learning is accomplished within the Probably Approximately Correct model with respect to the uniform distribution using membership queries). Of particular interest is the novel use of Fourier methods within the algorithm. Specifically, all prior Fourier-based learning algorithms focused on finding large Fourier coefficients of the function to be learned (the target). The Harmonic Sieve departs from this paradigm; it instead learns by finding large coefficients of certain functions other than the target. The robustness of this new Fourier technique is illustrated by applying it to prove learnability of noisy DNF expressions, of a circuit class that is even more expressive than DNF, and of an interesting class of geometric concepts. Empirically, the thesis demonstrates the significant particular potential of a classification- learning algorithm closely related to the Harmonic Sieve. The Boosting- based Perceptron (BBP) learning algorithm produces classifiers that are nonlinear perceptrons (weighted thresholds over higher-order features). On several previously-studied machine learning benchmarks, the BBP algorithm produces classifiers that achieve accuracies essentially equivalent to or even better than the best previously-reported classifiers. Additionally, the perceptrons produced by the BBP algorithm tend to be relatively intelligible, an important feature in many machine learning applications. In a related vein, BBP and the Harmonic Sieve are applied successfully to the problem of rule extraction, that is, the problem of approximating an unintelligible classifier by a more intelligible function."

On Learning of Ceteris Paribus Preference Theories

On Learning of Ceteris Paribus Preference Theories Author :
Release : 2004
Publisher :
ISBN :
File Size : 64.9 MB
Format : PDF, ePub
Download : 965
Read : 981

The problem of preference elicitation has been of interest for a long time. While traditional methods of asking a set of relevant questions are still useful, the availability of user-preference data from the web has led to substantial attention to the notion of preference mining. In this thesis, we consider the problem of learning logical preference theories that express preference orderings over alternatives. We present learning algorithms which accept as input a set of comparisons between pairs of complete descriptions of world states. Our first algorithm, that performs exact learning, accepts the complete set of preference orderings for a theory and generates a theory which provides the same ordering of states as the input. This process can require looking at an exponential number of data points. We then look at more realistic approximation algorithms and analyze the complexity of the learning problem under the framework of Probably Approximately Correct (PAC) learning. We then describe approximation algorithms for learning high-level summaries of the underlying theory.

The Harmonic Sieve

The Harmonic Sieve Author : Jeffrey Jackson
Release : 1995
Publisher :
ISBN :
File Size : 33.68 MB
Format : PDF, Docs
Download : 443
Read : 188

Abstract: "This thesis presents new positive results -- both theoretical and empirical -- in machine learning. The primary learning- theoretic contribution is the Harmonic Sieve, the first efficient algorithm for learning the well-studied class of Disjunctive Normal Form (DNF) expressions (learning is accomplished within the Probably Approximately Correct model with respect to the uniform distribution using membership queries). Of particular interest is the novel use of Fourier methods within the algorithm. Specifically, all prior Fourier-based learning algorithms focused on finding large Fourier coefficients of the function to be learned (the target). The Harmonic Sieve departs from this paradigm; it instead learns by finding large coefficients of certain functions other than the target. The robustness of this new Fourier technique is illustrated by applying it to prove learnability of noisy DNF expressions, of a circuit class that is even more expressive than DNF, and of an interesting class of geometric concepts. Empirically, the thesis demonstrates the significant particular potential of a classification- learning algorithm closely related to the Harmonic Sieve. The Boosting- based Perceptron (BBP) learning algorithm produces classifiers that are nonlinear perceptrons (weighted thresholds over higher-order features). On several previously-studied machine learning benchmarks, the BBP algorithm produces classifiers that achieve accuracies essentially equivalent to or even better than the best previously-reported classifiers. Additionally, the perceptrons produced by the BBP algorithm tend to be relatively intelligible, an important feature in many machine learning applications. In a related vein, BBP and the Harmonic Sieve are applied successfully to the problem of rule extraction, that is, the problem of approximating an unintelligible classifier by a more intelligible function."

Scaling Up Reinforcement Learning Without Sacrificing Optimality by Constraining Exploration

Scaling Up Reinforcement Learning Without Sacrificing Optimality by Constraining Exploration Author : Timothy Arthur Mann
Release : 2013
Publisher :
ISBN :
File Size : 52.11 MB
Format : PDF, Kindle
Download : 133
Read : 1070

The purpose of this dissertation is to understand how algorithms can efficiently learn to solve new tasks based on previous experience, instead of being explicitly programmed with a solution for each task that we want it to solve. Here a task is a series of decisions, such as a robot vacuum deciding which room to clean next or an intelligent car deciding to stop at a traffic light. In such a case, state-of-the-art learning algorithms are difficult to employ in practice because they often make thou- sands of mistakes before reliably solving a task. However, humans learn solutions to novel tasks, often making fewer mistakes, which suggests that efficient learning algorithms may exist. One advantage that humans have over state- of-the-art learning algorithms is that, while learning a new task, humans can apply knowledge gained from previously solved tasks. The central hypothesis investigated by this dissertation is that learning algorithms can solve new tasks more efficiently when they take into consideration knowledge learned from solving previous tasks. Al- though this hypothesis may appear to be obviously true, what knowledge to use and how to apply that knowledge to new tasks is a challenging, open research problem. I investigate this hypothesis in three ways. First, I developed a new learning algorithm that is able to use prior knowledge to constrain the exploration space. Second, I extended a powerful theoretical framework in machine learning, called Probably Approximately Correct, so that I can formally compare the efficiency of algorithms that solve only a single task to algorithms that consider knowledge from previously solved tasks. With this framework, I found sufficient conditions for using knowledge from previous tasks to improve efficiency of learning to solve new tasks and also identified conditions where transferring knowledge may impede learning. I present situations where transfer learning can be used to intelligently constrain the exploration space so that optimality loss can be minimized. Finally, I tested the efficiency of my algorithms in various experimental domains. These theoretical and empirical results provide support for my central hypothesis. The theory and experiments of this dissertation provide a deeper understanding of what makes a learning algorithm efficient so that it can be widely used in practice. Finally, these results also contribute the general goal of creating autonomous machines that can be reliably employed to solve complex tasks. The electronic version of this dissertation is accessible from http://hdl.handle.net/1969.1/148402

A Unifying Framework for Computational Reinforcement Learning Theory

A Unifying Framework for Computational Reinforcement Learning Theory Author : Lihong Li
Release : 2009
Publisher :
ISBN :
File Size : 83.32 MB
Format : PDF, ePub
Download : 944
Read : 685

Computational learning theory studies mathematical models that allow one to formally analyze and compare the performance of supervised-learning algorithms such as their sample complexity. While existing models such as PAC (Probably Approximately Correct) have played an influential role in understanding the nature of supervised learning, they have not been as successful in reinforcement learning (RL). Here, the fundamental barrier is the need for active exploration in sequential decision problems. An RL agent tries to maximize long-term utility by exploiting its knowledge about the problem, but this knowledge has to be acquired by the agent itself through exploring the problem that may reduce short-term utility. The need for active exploration is common in many problems in daily life, engineering, and sciences. For example, a Backgammon program strives to take good moves to maximize the probability of winning a game, but sometimes it may try novel and possibly harmful moves to discover how the opponent reacts in the hope of discovering a better game-playing strategy. It has been known since the early days of RL that a good tradeoff between exploration and exploitation is critical for the agent to learn fast (i.e., to reach near-optimal strategies with a small sample complexity), but a general theoretical analysis of this tradeoff remained open until recently. In this dissertation, we introduce a novel computational learning model called KWIK (Knows What It Knows) that is designed particularly for its utility in analyzing learning problems like RL where active exploration can impact the training data the learner is exposed to. My thesis is that the KWIK learning model provides a flexible, modularized, and unifying way for creating and analyzing reinforcement-learning algorithms with provably efficient exploration. In particular, we show how the KWIK perspective can be used to unify the analysis of existing RL algorithms with polynomial sample complexity. It also facilitates the development of new algorithms with smaller sample complexity, which have demonstrated empirically faster learning speed in real-world problems. Furthermore, we provide an improved, matching sample complexity lower bound, which suggests the optimality (in a sense) of one of the KWIK-based algorithms known as delayed Q-learning.