Online-to-batch Conversions: In this exercise we demonstrate how a successful online learning algorithm can be used to derive a successful PAC learner as well.
Consider a PAC learning problem for binary classification parameterized by an instance domain, X , and a hypothesis class, H. Suppose that there exists an online learning algorithm, A, which enjoys a mistake bound MA(H) < ∞.=”” consider=”” running=”” this=”” algorithm=”” on=”” a=”” sequence=”” of=”” t=”” examples=”” which=”” are=”” sampled=”” i.i.d.=”” from=”” a=”” distribution=”” d=”” over=”” the=”” instance=”” space=”” x,=”” and=”” are=”” labeled=”” by=”” some=””>
∈ H. Suppose that for every round t, the prediction of the algorithm is based on a hypothesis ht : X → {0,1}. Show that
where the expectation is over the random choice of the instances as well as a random choice of r according to the uniform distribution over [T].