Averaged Perceptron

Table of Contents

Overview

Averaged perceptron is a optimization algorithm which only has one hyperparameter: averaged_perceptron_e10e2b430f95617381cdd6d6b52aed29fb971dff.png, the # of iterations.

It will keep updating towards seperating the labels based on the inputs for as long as you specify, hence it's prone to overfit.

Notation

  • averaged_perceptron_ed39d9a397196f8f0ce6388b0ea4e0c1dd8becee.png - a single input vector, of averaged_perceptron_b689cba8d7566f6adaf605a844e193a27e155078.png dimensions
  • averaged_perceptron_a3a7f43f807b9e381fc50e0fab140c0df0a03e17.png - a single label, averaged_perceptron_443df7665b7d93f50e32faef7e54fdbe41c72279.png
  • averaged_perceptron_76820a5087c7f9d1c4a3a4588f278d93a104ce88.png - the dth component of some input vector
  • averaged_perceptron_bc3259af3adac75db5612073fc5313095b6ea0f3.png - weight for the dth component of the inputs
  • averaged_perceptron_311bf31f836cbf61de75e9461effe47dc0f184eb.png - bias

Algorithm

Training

  • averaged_perceptron_126d98893c4a077b350d3609f88ad9758580b58e.png
  • averaged_perceptron_273313cb3da00efa2ab824b7903b3f64d5f294da.png
  • for averaged_perceptron_548b54644f321491e31dc6a2d466895f21af0814.png do
    • for all averaged_perceptron_2f02fca8de51e92f55960fb5c0ca76ce2caa0f65.png do
      • averaged_perceptron_4f6a2930872ab82c536f82dcbb7713466e66b182.png
      • if averaged_perceptron_1dd77d373ecb986727f73091fbbf96608e046ee1.png then
        • averaged_perceptron_3bfdcf0fe99a30139ef5066556e1130e678f157c.png
        • averaged_perceptron_cfda5415480346c7441cf2753effd2be29c4d308.png
      • end if
    • end for
  • end for
  • return averaged_perceptron_c940d90fd99c3c91c5cd17e17420f057e8c8f608.png

Test

  • averaged_perceptron_0db1ec2c6d954712435973232059166fc6a1c329.png
  • return averaged_perceptron_7ad24e56dcf69d5d9bef0171ba16005812a7c970.png