How to build a predictive coding model from scratch

Predictive Coding from Scratch

This post is a tutorial on how to build a predictive coding model from scratch. This is something I had to do for my MSc Hillarty Term project.

The code will all be writen in python and will be using just the numpy library.

Lots of the ideas about predictive coding are written in a very abstract way which made it hard for me to to follow what the differential equations that update the parmeters of the model are actually doing. As such, I thought i would write a tutorial on how the very basics of predictive coding.

What is Predictive Coding?

A classification task

• How does classification work?
• Credit assignment problem

What are the steps of predictive coding?

1. Inference (dx)

2. Learning (dw)

3. Predicition This is the simpliest part of the model.

   Recommded Reading for understanding the ideas

   • Rafal’s Tutorial on Predictive Coding
   • Yuhang’s Paper on Predictive Coding