Meaning and mechanism
The weight vector determines the direction of the classification boundary, while bias determines its position. For two input features, the boundary appears as a straight line.
By adjusting the weights and biases in an unclassified region, the boundary is moved in the direction of the correct answer. The boundary or speed achieved can vary depending on the order of learning and the initial values.
Relationships and conditions
The experiment updates w to w ← w + η(y − ŷ)x, and b to b ← b + η(y − ŷ). If the difference between the actual value y and the predicted value ŷ is zero, the update amount is 0.
The convergence properties of a standard perceptron require linear separability. For example, arrangements like those found in XOR cannot be linearly separated, and simply increasing the number of training iterations will not resolve them with a single-layer straight line.
What to observe in the simulation
Linearly separable data can be learned with a straight boundary.
This is a single-layer binary classifier. XOR is not linearly separable. Initialization and order affect convergence.
Key relationships and symbols
w ← w + η(y − ŷ)x · b ← b + η(y − ŷ)
| Symbol | Meaning | Unit |
|---|---|---|
| w, b | Weights and bias | Model parameters |
| η | Learning rate | Model coefficient |
| y, ŷ | Actual and predicted classes | 0 or 1 |
Worked example
At 100% accuracy, how many errors remain?
Zero · Linearly separable data can be learned with a straight boundary.
Common misconceptions
- The reason that learning gaps do not decrease is not always because the learning rate is small.