[{"data":1,"prerenderedAt":164},["ShallowReactive",2],{"demo-xor-en":3},{"id":4,"title":5,"body":6,"date":151,"description":152,"draft":153,"extension":154,"image":155,"imageAlt":156,"meta":157,"metric":151,"metricLabel":151,"navigation":158,"order":159,"path":160,"seo":161,"stem":162,"__hash__":163},"en\u002Fdemos\u002Fxor.md","A network that learns XOR",{"type":7,"value":8,"toc":144},"minimark",[9,14,18,75,79,91,94,98,101,141],[10,11,13],"h2",{"id":12},"what-xor-is","What XOR is",[15,16,17],"p",{},"XOR is a rule with two inputs. The result is 1 when exactly one of the two is 1.",[19,20,21,37],"table",{},[22,23,24],"thead",{},[25,26,27,31,34],"tr",{},[28,29,30],"th",{},"x1",[28,32,33],{},"x2",[28,35,36],{},"XOR",[38,39,40,50,59,67],"tbody",{},[25,41,42,46,48],{},[43,44,45],"td",{},"0",[43,47,45],{},[43,49,45],{},[25,51,52,54,57],{},[43,53,45],{},[43,55,56],{},"1",[43,58,56],{},[25,60,61,63,65],{},[43,62,56],{},[43,64,45],{},[43,66,56],{},[25,68,69,71,73],{},[43,70,56],{},[43,72,56],{},[43,74,45],{},[10,76,78],{"id":77},"why-one-neuron-cannot-do-it","Why one neuron cannot do it",[15,80,81,82,86,87,90],{},"One neuron computes a weighted sum, ",[83,84,85],"code",{},"z = x1·w1 + x2·w2 + b",", and passes it through a sigmoid. All inputs for which ",[83,88,89],{},"z"," is exactly 0 lie on a straight line. On one side of that line the output is above 0.5, on the other side below.",[15,92,93],{},"In XOR the two ones sit diagonally opposite each other, at (0, 1) and (1, 0). No straight line separates them from (0, 0) and (1, 1). That is why there is a hidden layer in between. Each hidden neuron draws its own line, and the output neuron combines those lines. You can see it in the map above: the area becomes a band instead of a half.",[10,95,97],{"id":96},"how-it-learns","How it learns",[15,99,100],{},"Every step goes like this:",[102,103,104,112,122,132],"ol",{},[105,106,107,111],"li",{},[108,109,110],"strong",{},"Forward."," For each of the four examples: weighted sum and sigmoid in the hidden layer, then the same in the output neuron.",[105,113,114,117,118,121],{},[108,115,116],{},"The loss."," Per example ",[83,119,120],{},"(output - target)²",", averaged over the four. That is the number in the chart.",[105,123,124,127,128,131],{},[108,125,126],{},"Backward."," With the chain rule, each weight gets a number for how the loss changes when that weight grows a tiny bit. Through a sigmoid that is always ",[83,129,130],{},"output · (1 - output)",".",[105,133,134,137,138,131],{},[108,135,136],{},"Adjust."," Every weight moves a small step against that direction: ",[83,139,140],{},"weight = weight - learning rate · gradient",[15,142,143],{},"Use \"One step\" to watch a single round. With 2 hidden neurons the network sometimes gets stuck, depending on the start weights. Press \"Reset\" then.",{"title":145,"searchDepth":146,"depth":146,"links":147},"",2,[148,149,150],{"id":12,"depth":146,"text":13},{"id":77,"depth":146,"text":78},{"id":96,"depth":146,"text":97},null,"A small neural network in your browser. Press start and watch it adjust its weights with backpropagation until it can do XOR.",false,"md","\u002Fdemos\u002Fxor-en.jpg","The network halfway through learning, with the weights as coloured lines and the output per neuron.",{},true,1,"\u002Fdemos\u002Fxor",{"title":5,"description":152},"demos\u002Fxor","fLd7YvxFnqkwqASAEFt6nhu938q9QTX2BEo0G5yyb2o",1794308981402]