[{"data":1,"prerenderedAt":85},["ShallowReactive",2],{"schrijven-summary-index-en":3},{"id":4,"title":5,"body":6,"date":74,"description":75,"draft":76,"extension":77,"image":78,"imageAlt":78,"meta":79,"metric":78,"metricLabel":78,"navigation":80,"order":67,"path":81,"seo":82,"stem":83,"__hash__":84},"en\u002Fschrijven\u002Fsummary-index.md","Why I built a summary index and threw it away again",{"type":7,"value":8,"toc":65},"minimark",[9,14,18,22,25,29,41,44,48,51,55],[10,11,13],"h2",{"id":12},"the-idea","The idea",[15,16,17],"p",{},"When you search with embeddings, you normally store every passage of a document. My idea was more frugal: let a language model write a short summary per file and store an embedding of only that. A question about the contents then finds the file, without the full text having to be in the index.",[10,19,21],{"id":20},"the-test","The test",[15,23,24],{},"I built it as a trial and measured two things: how long it takes and whether the right files end up on top.",[10,26,28],{"id":27},"what-i-measured","What I measured",[15,30,31,32,36,37,40],{},"Writing a summary cost ",[33,34,35],"strong",{},"7.7 seconds per file",". Embedding the full text cost ",[33,38,39],{},"0.25 seconds per file",". So the summary index was roughly 30 times slower, for an index that holds less information.",[15,42,43],{},"Concrete questions found the right file well. Vague questions did not: their scores were so close to those of a completely unrelated question that a threshold could not tell them apart.",[10,45,47],{"id":46},"why-i-threw-it-away","Why I threw it away",[15,49,50],{},"It was slower, more expensive and no better on vague questions. The one advantage, storing less text, did not make up for that. The code is gone and the numbers are kept, so the next idea can be compared against them.",[10,52,54],{"id":53},"what-i-learned","What I learned",[56,57,58,62],"ul",{},[59,60,61],"li",{},"An idea that sounds frugal can be the most expensive one in practice. Measuring costs an afternoon, building by feel costs far more.",[59,63,64],{},"Keep the numbers of what you throw away. They are the baseline for the next idea.",{"title":66,"searchDepth":67,"depth":67,"links":68},"",2,[69,70,71,72,73],{"id":12,"depth":67,"text":13},{"id":20,"depth":67,"text":21},{"id":27,"depth":67,"text":28},{"id":46,"depth":67,"text":47},{"id":53,"depth":67,"text":54},"2026-11-10","A summary per file looked like a clever way to make documents findable. The measurement said otherwise.",false,"md",null,{},true,"\u002Fschrijven\u002Fsummary-index",{"title":5,"description":75},"schrijven\u002Fsummary-index","negmMUks6WpQNRppaq4aNcw5ejQtajq_MoxDrsBFrzM",1794308981477]