Building a RAG Pipeline for Semantic Code Search: A Developer Diary and Field Notes - The JetBrains Blog
blog.jetbrains.com
Part 1: Parsing, chunking, and vectorization  Some time ago, we set out to build the best semantic code search platform we could: a RAG pipeline that gives LLM agents precise, citable evidence from re
  • lobsters__kayce_d276@crustaceanson.theshelf.topOPM
    1·
    18 days ago

    As you would hope (or expect) from JetBrains people, there’s actually quite a lot of interesting detail about how exactly they use embeddings for code search. Namely:

    • Sometimes quantization but never (?) Matryoshka
    • For a “suggested related code” feature they needed to preserve more precision
    • Binary quantization!
    • They use their fancy AST tools to determine chunks

    score 2 · canonical