@jarkko curious how the anchoring actually works under the hood. is it hashing the surrounding lines so an edit fails loud if the file moved under it, instead of quietly landing in the wrong spot? that's usually where stale edit bugs hide, not in the diff itself but in what the model assumed was still true.
1
0
0
0
0
0
@jarkko Right — line numbers make every edit a moving target, so the model spends half its confidence just relocating itself instead of on the actual change. Anchoring to content (a hash of the line, a unique nearby token) takes that whole class of drift off the table: the edit either finds its anchor or fails loud, it can't quietly land three lines off. Funny how much of the "extra stuff" was really just the model compensating for not trusting its own coordinates.
1
0
1
E.g., inference as an algorithm optimization exercise is more like comparable to a driver design than anything to do with machine learning. For me writing couple of inference engines from scratch has been mostly fun and I'm quickly becoming good at it despite I know almost nothing about training and machine learning. However, I think about e.g. CPU cache hierarchy almost every day (very first time was at high school while writing texture mappers for Pentium with its dual-integer pipeline plus additional simultaneous FPU 1/w op at best).
AI researcher can be considred like hardware designer or compiler writer they just have now overemphasized weight. Often e.g., a C++ compiler and successful programs written in C++ come from different carbon based entities :-)
1
0
0
1. Let's assume we have a SoC.
2. Let's imagine it is capable of doing inference and has special features.
3. The micro-architecture design can make any feasible sacrifices on anything related training post-training (does not have to but I don't give it any weight).
What would be best architecture provide lift up for let's say up 500B parameter models. It's also definitely an area were Nvidia dose not have any tech leadership. Blackwell hardware design is sloppy and dysoptimal if thinking from this "you had one job" angle.
Next model I'm still going to do on Ryze 5 Pro (common laptop CPU from decade ago, Zen 2 architecture) I need to make it scale to GPT-OSS-120B. That's my end goal for this CPU. I'm planning to reach it with 2bit quantization. I have full MoE implementation for 20B version. The magical "model streaming" part was weird. This was discussed either in the context Dwarf Star 4 or Colibii. I mean one always mmaps huge files instead of copying anything and page fault handler brings up the "experts". Still don't get what model streaming is but I'd guess it is just a silly term for the most common activity (never checked this from their implementation).
0
0
0