Notes from Open Source AI Summit SF
Some afternoons turn into an unplanned family reunion. I walked into Open Source AI Summit SF expecting a lineup of talks. What unfolded instead was three generations of one idea taking turns at a single 🎙️ microphone.
Peter Norvig, co-author of the textbook much of the field learned from, told the story of how machines learned to learn. Illia Polosukhin, NEAR Protocol founder, one of the eight authors of “Attention Is All You Need,” explained why he thinks the future belongs to agents that hire other agents. And as the afternoon wound down, Adrian Kosowski of Pathway made the case that the architecture Polosukhin helped invent has started to hit its limits. Pathway CEO Zuzanna Stamirowska watched from the audience.
The lineup made the point without anyone stating it.

Peter Norvig: Down a Hall, 2017
Norvig opened with a small confession. His slides still carried the logo of another conference from two weeks earlier, because the host had asked for a rerun. He noted that things had already changed since.
His history of modern AI runs through his own lunch table. Around 2012, Geoff Hinton stopped by waving a laptop, eager to show him something that finally worked. It was AlexNet. Norvig recalls putting through one of Google’s early purchase orders for a GPU machine for researchers.
Then came 2017. A colleague down the hall was excited about a new paper. Norvig read it, thought it was kind of good and went back to his own project. “I was working on something else that went absolutely nowhere,” he said. That paper introduced the Transformer. One of its authors would share a stage with him later that afternoon.
Norvig’s longer arc starts in 1980s Berkeley, where building AI meant hand-coding inference procedures, user models and decision theory, powered mostly by the blood, sweat and tears of graduate students. Data was an afterthought. His 2009 paper with Alon Halevy and Fernando Pereira, “The Unreasonable Effectiveness of Data,” marked the turn. Progress stopped coming from ingenuity and hand coding and started coming from data plus search over enormous spaces of possibility. Today the machine exists. People mostly turn the crank.

Actual book. Real room, real slide, AI-touched-up. Photo mashup by author.
Mathematics makes the cleanest scoreboard. In 2022, language models fumbled three-digit arithmetic and could not count the letters in “strawberry.” Then came olympiad silver and gold. This year, problems that had stood for decades started to fall. Norvig cited the share of math preprints crediting AI climbing from about four percent in April to roughly a quarter by August.
Why math first? Because a proof can be checked. A formal verifier separates right from wrong in a way a policy proposal never can, and sometimes the answer fits in a tweet. AI also gets to be a specialist in every branch at once, stitching together results that human experts keep in separate rooms. Norvig asked whether that counts as creativity or exhaustive search, and reached for AlphaGo’s move 37 as the case that blurs the line. His framing: AI is filling in the convex hull of mathematics. The open question sits outside the hull.
Coding followed the same curve, from skepticism to routine. Yet Norvig drew a line. Software is more than code. It carries an unwritten theory of what a program is for and where it is headed, and when the original team leaves, projects often collapse even when the newcomers are talented. Coding agents do not hold that theory yet.
In his later fireside with Polosukhin, he brought the stakes down to earth. At Recursive, he said, probably half the effort goes to fighting reward hacking. His worry is not Terminator robots. It is whether a matrix multiplication algorithm the system came up with is actually any good. The bigger debt, in his view, is a digital economy built on convenience rather than security, a bill that agents are about to present.

Real room, real slide, AI-touched-up. Photo by author.
Illia Polosukhin: Specialists for Hire
Polosukhin left Google in 2017, fresh off the Transformer, to build program synthesis. The GPUs were not ready. Paying and coordinating people around the world turned out to need new infrastructure, which became NEAR Protocol, and crypto absorbed much of the work that followed.
His read on the present is simple. For years AI lived in the backend of recommendation feeds and ad prediction. ChatGPT pulled it front and center. The next step, he argues, is not one model that does everything but specialized agents that hire each other. NEAR’s Agent Market is his attempt at a matching and settlement layer for agent-to-agent work, with one slide reducing the thesis to a single line: specialized agentic economy over generalism.
Imagine sourcing a small production run from a factory in China. The website is in Chinese, the specs go by email, the supplier may only take WeChat Pay and there is no easy way to know if the company is real. Agent Market wraps that in escrow, refunds and dispute handling, connects to existing assistants over MCP, accepts card or crypto payments and runs inference inside confidential compute so operators cannot see user prompts.

Earlier, in a fireside with PyTorch CTO Matt White Matt White on the benefits and risks of open models, the two circled alignment. White called open AI dual use in the way Linux is dual use, argued the benefits still outweigh the risks and warned that capability keeps outrunning alignment work. Then he named who pays: “Liability is going to rest on whoever runs those agents, not who developed them, not who trained the models, but who is operating them.”
Polosukhin pushed back on the premise: “On alignment, my position is that you actually cannot align it in kind of a general sense. Because, like, we as individuals aren’t aligned.” His alternative is alignment to the individual user, closer to personalization than policy. Later, with Norvig, he upgraded Steve Jobs’ bicycle for the mind into a spaceship, one fast enough to burn up in the atmosphere if it is aimed wrong. His guardrail of choice: formally verified properties of what a system is meant to do.

Adrian Kosowski: Post-it Notes
Kosowski closed with an argument about the architecture in the room’s family tree. AI has lived through two scaling epochs, he said. The first, roughly 2020 to 2024, was scale, scale, scale: more parameters, more data, until returns diminished. The second began when OpenAI’s o1 shifted the weight to test-time compute. Reasoning models now generate more tokens, though not necessarily more intelligence, and that curve is bending too.
His thesis: compute scaling alone will not deliver the next leap. Architecture will. He named three bottlenecks. Token history keeps growing. Serial work repeats. Experience stays temporary. In a Transformer, a longer context window is not the same as learning from experience. Each session starts over and rebuilds context from scratch. One of his slides put it more bluntly: “Learnings are little more than a post-it note.”
Pathway’s answer is BDH, a post-transformer architecture that builds memory, adaptation and reasoning into the model itself - github.com/pathwaycom/bdh It has three properties:
-
Persistent adaptive memory. New information updates an internal state that carries across sessions.
-
Recurrent latent reasoning. The model iterates on internal representations before producing any output. It thinks before it speaks.
-
Unlimited horizon. No sharp context bounds, so refinement and learning can continue over long periods.

Real room, real slide, AI-touched-up. Photo by author.
The implications run past model design. A new architecture changes how hardware gets used, which means GPU and ASIC bets, KV cache habits and hand-tuned CUDA kernels all get rethought. Pathway pointed to hardware leaders already treating model architecture as strategy.
Three Threads
Verification is that quiet type hero. Norvig credits formal checking for AI’s leap in math. Polosukhin wants formally verified guardrails. Norvig’s own slide on processes that need reimagining puts testing and verification, code review and supply chains near the top. Reward hacking is what happens when a system optimizes an unverified number.
Specialization cuts both ways. Norvig sees AI as a specialist in every field at once. Polosukhin sees a market of specialists, each doing one job well. Kosowski wants smaller, faster models that keep general versatility. Each describes the same tension from a different floor of the building.
Memory is the frontier. Norvig worries that coding agents lack the unwritten theory a team carries. Mathematicians publish polished proofs and hide the crossed-out drafts, while code keeps its history. Kosowski says today’s models keep little more than a sticky note between sessions.

Which leaves a question I carried out the door into the beautiful Presidio evening. If the next architecture finally lets models remember, what will they remember? The finished proof, or the drafts that led to it? The code, or the unwritten theory behind it? And when an agent holds that memory, whose theory will it be?
Thanks to Erica Kang & team for such a memorable summit.
bsky.app/profile/did:plc:ex3p6sdow4bnhp3wz76xcc6i/post/3mylcddybz22d
twitter.com/schwentker/status/2113769547870412824
