Co-founder and COO Robert Wachen told TechCrunch that investors are so enthusiastic because Etched has designed two new components from scratch to speed up inference — the computing process that happens after a user submits a prompt.
“Inference is built in two stages,” Wachen said, “prefill and decode.” In the mathematically and compute-intensive “prefill phase,” the system must understand the prompt, including context. In the memory-intensive “decode” phase, the system generates output tokens, meaning the actual answer the user sees.
Etched created a prefill chip that operates at low voltage, allowing it to pack in more transistors without the typical heat problems of other high-end AI chips. It can therefore process more tokens faster. Etched created a new type of memory and an interconnect for the decode process that the company calls cluster-scale memory.
“It allows many chips to connect together and use a shared memory pool at a very, very fast, low latency,” Wachen said. The result, Etched promises, is higher speeds and lower costs.
Etched is still battling the perception from its early days that it etches a particular model into its chips, meaning that each chip is somehow custom-designed to run one frontier model. That was its original intention but is no longer the case. Etched’s systems can run any frontier model.
In the blog post announcing the new round, investment firm Jane Street said, “We tested the chip and are pleased with the early results. Etched’s unique approach to inference delivers the precision we will need to support our most demanding workloads. We’re excited to now have our own rack running in our datacenter.”
Other investors in Etched include Kleiner Perkins, Sequoia Capital, Andreessen Horowitz, Peter Thiel, Tiger Global, Bain Capital Ventures, Neo, Stripes, Primary, Positive Sum, Diffusion, Argo, and Blackstone.
