Pinterest cut AI costs 90% by gutting a frontier model's vision layer
Pinterest says it cut AI costs 90% and improved accuracy 30% by replacing part of Qwen3-VL with proprietary embeddings.
Intelligence analysis by GPT-5.4 Mini
Pinterest is reworking frontier open-source models to better fit its product and data. The company says a custom multimodal approach lowers inference cost, improves latency, and makes visual discovery more personalized.
Pinterest is like a giant idea board with hundreds of millions of people using it. To make suggestions smarter, it took a big AI model and swapped out one expensive part for its own home-made version.
That change helped the system work faster and cost much less. It is a bit like replacing a gas-guzzling engine part with a smaller one that fits the same car better.
Pinterest says the new setup also helps it guess what people might like next, not just what they clicked before. That makes its recommendations more useful when someone is still exploring ideas.
Analysis
What Pinterest changed
Pinterest CTO Matt Madrigal says the company has been pushing to customize open-source models more deeply inside the business. In the case of its conversational shopping assistant, Navigator 1, Pinterest used Qwen3-VL but did not leave the model intact. The team removed the vision encoder layer and replaced that part with proprietary multimodal embeddings built from Pinterest’s own data.
Why that mattered
Those embeddings let Pinterest precompute and refresh information about pins, images, and metadata offline instead of encoding every image at runtime. Madrigal said that approach improves runtime performance and inference efficiency, and he also said it reduced costs by 90% while improving accuracy by 30%. He added that doing image encoding one by one at runtime would make inference about 20 times worse on latency.
The product logic
Pinterest’s broader system is built around discovery rather than direct search. Madrigal described a “taste graph” that tracks evolving user preferences and helps move people from inspiration toward intent. The company says this is not a social graph; it is a preference graph that uses user embeddings and activity signals to keep recommendations fresh. That architecture is meant to help Pinterest show more relevant ideas and products as users move through the funnel.
What this signals
The article frames Pinterest as a company that prefers open-source models plus heavy in-house customization for important features. Madrigal argues that when data is unique and high quality, it can outweigh model size. For AI startups, the takeaway is blunt: if the use case is central, scalable, and data-rich, owning the embedding layer and the retrieval path may matter as much as the model itself.
Key points
- Pinterest says it cut AI costs by 90% and improved accuracy by 30% on a visual discovery setup.
- The company modified Qwen3-VL by removing its vision encoder and using proprietary multimodal embeddings instead.
- Pinterest says precomputing image and metadata context offline improves latency and runtime performance.
- The system supports a broader “taste graph” designed to track evolving user preferences and guide discovery.
- The article presents Pinterest as favoring open-source models that are heavily customized in-house.
