Как Instacart Использует Физический Искусственный Интеллект для переосмысления Продуктового Магазина

Самым умным компьютером в вашем местном продуктовом магазине вскоре может стать тележка, с грохотом направляющаяся к отделу с хлопьями.

В течение многих лет розничные продавцы делали онлайн-покупки более персональными и удобными, в то время как обычный поход по супермаркету упорно оставался аналоговым. Покупатели по-прежнему ищут продукты, подсчитывают расходы и обнаруживают на кассе, что забыли йогурт.

Instacart хочет сократить этот разрыв. Система Caper Cart сочетает в себе камеры, датчики веса и местоположения, передовые технологии и сенсорный экран для распознавания товаров, отслеживания расходов и выдачи рекомендаций. Корзина является видимой частью более масштабной стратегии, объединяющей полки, системы инвентаризации, электронную коммерцию и поведение в магазине.

Дэвид Макинтош, директор по сетевым магазинам Instacart, сказал мне, что компания видит в этой технологии нечто большее, чем просто новинку. "Мы определенно считаем, что это может стать стандартным способом совершения покупок в магазине", - сказал он.

Онлайн-магазины могут видеть результаты поиска, оставленные корзины, покупки и повторное поведение покупателей. В обычном продуктовом магазине все выглядит более туманно. Он может знать, что продукт продан, но не иметь четкого представления о том, где покупатель его нашел или почему на полке отсутствует товар, который, по-видимому, имеется в наличии.

Продуктовые магазины создают особенно сложную среду для искусственного интеллекта. В большом магазине представлены десятки тысяч товаров, многие из которых в одинаковой упаковке. Витрины перемещаются, поставщики пополняют запасы, а покупатели оставляют товары в неожиданных местах. Инвентарные отчеты могут свидетельствовать об одном, а на полках - о другом.

Instacart обработала более 1,6 миллиарда заказов за весь период своей работы, что позволило ей получить глубокое представление о продуктовых продуктах, заменах и схемах покупок. Компания использует эти данные для создания ИИ-моделей для продуктовых магазинов, которые объединяют историю онлайн-заказов, данные каталога продуктов и растущие сигналы в магазинах. Обычная модель может распознать на фотографии банку с соусом для пасты. Модель, работающая в продуктовом магазине, также должна распознать ее при плохом освещении, знать, где она находится, определить, что осталось всего две, и понять, что покупатель может приготовить из нее.

The Caper Cart uses basket-facing cameras, outward-facing shelf cameras, a certified scale, location signals and an NVIDIA Jetson computer. Sensor fusion combines these inputs to form a reliable interpretation of what is happening.

Vision has blind spots. Weight helps when products obscure one another. Location connects an item to where it was added. The system must understand products in bags, customers removing items, uneven floor tiles and someone leaning against the basket.

McIntosh compares the challenge to a miniature autonomous-driving system. Each cart receives several streams of information and must interpret them within a fraction of a second. Processing happens on the cart because supermarket connectivity can be unreliable, and sending every interaction to the cloud would create an irritating delay. The systems respond within hundreds of milliseconds, he explained.

This is physical AI in practical form, perceiving and responding to a messy environment under fluorescent lights, on bumpy floors and during the Saturday morning rush.

Instacart says thousands of Caper Carts are live across more than 100 cities, with its Caper business tripling year over year. The carts generate millions of sensor inputs each day. Every unusual interaction helps the models handle more real-world edge cases.

"You can imagine even frontier models have never seen the inside of a basket in a grocery store," McIntosh said. They have no experience interpreting a changing weight signal as a cart rolls across floor tiles and a shopper leans on the handle. This specialized data is difficult to copy.

The flywheel now extends to the shelf. Instacart recently acquired Arpalus to accelerate Store View, its AI-powered shelf intelligence technology, which uses computer vision to turn shelf imagery into near real-time insights about inventory and availability v . Around 600,000 Instacart shoppers can capture shelf imagery using their phones, while Caper Carts gather more information as they move through aisles.

The result is a richer picture of product availability, display locations and purchases. Better shelf intelligence can improve online fulfillment, reduce substitutions and make recommendations more useful. A suggestion for a product in aisle five quickly becomes annoying when it has moved to aisle seven.

For shoppers, the value is refreshingly simple. The cart shows a running total, surfaces deals, weighs produce and tracks the basket. Customers can pay on the cart in some stores or transfer the order to self-checkout in others. They can leave everything packed rather than unloading a full weekly shop and putting it all back again.

Budget visibility may be the most important feature. McIntosh said more than 80 percent of North American shoppers are shopping to a budget. A running total gives customers control before checkout, including people using food assistance benefits who need clarity about eligible items and their remaining balance.

The commercial case is equally clear. A personalized ‘Did you forget?’ prompt shown as shoppers approach checkout drove nearly a 1% absolute sales lift, according to McIntosh. . The same screen can surface relevant discounts, discovery opportunities and where appropriate, sponsored recommendations at the point of decision. .

There is a tension here. A screen that helps someone control a grocery budget can also encourage another purchase. Retailers need transparent rules around data collection, personalization and sponsored recommendations. Customers should understand why they see a suggestion and retain meaningful control over their information. Trust will determine how far this experience can go.

The challenge reaches beyond model accuracy. "Every store is different. Every retailer is different," McIntosh said. One chain may sell unusual bakery products, another may offer returnable beer crates, and each has its own checkout processes and staffing patterns.

Instacart’s answer is modularity. The carts resemble familiar carts, charge when stacked and connect to existing checkout workflows. Store associates need training and a reason to support the system. A clever AI demonstration that complicates daily operations will soon become expensive furniture.

This lesson applies well beyond grocery. Physical AI must fit the environment, the workforce and the customer journey. Technical performance earns a pilot. Operational fit earns a rollout.

The next stage moves from detection to action. Shelf intelligence can already identify a missing or misplaced product. In the future, a gentic AI could coordinate with a supplier, alert an associate, adjust an order or recommend a better display location. Store managers could query conditions in natural language and receive suggestions based on live data.

Over time, this could develop into a simulation showing how location, inventory, promotions and customer movement interact. Human oversight will remain essential where decisions affect pricing, suppliers, staffing or customer access. Useful agents will handle routine coordination and escalate decisions requiring judgment.

Instacart is also digitizing prepared-food ordering, shelf labels and trip planning. As these systems connect, the store starts to operate as an intelligent network rather than a collection of isolated technologies.

McIntosh’s long-term vision is straightforward: "It’ll be one single unified mode that will be powered by Instacart."

In that future, a list created online follows the customer onto the cart. In-store purchases improve future online recommendations. Shelf scans improve ecommerce availability. A forgotten-item prompt uses purchase history, live basket data and the shopper’s location in the store.

The broader lesson is that physical locations can become a unique source of AI advantage. Ecommerce companies have spent years learning from digital behavior. Store operators have millions of real-world shopping journeys. Turning them into useful intelligence requires edge computing, specialized models, operational integration and clear customer safeguards.

The humble shopping cart may become one of AI’s most consequential enterprise devices. It already has one essential advantage: it goes wherever the customer goes.