The journey into building AI-driven SaaS took us into uncharted waters, especially once we realized how badly these old-school industries needed it. In part 1 of this series, I talked about the roadblocks we hit in building our first ERP product. Now, let’s dive into how AI helped us pivot and unlock new opportunities in an industry that’s still trying to do things the old-fashioned way.
Where It All Started
When we kicked off, e-commerce seemed like the obvious solution for distributors. We built tools to modernize their outdated order-entry processes, but we quickly hit a wall—chefs still preferred picking up the phone to call suppliers. They wouldn’t budge from their old-school ways, and no sleek online system was going to change that overnight. We needed to meet them where they were.
So we went back to the drawing board and started calling up all the operators served by our supplier customers – everyone from active e-commerce app users, churned users, to chefs who vehemently opposed the idea of an app – to ask what they really thought. As it turned out, ordering from an app wasn’t the 10x-better experience we envisioned over calling/texting/emailing, unless we could somehow give them live visibility into product availability and delivery status, a problem we deemed too difficult to solve with a software product alone.
At the same time, we know wholesalers are still drowning in hours of manual order entry every night. How do we automate away all this tedious work? That’s when it clicked – large language models are a perfect fit to handle such workflows with unstructured data. AI dramatically changes the cost equation so we can automate operations that were previously too messy to – it’s the missing piece that may finally bring a technologically underserved industry like food distribution into the modern era. It’s truly a paradigm shift: around 80% of the world’s data is unstructured.
So, we shifted gears. Rather than pushing both suppliers and operators to adopt fully digital workflows, we built AI-powered tools like Butter’s AI Order Assistant that complemented their existing process.
Making the Pivot with AI (More than Promises)
We learned pretty quickly that the success of AI wasn’t about building a “sexier” product — it was about making sure it actually completes user tasks. The AI Order Assistant didn’t ask chefs or distributors to reinvent their process. It adapted to what they already knew, slotting right into their workflows.
That’s all it takes. By building AI capable of processing natural-language orders (think voice commands or texts), we made the process easier, not harder. And because it was an add-on, not a full system replacement, distributors were quick to adopt it. Within weeks of launching, dozens of suppliers and ERP partners expressed interest. They saw it as an easy upgrade without the headaches that typically come with “digital transformation.”
When a client onboards, we connect with their order desk email & voicemail inbox, and automatically start converting incoming client orders into structured purchase order entries, leveraging each customer’s order guide content as well as their historical order patterns (read from their ERPs). For the first time ever, knowledge of a chef’s preferences was finally transferred from sales rep Joey’s head to a digital system — when a chef simply orders “2 cases of shrimp,” the system can accurately understand if they mean “4-6 Tiger Shrimp Frozen,” not “16-20 EZ Peel Shrimp,” or the 80 other shrimp product variations sold.
Knowing that the AI suggestions won’t be 100% perfect, we conducted extensive UX interviews and ensured users can easily correct the model output so it gets it right the next time. Critically, we made sure everything can be performed with keyboard input alone, since they heavily rely on similar hotkeys in ERP systems to enter hundreds of orders. Users loved the experience and quickly jumped on board. The end result? Order processing time got reduced by over 96%, and supplier clients could reduce back-office staff count, or uplevel them to perform higher-value tasks like quality control and customer relations management.
After Butter was acquired by GrubMarket, we took the AI Order Assistant model and scaled it into GrubAssist. This tool sits atop existing ERPs, providing natural-language business intelligence and analytics. It integrates painlessly into what the food industry knows and uses. And as anyone familiar with warehouse receiving or BOH knows, uninterrupted workflows are the only things keeping people sane.
Takeaway: Start with an AI solution that fits into existing workflows, without overhauling everything. Ease of integration and familiarity are key to faster adoption.
Lessons from Building LLM Product
-
Design around tech limits. LLMs are powerful but still maturing; they can lag or miss the mark on reliability. Clever design can hide some of the technical shortcomings. For example, since restaurants/retailers place their orders for the next day ahead of time, we can afford to process them in the background (before the supplier staff comes to work in the early mornings), opting for models with greater reasoning capacity while sacrificing some speed.
-
Speed before perfection… In early stages, don’t get bogged down in finding the “perfect” model. Use what’ll get you to market first. Simple techniques like RAG work surprisingly well if you give it the right context. If built the right way, AI-infused products automatically get better on their own when the underlying foundation model improves.
-
…but nail the foundation. Experimentation needs flexibility. Create a modular architecture so you can swap out models or features, and integrate clear, quantifiable in-product feedback systems — “building by vibes” doesn’t cut it. Your architecture should give you a solid iteration speed advantage.
-
Your interface can make (or break) your product. Even with a “perfect” model, start with the assumption that 20% of a task will need a human in the loop to QC. Make this interaction as simple and intuitive as possible, or you’ll lose user buy-in fast. The more you empower the human in the loop, the more you can leverage them to improve your product.
-
Capture unstructured knowledge. In old school industries, vital knowledge isn’t digitized—it’s in people’s heads. If customer preferences exist only in Joey the Sales Rep’s mind, create an interface to capture it. These insights strengthen and differentiate your model, giving it a constantly evolving data advantage.
-
Feedback loops drive accuracy. Engineering takes your AI product far, but feedback takes it further. Provide a seamless way for users to input feedback directly within the product, and combine this with a tuning engine to drive more accurate, contextually relevant outputs.
The Secret of Working with Legacy Systems
Here’s the hard truth: no matter how great your AI solution is, you still need those old-school ERP players to want to integrate with you. It’s not enough to develop cutting-edge AI if it can’t communicate with the systems that distributors already rely on. And when you step in like you’re trying to replace them, you become impossible to work with.
In our case, we needed legacy ERPs to enable integrations through methods like EDI (Electronic Data Interchange) or SFTP file exchange. These legacy systems are deeply embedded, and convincing (then architecting) them to connect with new AI tools isn’t always easy. But we found a sweet spot by offering an add-on that actually improves their existing product, encouraging clients to stick to their existing infrastructure, while getting all the benefits of AI. The magic was in showing both the business and their infrastructure provider how our AI was a net positive, without making them feel like they needed to scrap everything or lose a partnership. Don’t get cut out by overlooking the existing network and overplaying your hand.
That said, the golden window for this kind of integration is closing fast. AI expertise is spreading, and even the slower, old-school service providers are getting into the game. You’ll need to act quickly, find your angle, and work with the existing players.
For industry incumbents, beware of new software solutions that take an integrate and surround approach. These are products that provide a fully self-contained business unit (e.g. field sales) and shift the cost/revenue equation significantly in their favor. Understanding these dynamics early on is key to choosing the right partners.
Takeaway: Work alongside legacy systems, showing clear benefits and enhancements that don’t force a complete overhaul. Help them see the value of a low-risk, high-reward addition.
Takeaways for the Future
These traditional sectors that have relied on unstructured data—like handwritten logs and audio records—are finally accessible to modern tech solutions, thanks to LLMs. Vertical SaaS is quickly becoming more viable across these industries, and it’s tempting to apply AI to everything.
Still, remember that AI’s success doesn’t come down to the tech alone. The key challenge still lies in achieving product-market fit. While AI breakthroughs open doors, they don’t change the fundamentals of product development. Start by developing a clear understanding of your users and their needs; the technology will follow.
Looking back, the biggest lesson we learned was that AI succeeds best when it fits into existing processes, not when it tries to upend them. The question is, who’s going to seize the opportunity before it’s gone? There’s a little more to the story. In part 3, I’ll explore how combining a rollup strategy with AI creates a winning formula for SaaS in traditional industries.
