FriendliAI Corp., a startup that helps developers speed up their artificial intelligence models, has raised $20 million in funding.
Capstone Partners led the seed extension round. FriendliAI detailed in its announcement of the raise on Thursday that Sierra Ventures, Alumni Ventures, KDB and KB Securities participated as well. The company previously raised $5 million in 2021.
FriendliAI offers a software platform called the Friendli Engine that it claims can reduce inference costs by up to 90%. It also boosts AI response times in the process. According to the company, the Friendli Engine provides those efficiency gains by applying low-level optimizations to customers’ AI workloads.
Large language models often process user requests in batches rather than one at a time. If a request is processed faster than other prompts in the same batch, the results are only delivered to users until those other prompts are answered. That delay can significantly slow down LLM response times.
FriendliAI has developed a processing technique dubbed continuous batching that it claims can address the issue. According to the company, the technology changes the order in which inference requests are processed such that unnecessary delays are avoided. It says that continuous batching can increase LLM throughput more than tenfold in some cases.
The company also uses other methods to speed up customers’ AI applications. Earlier this month, it introduced support for an AI processing technique called N-gram speculative decoding. It lets LLMs reuse data from past prompt responses when generating new output, which is more efficient than generating everything from scratch.
FriendliAI commercializes its technology with three offerings. The first, Friendli Container, allows organizations to run the company’s software on their private graphics card clusters. It also offers two cloud services that remove the need for customers to maintain infrastructure.
One of FriendliAI’s cloud services is built to perform inference using open-source AI models. The other offering, Friendli Dedicated endpoints, allows customers to use custom LLMs. It can automatically adjust the number of graphics cards assigned to a workload as inference requirements change.
Crunchbase reported that FriendliAI currently has “25 to 30 large clients.” Those customers are expected to help the company grow its revenue by up to 600% this year. It is reportedly not yet profitable but maintains “strong” gross margins.
The company will use its new funding to accelerate go-to-market initiatives in North America and Asia. The company also plans to enhance its inference software, as well as procure more graphics cards for its cloud services.
Image: Unsplash
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
- 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
- 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network.
About News Media
Founded by tech visionaries John Furrier and Dave Vellante, News Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
