AI inference startup Baseten closes $150M investment backed by CapitalG – News

Artificial intelligence startup Baseten Labs Inc. today announced that it has closed a $150 million late-stage investment at a $2.15 billion valuation.

BOND led the Series D round, which comes about six months after the company’s previous raise. It was joined by Alphabet Inc.’s CapitalG fund, Conviction, Premji Invest, 01A, IVP, Spark, Greylock and Scribble Ventures.

Baseten offers a software platform that promises to help enterprises speed up their AI inference workloads. According to the company, its technology provides up to 50% faster performance than competing products. Customers can deploy the software on their own infrastructure or use a cloud-based version managed by Baseten.

The cloud version runs across 10 different infrastructure-as-a-service platforms. When traffic to an AI application increases, Baseten automatically provisions additional graphics cards on the most suitable platform. If one of the public clouds used by a customer experiences technical issues, the software can automatically switch to another.

The company uses an approach called topology-aware parallelism to optimize customers’ infrastructure. When an AI model runs across multiple graphics cards, those graphics cards must coordinate their work by exchanging data over the network. Topology-aware parallelism reduces data traffic and thereby lowers hardware usage.

Baseten says it optimizes not only the hardware on which customers’ AI models run but also the models themselves. It uses an approach called operator fusion to combine calculations that are usually done separately into a single computation, which saves time. A quantization tool shrinks neural networks to reduce their memory requirements.

The core inference features are offered alongside a set of developer tools. Software teams can use those tools to automate some of the work involved in releasing AI models to production. Baseten also makes it easier to deploy a model’s dependencies, or the software module on which it relies to run.

Once a model is deployed on the platform, developers can use a built-in observability tool to monitor its performance. The software tracks the number of requests sent to the model, response times, hardware usage and related metrics.

In May, Baseten expanded its focus beyond inference by launching an AI training service. It provides access to infrastructure that companies can use to build new AI models. It also allows software teams to periodically save AI models during training for backup purposes. If the training process is interrupted, developers can recover the most recent backup instead of starting from scratch.

“Every breakout AI application depends on fast, reliable, and cost-effective inference, the same way the last 15 years of companies depended on the cloud,” said co-founder and Chief Executive Tuhin Srivastava.

The company will use its new funding to expand its lineup of developer tools. Baseten also plans to research new methods of speeding up AI models.

Image: Baseten