Facebook's Meta Platforms to Revolutionize Data Centers with New Custom Chip

Facebook’s Meta Platforms to Revolutionize Data Centers with New Custom Chip

In a move poised to shake up the tech industry, Meta Platforms, the parent company of Facebook, is gearing up to launch a new version of its custom chip designed to support its artificial intelligence (AI) initiatives. This second-generation chip, following the in-house silicon line announced last year, is set to reduce Meta’s reliance on Nvidia’s dominant chips and tackle the mounting costs associated with running AI workloads.

As Meta continues to push its power-hungry generative AI products into popular apps like Facebook, Instagram, and WhatsApp, as well as hardware devices like Ray-Ban smart glasses, the need for increased computing capacity becomes paramount. To meet this demand, Meta has invested billions of dollars in specialized chips and the reconfiguration of its data centers. However, the scale of Meta’s operations necessitates a cost-effective solution in order to optimize energy consumption and reduce chip purchasing expenses.

According to Dylan Patel, founder of the silicon research group SemiAnalysis, successfully deploying its own chip could save Meta hundreds of millions of dollars in annual energy costs and billions in chip purchasing costs. Tech companies have found themselves in a challenging situation whereby the expense of chips, infrastructure, and energy required to run AI applications offsets gains made in the technological advancements of AI.

Confirming the plan to put the updated chip into production this year, a Meta spokesperson explained that it will work in conjunction with off-the-shelf graphics processing units (GPUs), which are currently the go-to chips for AI. The spokesperson emphasized that Meta’s internally developed accelerators will complement commercially available GPUs, delivering an optimal balance of performance and efficiency for Meta-specific workloads.

In a recent announcement, Meta CEO Mark Zuckerberg revealed the company’s intention to obtain around 350,000 flagship “H100” processors from Nvidia by the end of the year. This would provide Meta with a compute capacity equivalent to 600,000 H100s when combined with supplies from other suppliers. The deployment of Meta’s own chip aligns with this plan and marks a positive progression for the company’s in-house AI silicon project.

Initially, executives made the decision to halt the development of the first iteration of the chip in 2022, opting instead to purchase billions of dollars worth of Nvidia’s GPUs due to their near monopoly on the AI training process. Training involves feeding vast datasets into models to teach them how to perform various tasks. However, the new chip, internally referred to as “Artemis,” can only perform a process known as inference. Inference entails using algorithms in the models to make ranking judgments and generate responses to user prompts.

It is worth noting that Meta is also working on an even more ambitious chip, which, like GPUs, would be capable of performing both training and inference tasks. The company shared details about the first generation of its Meta Training and Inference Accelerator (MTIA) program last year, implying that chip as a learning opportunity. If successful, an inference chip could prove significantly more efficient in handling Meta’s recommendation models compared to the power-hungry Nvidia processors.

Dylan Patel emphasizes the potential for substantial cost savings through the utilization of an inference chip, stating, “There is a lot of money and power being spent that could be saved.”

As Meta Platforms takes control of its AI initiatives by developing its own custom chip, the tech landscape stands on the edge of transformation. Through increased computing capacity, improved energy efficiency, and reduced costs, Meta aims to push the boundaries of AI and forge new paths in the digital realm.