Technology
Navigating EV Manufacturing Quality with Karthick Ravichandran
Karthick Ravichandran’s core argument is one the automotive industry is learning under pressure: quality planning ...
Souvik Ghosh explains that modern AI workloads demand a radical rethinking of network architecture, where lossless, ultra-low-latency fabrics become as critical as GPUs themselves. By treating networks as code and automating design from Day-0, his approach enables deterministic, hyperscale deployments that eliminate congestion and idle compute waste. His work shows that infrastructure innovation—especially in networking—is now a primary driver of AI performance, not just a supporting layer.
The artificial intelligence boom is powered by more than just algorithms and data; it runs on a physical foundation of immense scale and complexity. As AI models grow, the demand for computational power has driven the evolution of datacenters into specialized ‘AI factories,’ where the network is no longer a simple data transport but a critical component of the computing fabric itself.
This shift has created a new set of engineering challenges, demanding unprecedented performance and reliability from the global infrastructure that underpins it.
Souvik Ghosh, an infrastructure networking leader with nearly two decades of experience, has designed and scaled some of the world’s most complex networks. As a dual CCIE with a history of leading global programs at hyperscale, his work focuses on developing the high-performance fabrics and automation frameworks necessary to support modern AI workloads and offers insights into the future of distributed AI.
Designing a network for AI training clusters requires a fundamental departure from traditional datacenter architectures. The massive, parallel workloads common in AI generate traffic patterns, known as elephant flows, that can overwhelm conventional network designs and stall expensive GPU pipelines.
According to Ghosh, the solution lies in creating a more predictable and stable environment. “Al clusters require a non-blocking, ultra-low latency network fabric,” he explains. This means moving beyond legacy models that break down under the strain of synchronized, high-volume data transfers, which are now recognized as a primary bottleneck in modern AI systems.
This architectural shift is not merely about adding more bandwidth; it involves rethinking the core logic of data movement. Ghosh notes, “This means moving away from oversubscribed leaf-spine models toward lossless RoCEv2 fabrics using mechanisms like Dynamic Load Balancing (DLB) and Deterministic Scheduling Fabric (DSF).”
These technologies are critical for ensuring that data flows evenly and without loss, a prerequisite for efficient, large-scale training, and underscore the difficulty of building AI/ML clusters on Ethernet compared to more specialized interconnects offered by companies like CoreWeave.
At the scale required for modern AI, manual configuration is not just inefficient; it is a significant source of risk and inconsistency. The speed at which GPU clusters must be deployed and expanded has driven a move toward comprehensive automation, treating the entire network as a single, programmable system.
Ghosh emphasizes that automation must be foundational, not an afterthought. “I’ve always believed automation starts at Day-0, not Day-2. If deployment isn’t automated, operations become perpetual catch-up,” he states. This approach aligns with modern practices for end-to-end lifecycle management in hyperscale environments.
The key innovation was to codify the entire network design, from individual nodes to the complete topology. “We rebuilt our process to treat the entire network as code. This intent model generated configuration artifacts programmatically, which were then validated and deployed through our existing ZTP (Zero-Touch Provisioning) system,” Ghosh adds.
This workflow, where design intent is systematically translated into configuration and deployment, is central to methodologies like Intel’s Day 0-2 automation framework and is supported by platforms from vendors like Nokia. “That foundation eliminated human drift, reduced cycles from days to hours, and gave us deterministic builds at hyperscale.”
For AI workloads, the term ‘lossless’ is not an ideal; it is a strict requirement. In the tightly synchronized communication patterns of distributed training, the loss of even a single packet can cause a cascade effect, forcing thousands of GPUs to wait idly and wasting immense computational resources.
“Al training workloads depend on synchronized, all-reduce communication — a single lost packet can stall thousands of GPUs,” Ghosh explains. To prevent this, engineers employ a combination of technologies designed to manage network congestion proactively rather than simply reacting to dropped packets. A well-designed fabric uses a combination of Priority-based Flow Control (PFC) and Explicit Congestion Notification (ECN) to signal and manage traffic pressure before it leads to packet loss.
However, these mechanisms are notoriously difficult to tune at scale, a challenge noted in research on congestion control in RoCEv2 networks and in practice, as seen in Meta’s experience with DCQCN. “This combination allows the network to behave like an extension of GPU memory, not a best-effort transport,” Ghosh says. Preventing side effects like PFC storms, where pause frames propagate uncontrollably, requires additional safeguards such as the PFC Watchdog feature found in modern network operating systems.
As the thirst for AI computers continues to grow, the industry is approaching the physical and power limits of single datacenter sites. The next frontier of scale involves connecting multiple datacenters into a single, cohesive fabric, allowing AI models to be trained across geographical distances.
This ambition introduces a new set of challenges, governed by the laws of physics. “The central question is: how do you preserve lossless, low-latency communication across geography while staying within the constraints of physics?” asks Ghosh.
A promising solution lies in optical circuit switching (OCS), a technology that can dynamically create high-bandwidth optical paths between locations. OCS is already being used in Google’s TPUv4 clusters and is the focus of industry-wide collaboration through efforts like the Open Compute Project.
The goal is to create a network that can operate as a unified whole, regardless of physical boundaries. According to Ghosh, “The real frontier is inter-datacenter coherence that acts like one logical cluster.” Leading hardware vendors are developing solutions to address this, and new OCS products from companies like Lumentum, based on different technologies such as MEMS and silicon photonics, are poised to enable this next wave of scaling.
Managing budgets that exceed $750M requires a disciplined approach to technology adoption, balancing the immediate needs of production environments with strategic investments in next-generation hardware. This involves carefully navigating the landscape of competing technologies, from established standards to promising but unproven innovations.
“It’s always a balance between operational certainty and strategic risk,” Ghosh says. The strategy involves running parallel tracks: one for deploying proven, production-ready technologies and another for R&D on platforms that are six to twelve months ahead of the curve. This allows for rigorous testing and validation, such as comparing the performance of different congestion control schemes or evaluating competing interconnects like InfiniBand and AWS EFA.
This dual-loop evaluation ensures that future investments are well-informed and that vendor roadmaps are aligned with long-term needs. “Vendor partnerships are structured to ensure our learnings influence their silicon roadmap early — so even if a new technology doesn’t deploy immediately, it evolves in our direction by the time it’s mature,” he notes.
The complexity of AI fabrics has rendered traditional network monitoring obsolete. It is no longer sufficient to poll device counters; operators need deep, real-time visibility into the behavior of individual data flows to diagnose and prevent performance degradation. This has led to the rise of advanced observability platforms and is an area where Ghosh holds a U.S. patent.
“Observability has evolved from device-centric polling to flow-correlated, packet-level visibility,” he states. Modern networks require cross-domain telemetry that aggregates signals from compute, storage, and the network itself. This is often enabled by hardware-accelerated telemetry features that provide deep insight into traffic behavior.
The ultimate goal is to move from reactive troubleshooting to proactive, automated remediation. “That allows us to build predictive models where the system learns what ‘healthy’ looks like and triggers auto-remediation before humans notice degradation,” Ghosh explains.
This concept is closely related to the principles of ModelOps and GitOps, where Gartner predicts that by 2025, 50% of all AI models will be managed using such practices to ensure scalability and effectiveness, often within automated frameworks like an Azure Pipelines baseline architecture.
Before focusing on AI infrastructure, Ghosh’s experience included engineering networks for major global events like the Rio Olympics and nationwide 5G buildouts. These projects instilled fundamental principles of resiliency and scale that are now directly applicable to the challenges of building AI networks.
A key lesson from this work is that infrastructure is not just a facilitator but a primary driver of innovation. “I wish more people understood that infrastructure innovation is as critical as model innovation,” Ghosh reflects. The breakthroughs in training efficiency often come from incremental improvements in the underlying hardware and software, a concept explored in advanced architectures like programmable hybrid optical networks.
This foundational work is often invisible when it is successful. “When the foundation is invisible, it’s working perfectly — but every model breakthrough stands on years of engineering that make the impossible seem routine,” he says. This includes complex systems like knowledge-defined network orchestration, which are essential for managing resources at an enormous scale.
As networks expand to exascale, engineers are confronting the hard limits of physics, particularly the speed of light. This has forced a paradigm shift in how architects think about latency, moving away from trying to eliminate it and toward designing systems that can tolerate it.
“The biggest shift was abandoning the idea that latency can always be ‘engineered away,'” Ghosh asserts. At a certain scale, the time it takes for a signal to travel becomes a non-negotiable architectural constraint. This reality has spurred research into novel hardware solutions, such as scalable optical circuit switching drivers and nonreciprocal optical switching, aimed at mitigating these physical barriers.
This requires a complete re-evaluation of how distributed systems are built. “This forced a rethinking of synchronization, buffering, and congestion models — a move from ‘reliable transport’ to ‘predictive transport,'” he concludes. This evolution in thinking is a hallmark of the engineers who are shaping the future of technology by mastering its fundamental constraints.
The continued advancement of artificial intelligence relies on the work of engineers like Ghosh, who are designing the global superhighways that power it. As AI systems become more integrated into our world, the stability, scale, and efficiency of this underlying infrastructure will be more critical than ever, forming the foundation upon which the next generation of innovation is built.
