Redditor acquires decommissioned Netflix cache server with 262TB of storage

Contents

1 Redditor acquires decommissioned Netflix cache server with 262TB of storage

Interestingly, the now-defunct dial-up online service Prodigy used a local caching system to distribute data more efficiently using the same basic principle as Open Connect in the 1980s and ’90s. Instead of streaming video, that service merely served text data and vector graphic NAPLPS files. Times have changed, but we still want speedy data.

A look under the hood of the most successful streaming service on the planet

Netflix’s secret sauce is something none of us ever see

Nov 17, 2021, 5:23 PM UTC | Comments

Share this story

As hundreds of thousands of households worldwide tapped into Netflix’s Squid Game last month, viewers may have taken something fairly extraordinary for granted. Netflix didn’t buckle under the unprecedented demand for the dystopian drama that would become its most successful title to date — even as other services have struggled to keep their products sturdy under less demanding circumstances.

When many of us fire up our favorite streaming services, we often bump into various fury-making problems: stuff freezes, controls don’t work, or the service crashes entirely. None of these are ideal, but all seem to have become a widely understood cost of cord-cutting. For example, Disney Plus crashed its very first day because its software couldn’t handle the demand (and then it buckled again under demand for WandaVision). HBO Max is so fundamentally broken that its own leadership has admitted that the app is a mess. Even Instagram, whose Stories feature makes it a kind of streaming service in its own right, crashes so frequently it’s started alerting its users when it’s borked. Streaming can be maddening!

A service’s guts, the engineering behind the app itself, are the foundation of any streamer’s success, and Netflix has spent the last 10 years building out an expansive server network called Open Connect in order to avoid many modern streaming headaches. It’s the thing that’s allowed Netflix to serve up a far more reliable experience than its competitors and not falter when some 111 million users tuned in to Squid Game during its earliest weeks on the service.

“Anyone who wants to improve performance is going to try to put a server as close to the end user as possible.”

“One of the reasons why Netflix is the leader in this market and has the number of subs they do [. ] is something that pretty much everybody outside of the technical part of this industry underestimates, and that is Open Connect,” Dan Rayburn, a media streaming expert and principal analyst with Frost & Sullivan, tells The Verge. “How many times has Netflix had a problem with their streaming service over the last 10 years?”

Certainly not as many as HBO Max, that’s for sure.

Open Connect was created because Netflix “knew that we needed to build some level of infrastructure technology that would sustain the anticipated traffic that we knew success would look like,” Gina Haspilaire, Netflix’s vice president of Open Connect, tells me. “We felt we were going to be successful, and we knew that the internet at the time was not built to sustain the level of traffic that would be required globally.”

Nobody wants to sit down to watch a movie only to have their app crash or buffer for an eternity. What Netflix had the foresight to understand was that if it was going to maintain a certain level of quality, it would have to build a distribution system itself.

Popping the hood on Open Connect

Open Connect is Netflix’s in-house content distribution network specifically built to deliver its TV shows and movies. Started in 2012, the program involves Netflix giving internet service providers physical appliances that allow them to localize traffic. These appliances store copies of Netflix content to create less strain on networks by eliminating the number of channels that content has to pass through to reach the user trying to play it.

Most major streaming services rely on third-party content delivery networks (CDNs) to pass along their videos, which is why Netflix’s server network is so unique. Without a system like Open Connect or a third-party CDN in place, a request for content by an ISP has to “go through a peering point and maybe transit four or five other networks until it gets to the origin, or the place that holds the content,” Will Law, chief architect of media engineering at Akamai, a major content delivery network, tells The Verge. Not only does that slow down delivery, but it’s expensive since ISPs may have to pay to access that content.

To avoid the traffic and fees, Netflix ships copies of its content to its own servers ahead of time. That also helps to prevent Netflix traffic from choking network demand during peak hours of streaming.

“We, Open Connect, bring a copy of Bridgerton at the closest point to your internet service provider — in some cases, right inside your internet service provider’s network — and that basically avoids the burden of the internet service provider having to go get it and transfer it through all these servers on the internet over to you,” Haspilaire tells The Verge.