Popular online gaming platform Roblox is undergoing a major infrastructural upgrade to improve its resilience and efficiency. According to a post on the Roblox Blog, the improvement process came after a 73-hour outage in October 2021, highlighting the platform’s vulnerabilities.
According to the blog post, the company is constructing a second data center, allowing for faster failover in case of issues. Additionally, it is implementing a “cellular infrastructure” within its data centers. This method creates isolated units called cells, which act as “blast walls” to contain failures and prevent them from spreading throughout the system. This is a lot of tech speak, but it essentially means they’re fighting outages that come from servers being overwhelmed.
I remember the outage in October of 2021, and I was actively trying to log in during that period. I remember it was a really big deal and showed how big Roblox had become. It was also a great way to see how it could go down. To combat the possibility of this happening again, Roblox is working on “active/active” infrastructure, which utilizes both data centers simultaneously to distribute workloads and enhance reliability.
To achieve greater efficiency and seamless migration, the company has standardized its cells, ensuring they are uniform and interchangeable. This allows for automated monitoring, troubleshooting, and workload shifting, further optimizing performance. Over 70% of Roblox’s back-end service traffic currently operates within cells, which shows the transition is working. This ongoing project is expected to take most of 2024 and potentially part of 2025 because Roblox has to migrate thousands of services and infrastructure components.
The long-term goal of this initiative is to achieve 99.99% user uptime, meaning minimal disruptions and a virtually flawless experience for players. This significant improvement will allow Roblox to support its growing user base, eventually reaching its goal of connecting a billion people in real-time. That’s an awesome goal to reach.
