Few things are more important to the operation of state government than the State Data Center (SDC), Quincy Data Center (QDC) and our Cloud Highway hardware in Seattle. It is the central nervous system for the delivery of vital services for Washingtonians.
So, you can imagine the amount of pressure that was placed on Chris Craig, WaTech's Network Operations Manager, and his team when they were tasked with replacing all the equipment in these data centers that was aging and would soon no longer be supported by the vendor. The Data Center Lifecycle Management Switching project not only replaced legacy hardware, but also implemented a completely new design to modernize the data centers architecture and make it more resilient and easier to manage.
Stephanie Ison from the PMO office was assigned to the project that initiated in April 2021, and was completed in June 2023. It replaced existing legacy switching hardware in SDC and the backup Quincy Data Center (QDC), and Seattle Equinix data centers with a new infrastructure, architecture, and associated technology. The new hardware increased bandwidth, capacity, and improved the speed for state agencies to transfer data to and from the state's data centers.
Chris said he's never worked on a project of this magnitude before and there was a lot of stress that came with it. "Boy, it's been a rough one," he said. "But I'm just super proud of everybody on the team and how they all worked together across different divisions and pulled this off."
He noted that Mike Costello's Architecture team worked with various consultants and created an all-new design for the data center infrastructure. This involved replacing over 100 devices in our data centers that all data center customers connect to, as well as installing and running thousands of new cables in the data centers by the SDC facilities team. "It was a huge effort to run all these cables in multiple data centers and to implement this new design," he said. "Not to mention that much of the work was done during the COVID pandemic which caused delays due to supply chain issues that resulted in a shortened timeline to complete the customer migrations."
Even with all those delays and headaches, the Data Center Life Cycle Switching Project Team, led by the project manager Stephanie Ison was able to complete the hardware replacement, implement the new design and migrate every single customer in both the Olympia and Quincy data centers without any significant issues. In this case, no news really was good news. Stephanie did a great job addressing and documenting the numerous hurdles the team faced along the way.
"We had people working with us after hours until 5 a.m. in the morning testing their services. A lot happen behind the scenes that not a lot of people knew about," Chris said. "The amount of progress we made and the teamwork was pretty amazing to witness. I've been involved in many projects over the years, and this is by far the biggest change that we've experienced in the data centers since they were first built."
The redesign is transitioning the SDC to a modular design for the state government network. In the past, if a key piece of equipment for the network went out, the entire data center would go down until it was replaced. The modular design allows for a piece of equipment to be swapped out with a replacement without taking the portions of the network offline. The new design makes the entire network more resilient, Chris said.
At any given time, there were about 20 people working on the project, including network operations, the SDC facilities team, network architecture and core engineers and many others. Chris said that what impressed him the most was how many different groups within the agency solved problems and overcame many obstacles along the way by working together.
Special thanks to the project team and all who made this project happen.