IaC and tested backups prevented data loss when Yandex data centers went down

The attack that took a client offline

In October 2026, a series of attacks hit Yandex data centers in Russia. According to the company, the Sasovo facility was seriously damaged and shut down completely, part of the Kaluga infrastructure went offline, and later the Vladimir data center also stopped working. The account comes from KTS, a Russian DevOps consultancy, published on Habr (a Russian tech blogging platform) on October 30, 2026, by engineer Sergey Kussev (source).

A KTS client had practically all of its infrastructure in a single Yandex Cloud failure domain. When the region went down, the VMs, the network, and the applications that depended on them went down together. There was nothing to fix on site: the critical environment had to be rebuilt elsewhere, under time pressure, with other companies competing for the same resources at the same time.

A risk that was already known before the incident

This client's infrastructure had existed and grown for years alongside the business, concentrated mostly in a single data center. A year before the incident, KTS had carried out a full audit, mapped this risk, and proposed a one-year plan: distribute services, evolve orchestration, migrate components, convert configurations into code, and gradually eliminate single points of failure.

Part of that plan was already in place when the attack happened; another part was not. That is the honest part of the account: no architecture becomes resilient overnight, and the risk materialized before the work was finished. Even so, what had already been done was what sustained the recovery.

Separating engineering time from calendar time

The article proposes a distinction useful for any infrastructure incident: engineering time (how long it takes the team to deploy, configure, and validate) and calendar time (how much time actually passes between the start of the outage and the service coming back, including external waits).

Part of the recovery depended on Yandex's support: the team handed over snapshots and arranged the export of data from object storage (S3) as it became available. The data itself was not lost, so the data SLA was met, but transfer speed became the bottleneck. Meanwhile, the KTS team prepared configurations, dependencies, and checks in parallel, getting ahead on everything that did not depend on third parties.

What actually sustained the recovery

Four layers of preparation, built over the previous year, were what set this case apart from a rebuild from scratch:

• Dependency map: the team already knew which services directly affected business operations, which databases and storage they required, and in what order to check everything.
• IaC with Terraform, Ansible, and GitOps: made it possible to reproduce infrastructure components straight from the repositories, instead of recreating configurations from memory or screenshots of the provider's interface. Several engineers could work in parallel without conflicts, and every fix made during the recovery was saved into the code.
• Eliminating bus factor: more than one engineer knew the environment in depth and had access to the documentation and the code, so the recovery did not depend on a single available person.
• Living documentation: descriptions of services, dependencies, configurations, and operating instructions, tested against a simple criterion: can an engineer who did not build that system, by reading the documentation, understand where it fits in and take the next step?

Not every deployment was described in IaC, because the one-year plan had not yet finished. But the team knew about the existence of each manual component, its function, and its restoration order.

IaC didn't rebuild the system with a single click, but it turned the recovery from archaeology into a sequence of clear steps.

Sergey Kussev, DevOps engineer at KTSОдной кнопкой IaC систему не восстановил, зато превратил восстановление из археологии в последовательность понятных шагов.

The backups had also already been tested before the incident: the team knew whether the copy would survive the same kind of failure as the source, how long it took to restore, what the real RPO was, how long the deploy from it took, and who validated the result. That took off the table the most dangerous question in a disaster: will the backup actually work?

The eight steps of the recovery

The restoration work followed a sequence:

1. Define the minimum target: only the footprint without which the business could not operate, nothing beyond that for now.
2. Cross-reference knowledge, code, and data to see the entire recovery chain before starting.
3. Choose the available platform, not necessarily the ideal one, prioritizing wherever resources would come online fastest.
4. Prepare the basic infrastructure (network, access, compute, platform components) reusing what already existed in Terraform, Ansible, and GitOps.
5. Restore the data at the speed allowed by external systems, without bringing up dependent services before the integrity check.
6. Bring up the applications respecting the dependency order: first the base layer, then the critical one, and last the secondary one.
7. Validate end-to-end user scenarios, not just a green healthcheck.
8. Return the critical services to operation, and only then continue restoring the rest.

Less than 24 hours, but not by magic

The services without which the client could not operate were back online in less than 24 hours on the new platform. That time included the client's decision about whether to migrate or wait for the original data center to come back, the technical work itself, and the wait for resources and data transfer. What did not happen was losing days trying to figure out what the system was made of or manually recreating configurations.

In the post-mortem, the team separated three issues: the root cause (the attacks on the data centers), the extent of the impact (concentration of the critical footprint in a single failure domain), and the quality of the recovery (the actions, data, and tools that brought the services back). Because the architecture's limitations had already been discussed before the incident, there was no dispute over whose fault it was, and the next improvement step started right away.

What changes for those running critical services in the cloud in Brazil

AWS has the sa-east-1 region in São Paulo, Google Cloud has southamerica-east1, Azure has Brazil South. It is common for Brazilian teams to concentrate all critical load in a single region, sometimes in a single availability zone, for the same reason as the KTS client: it worked well for years, and the cost of redundancy seemed hard to justify.

The Russian case is not exotic for those who develop here. It's worth using it as a practical checklist before an incident forces a response under pressure:

• Is the critical infrastructure described in Terraform, Pulumi, or equivalent, or does part of it exist only as clicks saved in someone's head?
• Would a terraform apply recreate the environment in an empty account, or are there undocumented manual dependencies?
• Has anyone actually tested restoring a real production backup, timing RPO and restore time, or is the backup only checked for existing?
• Does more than one person on the team know how to operate the environment end to end, or is there a bus factor of one?
• Were RTO and RPO decided together with whoever pays the bill, or did they stay only in the infrastructure team's heads?

What's still open

The KTS client changed its stance after the incident: Yandex Cloud remains the main platform, but critical services now have a backup in case of another failure. The next step, according to Kussev, is to set up a standby environment on another provider, deciding between hot, warm, or cold standby, the data synchronization method, and regular failover tests. This process has real cost and complexity, and KTS promises to detail that choice in a future article.