A backhoe cuts your fibre. How long does your business stop?

It is 9:07 a.m. A backhoe is working in front of the building and then, suddenly, everything stops.

Outlook stops syncing. Teams hunts for a connection. The IP phones go silent. The line-of-business application throws an error and the payment terminal spins. Someone offers to tether their phone, but the cellular signal barely reaches the warehouse and nobody knows which systems should come back first.

The fibre is cut.

A few years ago, an internet outage mostly meant you could not browse the web. Today it can effectively close the office. So the question is no longer whether your provider delivers a good connection. It is what happens when that connection disappears.

One cable, far more than internet

The move to cloud services has meaningfully improved collaboration, remote work and data resilience. It has also created a new dependency: to reach those services, you need a connection.

In a modern small or mid-sized business, the internet link may carry:

email and Microsoft 365;
VoIP telephony;
the CRM and accounting software;
production applications;
payment terminals;
VPN access and remote work;
cameras, alarms and control systems;
backups and remote monitoring.

A single outage can therefore hit several departments at once. This is no longer just a networking incident: it is a business continuity issue.

Canada got a spectacular demonstration on July 8, 2022. An outage in Rogers' IP core network cut off more than 12 million wireline and wireless customers for 26 hours — from 4:58 a.m. on July 8 to 7:00 a.m. on July 9, Eastern time. Critical services were affected as well, including electronic payments and Interac e-Transfer. The independent assessment published by the CRTC describes in detail what prolonged the outage — and those mechanisms are not exclusive to large networks.

Your outage probably will not make the national news. For your employees and your customers, the effect can be immediate all the same.

Two invoices do not guarantee two paths

The answer seems obvious: get a second connection from a different provider. That is an excellent starting point, but it does not guarantee genuine redundancy.

Two providers can share the same pole, enter the building through the same conduit, or lease a segment of network from the same carrier. One well-placed backhoe can then take out both services at once.

So it is worth distinguishing two things:

commercial diversity, meaning two contracts or two providers;
physical diversity, meaning two technologies or two genuinely independent paths.

The CRTC report is explicit on this, and its lessons reach well beyond the Rogers case. Among the formal recommendations addressed to network operators generally: provide critical sites with backup connectivity from a third-party operator, equip critical staff with secondary means of communication — such as SIM cards from another provider — and leverage low earth orbit satellite constellations for backup connectivity at remote sites.

For a smaller business, this does not necessarily mean digging a second fibre entrance. A primary wired connection can be paired with 5G cellular failover or, in some locations, with a satellite link.

The right choice depends on the building, the coverage, the number of users, and which applications have to keep running.

Manual failover is not really failover

Suppose your secondary connection works perfectly. If switching over requires someone to walk over, find a cable, call the provider and reconfigure the firewall, your business still experiences an outage.

Real failover should activate automatically when the primary link becomes unavailable. The firewall or network management appliance continuously checks the state of each connection, detects the failure, then redirects traffic to the secondary link.

Depending on the configuration, employees may notice a brief interruption, but essential operations resume without turning the server room into a crisis centre.

That switch has to work in the other direction too: when the primary service returns, traffic needs to move back cleanly, without causing a second outage.

This is fundamental to sound network management : failover is not just an extra subscription. It is a monitored, tested configuration.

The trap nobody mentions: your IP address

Here is the obstacle that defeats technically well-built failover.

Automatic switchover handles outbound traffic: your employees get Microsoft 365, the web and cloud applications back. But anything coming in from outside can stop working, even when failover activates correctly.

The reason is simple: your public IP address changes. And a cellular failover link rarely comes with a static IP — it is often dynamic, and frequently sits behind an address-sharing mechanism that blocks inbound connections entirely.

In practice, here is what may fail during the outage despite working failover:

site-to-site VPN tunnels to your other offices;
inbound remote access for your employees or your IT provider;
any port forwarding to a server, camera or control system;
on-premises services that third parties need to reach;
partner firewall rules or allow-lists that filter by IP address.

This gets solved at design time, not during the outage. Solutions exist — a second wired link with a static IP, a cloud tunnel service, an SD-WAN architecture, hosting the VPN termination point off-site rather than in your building. But they all assume someone asked the question beforehand.

Does your troubleshooting tool depend on the link you need to troubleshoot?

This is the most transferable lesson in the CRTC report, and it went largely unnoticed in the news coverage.

Rogers' management network — the very network used to diagnose and repair — ran on the company's own IP core. When that core failed, remote employees could no longer reach it. The network operations centre and other critical sites had no backup connectivity from a different provider. The result: Rogers had to dispatch staff on site to physically access the affected routers, which delayed recovery. And because the incident management staff were themselves using the company's mobile services to communicate, SIM cards from other operators had to be shipped to remote sites before the teams could talk to each other.

Now scale that down to a smaller business. During an outage:

your remote management tool can no longer reach your workstations and servers;
your IT provider cannot connect to diagnose anything;
your monitoring stops reporting — or sends exactly one alert, the one saying everything is unreachable;
your off-site backups stop running;
and if your phone system rides the same link, you cannot even call to report the outage.

Put differently: the moment you most need your IT provider is precisely the moment they can do nothing remotely. Well-designed failover should include an independent administrative path — even a modest, slow one. It serves one purpose only, but it serves it on the day everything else is down.

Not everything can be a priority

A 5G backup connection will not have the capacity of dedicated fibre. It may be enough for email, telephony and critical transactions, but not to simultaneously carry backups, all-hands video calls and large file transfers.

So you need to decide in advance what goes first.

During an outage, a business might prioritize:

telephony and customer communications;
the applications needed for sales or production;
payments and transactions;
essential administrative access;
basic email and collaboration.

Less urgent uses can be throttled or paused temporarily. That hierarchy keeps the failover link from saturating at the exact moment the business needs it most.

Does the phone system ride the same link?

With the gradual retirement of traditional telephony , more and more businesses run VoIP. The technology is flexible and effective, but it generally depends on the local network, on power, and on internet.

A failover strategy therefore has to cover voice, not just data. Can calls automatically use the secondary link? Can they be redirected to mobile phones? Does the team have a contact list that is reachable without Microsoft 365?

The CRTC report says as much in its formal lessons: critical staff need secondary means of communication. At Rogers, the absence of those means slowed the response to the outage. In a smaller business, it comes down to a simple question: if the network goes down, how does your team coordinate?

Your communication plan has to survive the outage it is meant to manage.

Do not forget the power

The fibre may be fine, the 5G connection may be available and the automatic switchover may be correctly configured. If the firewall and switches lose power during an outage, none of it matters.

Essential equipment — modem, firewall, switches, wireless controller and, depending on the environment, phones — should be protected by an appropriately sized UPS. You also need to know the real runtime: ten minutes absorbs a momentary dip, but not necessarily an extended outage.

Internet, the local network and electricity form a chain. Failover is only as strong as its weakest link.

The test nobody wants to run

Many businesses pay for a secondary connection for years without ever verifying that it actually takes over. The first real test then arrives on the day of the outage — at exactly the worst moment.

The CRTC report contains a detail on this that should give everyone pause. The Rogers outage occurred during the sixth phase of a seven-phase upgrade. The company had initially rated the risk of that process as "high." But because the earlier phases had gone through without incident, the risk assessment algorithm downgraded the sixth phase to "low" — which exempted staff from the lab testing and additional approvals normally required for that kind of change.

It had worked five times, so they stopped checking.

That is exactly the mechanism that keeps a failover connection from ever being tested: it has never been needed, so everyone assumes it works. The report accordingly recommends that all network operators simulate and rehearse failure scenarios to uncover weaknesses before they show up on their own.

A controlled exercise answers some very simple questions:

Does the switchover happen automatically?
How long does it take?
Do the phones still work?
Do critical applications stay reachable?
Do inbound connections — VPN, remote work, remote administration — survive the IP address change?
Does the secondary connection carry the expected load?
Do alerts reach the right person?
Does the return to the primary link happen cleanly?

This test should be scheduled periodically, the same way you test a backup restore or a disaster recovery plan . Better to find a problem during a fifteen-minute exercise than during a full day of downtime.

Six questions to take stock

An internet outage can come from an excavation, a failed piece of equipment, a power outage, a configuration error or a major problem at your provider. You do not always control the cause. You do control how your business responds.

You can sketch a first picture with six questions:

Which activities stop if internet goes down right now?
Do we have a genuinely independent second connection — not just a second invoice?
Is switchover automatic and monitored?
Do our inbound connections survive the IP address change?
Can our IT provider still reach us during the outage?
When did we last test the scenario?

If the answer to the last one is "never," you have already found your next step.

The best failover connection is not necessarily the fastest or the most expensive. It is the one that uses a sufficiently independent path, activates without improvisation, protects essential services, leaves a door open for the people who need to intervene — and has already proven that it works.

If a backhoe cuts your fibre tomorrow morning, will your employees wait for the technician — or keep working?

An IT project or a question?

MMO Techno can assess your dependencies, configure a failover connection and test the switchover with your team before a real outage puts your operations on hold.

Talk to an MMO Techno expert .