In an incident yesterday in Reading, a downed router on BT’s network took out 9,000 homes. If you believe the press release.
If you actually check however, you’ll find that BT has once again lied. Plus.net’s user graph (remember that Plus.net is a BT subsiduary?) alone shows a drop of nearly 25,000 users.
How does a nationwide broadband and telephony provider in this day and age allowed to get away with such a crap non-resilient network design and still be handed the sole country-wide monopoly?
One of the telecoms engineers in my workplace spoke to a source inside BT and found out the true story behind the two outages. Last Friday was not a power cut. A card in a router locked up, and rather than replacing it BT took the decision to simply reboot it and hope it doesn’t happen again.
This goes against general practices in any large datacentre where you have a multitude of people relying on a single device to be working properly 100% of the time – when something that critical fails, you replace it immediately. It also means they really don’t have any redundancy as it appears there was no second card or router to take over when the first failed, and that means a 2-6 hour outage while engineers are gotten out of their beds, travel to site, diagnose and fix.
So it seems I was right, given that this morning once again the entire of Northern Ireland, Scotland and parts of England all lost their broadband connections again. Except this time BT is blaming an unnamed hardware vendor for the issue, again at the Edinburgh datacentre.
Imagine the broadband access for the entire of California, of even just a large city like New York going out for hours on end. There would be a senate hearing on the cause, loss of manhours for businesses and public outrage. In the UK… our government just gives the incumbent telco some more money and a tells them to try and not do it again.
There was an overnight outage where BT’s Edinburgh datacentre lost power. Now bearing in mind that the BRAS’s in this datacentre service pretty much the entire of Northern Ireland and Scotland, and parts of the North of England, why would they claim that only 20,000 homes were affected?
My workplace specializes in providing DSL to businesses in Northern Ireland, and I know for a fact that every single one of them was down aside from a small of handful of lines that we have in England. These are lines spread across the entire province, from Belfast to Derry. None of my neighbors had working broadband either, except for the two that were using Sky LLU. My Dad didn’t and he’s on Plusnet, many miles from where I live.
20,000 homes is about an 8th of the numbers of homes in Belfast alone, never mind across Northern Ireland, Scotland, or the North of England. Why won’t BT admit to the real number of homes affected? Because then people might realize how utterly incompetent they are to design such a widely used and relied upon system to be able to be take down nearly half the country because of a supposed power outage in a single location (which in a properly built datacentre, should be nearly impossible).
BT – Liars and idiots. Can someone remind me again why we gave these people the telecoms monopoly?
If you have a system built with the still quite good Nvidia GeForce 8800GTS graphics card, and decide to upgrade by adding a second one and running SLI, beware the Nvidia Fail.
Nvidia in their wisdom made an upgraded version of the card called the 8800GTS 512. It is not SLI compatible with the original 8800GTS, and the odds are you won’t know it until you’ve bought one and spent an afternoon wondering why SLI won’t work.
God damn you Nvidia.
A new level of fail from our friends at BT Wholesale. They have actually willingly provided proof that they do not read fault reports the first time around:
Yes.. that is a grand total of 43 seconds from reporting the fault to BT Wholesale rejecting it. This was even reported via KBD, which lets you confirm that the user has already attempted to replace his router, cables, filters and even tried from the test socket. 43 seconds is not enough time for most people to type that long-winded reply about SFI appointments, let alone for BT to run the necessary diagnostics to determine if there is a fault or not.
At my place of work we have suspected that BT was doing this for a long time as all too often, and 9 times out of 10 blatently obvious faults are rejected with the message “not due to a network fault”. Now I have a handful of faults, some where it was customer some, but some where there was genuine faults such as the DSLAM being faulty where BT has denied anything being wrong and cleared the fault in less than a minute.
