BT Fail :: Part 2

A new level of fail from our friends at BT Wholesale. They have actually willingly provided proof that they do not read fault reports the first time around:

Yes.. that is a grand total of 43 seconds from reporting the fault to BT Wholesale rejecting it. This was even reported via KBD, which lets you confirm that the user has already attempted to replace his router, cables, filters and even tried from the test socket. 43 seconds is not enough time for most people to type that long-winded reply about SFI appointments, let alone for BT to run the necessary diagnostics to determine if there is a fault or not.

At my place of work we have suspected that BT was doing this for a long time as all too often, and 9 times out of 10 blatently obvious faults are rejected with the message “not due to a network fault”. Now I have a handful of faults, some where it was customer some, but some where there was genuine faults such as the DSLAM being faulty where BT has denied anything being wrong and cleared the fault in less than a minute.

BT Fail :: A new level of failure

Customer complains of a fault. I diagnose and determine that the customer is doing nothing wrong, and it’s probably a line fault. BT’s response is near instant, lambasting me for reporting what is clearly not a BTW (BT Wholesale) fault. An entire day later, they update it to say that there’s a fault with the DSLAM. This is of course long after I’ve told the customer there must be something wrong on their end as BT says there’s no fault.

Is it any wonder that people think their ISPs are incompetent and BT can do no wrong? It’s because when BT has a fault they have the ability to actually go and look at the DSLAM and see for themselves. When anyone else has a fault, they have to rely on the BT Wholesale broadband diagnostics systems which are quite frankly, half assed and go down more often than a thai hooker.

This is what happens when someone has a monopoly but are mandated to supply wholesaler services to others to make it look like they don’t have a monopoly. Their own service is perfect and can do no wrong, while all those wholesalers can scream and shout and cry about the godawful service they’re being given.

Yet Another Ignorant ‘Your ISP is screwing you’ article

I hate these articles. They’re always written from the perspective of a consumer who knows some of the buzzwords but doesn’t actually bother to take the time to understand any of it or the technology.

Bandwidth Throttling

One oft-protested behavior of various ISP’s is the throttling – that is, limiting – of bandwith at certain times or for certain uses.

Yes, some ISPs shape the bandwidth supplied to the customer. There’s good reason for this however. Residential broadband connections are contended services. That means that you share that bandwidth along with a number of other people. In the UK, the standard contention ratio for residential users is 50:1. If bandwidth is not shaped, then just one customer abusing Bittorrent to download pirate movies or games or music can use up all the bandwidth of 50, leaving those other customers shit out of luck. You’ll also note that the only people who ever complain about shaping are those pirating content.

Imagine that the bandwidth is a 3 lane motorway. If everyone behaves, we can all drive down the motorway at reasonable speeds, occasionally going faster where possible. However, if one driver in an articulated lorry starts swerving all over the road, everyone else has to slow down and be late. Bandwidth shaping is done to preserve the use of the road for everyone.

Deceptive Speed Claims

Examine the fine print on most ISP commercials, and you will likely find that the promised Internet speed (say, 10MBPS) has the words “up to” in front of it. As it turns out, this is often a clever means of dodging the truth about the actual speeds you are likely to receive.

Again, the service is contended. If you have an 8Mb 50:1 service, you are sharing that 8Mb with potentially up to 50 other people. There’s also the technology involved; ADSL is distance limited so the further away from the telephone exchange you are the lower it will connect at. “Upto 8Mb” covers all of this in two words. You get what you pay for and at £20-25 a month, no ISP could guarantee what this writer is demanding. Let me break it down:

  1. ADSL line, up to 8Mb 50:1 – £11.90/month
  2. Link to ADSL network to cover a single user on 8Mb 50:1 – £1400/month
  3. 8Mb bandwidth – £176.00/month
  4. Upstream circuit links – 2x£2,500/month

Never mind the infrastructure required to deliver such a service, servers for providing DNS and email, datacentre space, cooling and electricity and so on – but to give you your 8Mb ADSL without it being shared with anyone would cost somewhere in the region of £6,500 per month to have their own dedicated ISP not shared with anyone else. NOT £25. It’s only by sharing infrastrucure and bandwidth with other customers that cheap broadband actually becomes economical. If you want a dedicated guaranteed 8Mb circuit, feel free to go talk to your ISP – they’ll quote you a figure probably somewhere between £10,000 to £12,000 a year, plus probably a £15,000 install.

Targeted Advertising

Increasingly, some of the most passionate complaints against ISPs have involved privacy concerns. A case in point is Charter’s decision in 2008 to begin tracking its users’ search behavior and using them to insert ads into their results.

Never mind that the example screenshot given is from Google, highlighting Google’s own advertising which has absoloutely nothing to do with whatever ISP you are using to connect with; the article in question successfully gives a single example of a single large ISP abusing the Phorm advertising system. As far as I am aware the only other ISP to consider using this system is BT in the UK, and they were smacked down for being in breach of privacy laws.

This is tarring all ISPs with the same brush for the sake of a one or two bad apples.

ISP Wiretapping

2007’s Communications Assistance for Law Enforcement Act mandated that all ISPs enable the feds to “wiretap” Internet transmissions in much the same way they do phone calls.

Note the word “mandated”. That’s not your ISP screwing you over – that’s your government. Most ISPs have this capability anyway in order to comply with court orders or police investigations. Do you really want to be responsible for some 3 year old getting raped by a pedophile because it was made illegal for ISPs to help catch sick fucks just so you could download your pirate movies without having to worry that someone might be watching you because the police told them to?

Ad-Filled “Website Not Found” Pages

Always on the lookout for new sources of revenue (however small), some ISPs have taken to displaying ads in their error pages.

Some ISPs do this, however the good ones will give you a source of opting out, and as this is usually DNS based if you don’t like it you can always either setup your own DNS server or use OpenDNS.

Deep Packet Inspection

Another serious gripe privacy advocates have with ISPs is what is known as “deep packet inspection.”

That would because privacy advocates don’t actually understand DPI, which looks for patterns in order to recognize traffic types. The actual content cannot usually be observed, but it can also log when someone is for example, using Bittorrent. These boxes are usually used to packet shape your traffic (see “bandwidth throttling” above), but are also extraordinarily expensive – usually only affordable by quite large ISPs. But wait! There’s more…

However, it is also been used by ISPs to police copyright infringement by detecting when someone is or may be downloading songs or movies – and some ISPs go a step further by turning this information over to inquiring record labels.

And so we get to the real crux of the issue that the author has with DPI – pirating movies and music! Of course, if you’re not doing anything illegal then DPI really isn’t something you have to worry about. Damn you pesky ISPs! Conforming with the letter of the law and trying to prevent yourselves from being used to commit illegal activties! Grr and much fist-shaking and so on.

Packet Spoofing/Forgery

Comcast engaged in what is known as “packet spoofing” (or packet forgery) by interrupting file transfers with bogus packets that killed any P2P downloads a user happened to be engaging in.

One ISP does something that contravenes the way that TCP/IP is supposed to work and we all get tarred with the same brush again. I can assure you that few if any ISPs that have any sort of technical savvy would even consider doing this. Comcast are the only ISP known to have deployed this system as it far too heavy handed – affecting both legitimate and illegitimate traffic. It also opens the ISP up to a certain amount of liability for having demonstrated that they can block certain traffic types, for then not blocking other traffic types such as viruses or spam, which happily leads me to the next point in this ignorant article.

Inadequate Virus/Spyware Protection

ISP’s have also come under fire for charging high subscriber fees without adequately protecting consumers from spyware, viruses and other forms of online fraud.

As I think I’ve already demonstrated, most ISPs are not charging “high subscriber fees”, and in one sentence the author of this article has demonstrated his complete ignorance of any of the previously mentioned technologies. If you want your ISP to prevent you from idiotically downloading a virus and running it, they have to install what is known as an IDP, or Intrusion-Detection-Prevention device. What is an IDP? Basically, it’s a Deep-Packet Inspection device configured to look for viruses, trojans, spyware and known hacks. It would then have to use Packet Spoofing to block your attempt to download that virus. So the author wants us to protect him from viruses without actually using any of the known technologies to do so. Does he want us to send someone around to his house to operate his computer for him or what?

Generally speaking, service agreements between you and your ISP indemnify them from responsibility for any damage or losses caused by spyware or viruses you get infected with on their network.

…because we all know that it’s the pesky ISP forcing viruses and spyware onto your computer. Out in reality-world (as opposed to this crack-smoking monkey of an author’s fantasy world), 9 times out of 10 virus and spyware infections are because the user actively downloaded that cute new screensaver of the puppies doing barrel rolls and installed it, and that screensaver was actually a shell for a massive spyware infection. Or the user received an email from King Mambatu who wanted his help to move $9,843,699 dollars out of the Bank of Nambia and needed him to open this harmless attachment to get the process started. Yup, all the ISPs fault that is. This couldn’t possibly be the reason why ISPs have had to indemnify themselves against protecting you from viruses and spyware because a certain section of society wouldn’t sue them into oblivion with frivolous lawsuits over their own stupidity or that when they do offer antivirus or antispam services on email, that the technology cannot guarantee that it will catch everything.

Sneaky Fees

MSNBC reports on a telling example back in 2006, when a a $2-$3 per month federal tax on DSL users was taken off the books. But rather than lowering its subscriber fees by $2-$3, Verizon thought better of it and kept fees the same by adding a “supplier surcharge” fee.

It’s hardly a sneaky fee if it’s listed on the bill. It might be sneaky for the muppets who don’t bother to read their bills, which is rather disturbing seeing as this is written by someone calling himself “Bill Shrink Guy”.

I won’t deny that doing such a thing isn’t disreputable, but the real issue here is to make sure you read all your bills and ensure you understand what you’re paying for. If your gas supplier suddenly adds a “Boiler Maintenance” surcharge to your bill and you pay it without question, you’re an idiot.

Bandwidth Throttling

IPsec VPNs for Mikrotik RouterOS

It’s unfortunate that the Mikrotik RouterOS manual on IPsec is not great – it’s sorely lacking in details  and good examples, and what examples it does have are not well explained.

Recently I had to setup several Mikrotik RouterOS to ZyXEL VPNs and through I would document how it’s done.

Read more »

PSPgo so awful, Sony expects increase in sales of PSP-3000

No other company in the world has the balls to attempt what Sony is trying, and I would seriously love to know what crack they’re on and where I can get some.

Sony Schmuck A: So, sales of the PSP-3000 are okay, but not great.

Sony Schmuck B: I know! Let’s make a new version. But lets make that version so god damned hideously awful that people will scramble to buy the old one!

Sony Schmuck A: Excellent idea!

… and so the PSPgo was born. I mean really, it’ll not be long before the next Sony console shoots acid in your face to try an encourage you to buy a newer/older model.

And just to add insult to injury? The PSPgo is only £25 quid cheaper than a full blown PS3.

Sony have cojones the size of the moon. Seriously.

ISP fined $32 million for helping sell illegal fake goods

In this article, it’s reported that Louis Vuitton – a fashion designer – has sued and won $32 million US from an ISP, Akanoc Solutions Inc, which hosted a customer who was selling forged Louis Vuitton products.

This has caused uproar in the ISP community as at first glance it appears as though the ISP in question is being held responsable for the acts of it’s customers, but really they are simply being held responsable for refusing to act against a customer who was involved in an illegal practice.

The court documents detail how Louis Vuitton notified the ISP no less than 15 times – giving them ample opportunity and evidence to terminate the customer. Instead the ISP allowed that customer to juggles his sites around on different domain names and IP addresses and continue to sell the fake goods. The ISP was found guilty because they were complacent in allowing a customer to use their service to break the law. They tried to claim safe harbour under the DMCA act, however here we are exactly 2 years after the initial filing and the websites listed in the initial claim are still operating in Akanoc IP space.

It’s a tricky line for ISPs to cross. In my day to day work, I receive notices of copyright infringement from the MPAA/RIAA every week – but how are we supposed to act? We do not have the technology to actively monitor accused customers as the equipment required ranges into the tens of thousands to the hundreds of thousands of dollars. The MPAA/RIAA provide scant evidence – evidence which has been shown in the past to be very, very incorrect at times. We do what we can under UK law, notify the customer that we have received an infringement notice and notify the MPAA/RIAA that the customer has been warned.

The UK government wants to make deep packet inspection boxes mandatory for all ISPs, without regard to the cost  (which will cripple any medium-to-small service provider, if not put them out of business) and on top of that they appear to think that these boxes can log everything and anything regardless of software or encryption – there isn’t a DPI box yet which can monitor Second Life traffic… but that’s what the UK government is expecting ISPs to do.

However, this is missing a key point in this case in California: The ISP in question was provided with verifiable evidence that thier customer was selling fake, knockoff and illegal goods – and they declined to act on it (and in fact are still declining to act upon it to this day). Other ISPs spend a great deal of time and money ensuring that they are reacting to spam issues and hacked servers being used to host fake paypal logins. Akanoc Solutions Inc. took an active decision in deciding to not enforce their rights to terminate an obviously fraudulent customer and allowed them to continue in their business. They deserve everything they got.

Ghetto Antenna

Possibly the most ghetto thing I’ve ever built: The Ghetto Wireless Antenna.

Ghetto Antenna

Don’t worry, it’s only temporary. It’s a twist tie with an end stripped off, folded over and shoved into the N-type antenna connector. Suprisingly it gives quite good reception, increasing the signal strength to nearby laptops by about 20db.

Photos :: City on fire

Postfix Mail Queue statistics via SNMP

This post documents a small shell script designed to provide basic mail queue statistics via SNMP for Postfix.

Requirements

Installation

The code can be downloaded here.

To install, place the script anywhere in your system and edit it to provide the correct path to the “qshape” perl script that comes with Postfix.

Note: Under openSUSE qshape.pl is part of the postfix-docs package and is not installed by default.

To configure net-snmp, edit your snmpd.conf line and add a line as follows:

pass [oid-of-choice] /bin/snmpqshape.sh [oid-of-choice]

For example, due to a quirk in a paticular SNMP monitoring package I use, I had to use an OID belonging to Motorola:

pass .1.3.6.1.4.1.17713.2 /bin/snmpqshape.sh .1.3.6.1.4.1.17713.2

Net-SNMP will return 3 OIDs on query:

.0 :: Incoming
.1 :: Active
.2 :: Deferred

MRTG / RRDTool

Since the setup of monitoring / statistics tools such as MRTG or RRDTool is site-specific, no provisions are made on this page to provide a complete usage example. A minimal example for RRDTool:
#! /bin/sh
STR="`snmpwalk -OvQ -r 10 -t 5 -v 2c -c publicommunity hostname.site.com \
.1.3.6.1.4.1.17713.2 | perl -ne 's/^/:/;s/\n//;print'`"
rrdtool update /path/to/rr-database.rrd -t incoming:active:deferred N${STR}

DISCLAIMER
This code is free to use and distribute, and the author offers no liability or warranty for it’s misuse.

Photography :: Titanic Quarter