Archive for the 'Internet' Category

Linux command line tips & Stuff

I’ve been taught a couple of command line tips at work, and thought it wouldn’t be fair if I didn’t pass them on. So, we begin.

CTRL-R

This insanely useful trick, in a terminal or a console, will allow you to search your bash history for any command you’ve previously run and re-run it. For example, quite often on my laptop, type “CTRL-R upg” in a terminal window, which runs the following command:

sudo apt-get update && sudo apt-get upgrade -y

If you don’t know, that command updates your package repository listing (what programs have been updated), and then goes and upgrades all of the packages that have been upgraded, with the only exception of the more significant upgrades, like to the kernel. (A human has to activate those particular upgrades - and the -y tag doesn’t signify human, as that command can be cron jobbed very easily…)

CTRL-O

This one I was taught in my interview for Positive Internet. (So, I’d better not get this wrong! ;))

If you have run a series of commands in a terminal or console repeatedly, say editing a file, doing a config check and then restarting apache (as I have done whilst I’ve been playing with my Apache2 config file for this blog), then this little switch is priceless. Basically, once you hit the up arrow to find the command you wish to use, hitting CTRL-O instead of Enter, will execute the command, and then once you’re back at the shell prompt list the next command in the series. So for the first set of commands:

vi /etc/apache2/sites-available/kirrus  [ENTER and edit the config]
apache2ctl configtest [ENTER]
apache2ctl graceful [ENTER]

Ooops, I’ve just killed my blog by way of a looping redirect! Quick, undo!

[UP ARROW, UP ARROW, UP ARROW]
vi /etc/apache2/sites-available/kirrus [CTRL-O and fix the config]
apache2ctl configtest [CTRL-O]
apache2ctl graceful [ENTER]

The benefit? The second time round, once I found and initiated the series of commands, I didn’t need to type anything, other than the changes to the config file, and the initiating control sequences. Annoyingly, you can’t just hit CTRL-O once, and then expect to be in the chain next time you hit enter - hitting enter won’t provide you with the next command in the chain once you’re finished. Although, this of course can be a good thing, if you want to return to a clean command prompt.

Hopefully one of those will be useful to you.

Ubuntu-UK Planet, Caffeine and Rambling.

For some strange reason, the Ubuntu UK planet didn’t pick up my last post as a new post. Possibly because it got a little confused with the server move and IP address change? Anyway, for those of you reading this on the planet, I have a post about a couple of the tools that come with apache2 on my blog. Not much, but hopefully interesting.

Caffeine: I’ve pretty much overdosed this evening. Head’s swimming right now, and the screen appears to be filling my vision (hence the more than normal ramblingness [yes I invented a word :)] going on in this post). Stayed on at work for an hour and a half, pushing me closer to the tiredness limit. So, on the way home I drank a bottle of Coca Cola, (the tube section) and a small americano coffee (the train section). It kept me awake (yay!) at the cost of me being a little… jumpy at the moment. Still, it’ll wear down shortly, especially since I finished my food about 20 minutes ago. That always helps clear the caffeine effect. So, shortly I’m going to crash from my caffeine high, and be a Zombie. Hopefully won’t be that way tomorrow morning, but at least I can sleep on the train in and if I’m lucky and get a next-to-the-door seat on the tube quickly, on the tube in. (The glass to your left or right acts as a good, if a little hard, pillow. The glass behind you, unfortunately, moves too much, and gets painful quite quickly.)

Right. I can feel myself starting to slow down, so I’d better sign off before the Zombieness (Yay for creating random useless words!) comes into play.

Stay safe :)

Oh my. The grandeur. The absurdity.

http://news.bbc.co.uk/1/hi/technology/7492907.stm

…Skype or even Firefox might be declared illegal in Europe if they are not certified by an administrative authority

The day, literally, the DAY that I publish my warning and thoughts on exactly this.


Time, privacy and its lack thereof

Over the weekend I watched “V for Vendetta”. Its come up recently in a forum I frequent, specifically the quote at the beginning - about ideas being impossible to kill.

Its wrong.

Ideas can be killed. Eventually. It takes time, patience, and willingness on the part of those whose minds you are changing.

I wonder, do you ever worry about this country? (I’m talking to the Brits here. Sorry everyone else, this post is aimed at the UK, our laws and situation though you may find parallels about where you live.) We have a government that (almost) no-one likes. We have a leader, who at best could be termed ignorant. At worst a devious schemer.

Not worked out what I’m talking about yet? This is what I’m talking about. I’m talking about the erosion of our privacy. I’m talking about the surveillance state we are in. Do you even know what powers our oh so wonderful government has over us now? Read some of the above links. Heck even the Information Commissioner, the guy supposed to be looking after our privacy as relating to computers is saying its here!

I’ve been reading all these news items one by one, slowly, over a period of the three or so years that I’ve been watching and reading all articles that come up in BBC News’s tech section. As I’ve read them one by one, and not put them together, I haven’t worried too much. Till now. Apart, each are only merely slightly worrying. Together, we’ve got a problem. If you’ve not already, I recommend you now go and watch “V for Vendetta“, and then read “The Traveller“. Then continue reading below the line.

———————————————————————————-

You think the technologies to create this don’t exist? As someone who sits on the edge (or maybe inside) of geek-dom, I can say with knowledge and certainty, they do. We could be living in George Orwell’s 1984’s level of observation in a simple matter of 6 months. Heck, if you’re in London, you’re probably already carrying at least one unique ID tracker, most of which can be read remotely at 30 feet (or more). Your RFID Oyster card. Your mobile phone. Your RFID chipped passport. Your RFID credit/debit card.

Don’t believe me? Read up on RFID and RFID skimming.

Literally, all we have to do now is wait, as our last safeguards are removed or bypassed, and before too long all it will take to bring us to the societies depicted in “1984″, “V for Vendetta” and “The Traveller” is one crack pot power maniac in government. Need I mention, I don’t have a high opinion of Gordon Brown?

Its not as if it hasn’t happened before in history, one man taking over a democratic country and turning it into a dictatorship. But much worse, as in our country is all the tools needed to *keep* our country in a dictatorship and aid its passage to one.

Now, I can say all this, as we’re not there yet. However, I see the day fast approaching when I would be arrested for what I have written here, and asked to relinquish my encryption key’s pass phrase. (You’ll have to torture me before that happens.)

Someone once told me, that we’d never get to the point where the EU constitution comes into play. The treaty that was ratified by the UK merely a few weeks ago, was basically the EU constitution.

I have heard many times, “I’ve got nothing to hide”. No, not yet. Until you have to hide your sexuality. Your Race. Your religious beliefs. Your musical preferences. Your secrets.

I leave you with a link and this old, poem. How long before it becomes relevant once more?

=======================

When the Nazis came for the communists,
I remained silent;
I was not a communist.

When they locked up the social democrats,
I remained silent;
I was not a social democrat.

When they came for the trade unionists,
I did not speak out;
I was not a trade unionist.

When they came for the Jews,
I remained silent;
I wasn’t a Jew.

When they came for me,

there was no one left to speak out.

=======================

Click Here to talk to others worried about this

Want to know more? Click the image.

Want to fight? Click the image.

Will you remain silent?

Attacked by the Storm Botnet

Once upon a time there was no internet crime. Then humans came along…

Recently, we came under attack from the Storm / Nuwar Botnet. The post I made about it on the third of October: We had mis-identified it as a referral spam attempt. Close, but no cigar.

Now, I’ve always tried to keep my name & employer from becoming too widely spread on the interweb, although there is a couple of really, really easy ways you can find it, just from this website. (One of them being, ask me ;) )

It appears, that as a result of the two posts I’ve made about the Storm Worm, someone decided to DDOS not this blog, but my employer’s un-related servers, attacking one of our customers’ managed servers, and then our webmail server. (This blog is hosted from servers inĀ  the same rack as those servers.)

At its peak, the attack was drawing 8Mbps of data transfer. (About 1MB per second.)

Encryptec ddos attack

Graph is read from right to left. <<<<<<< Time Flows that way. <<<<<<<

You can see at 0930, when I got in work and started combating the attack. We only really stopped it the morning this graph just ends on…

Only problem, was that they were flooding our server with requests, literally using every available incoming connection on the server all the time.

For non-techies, a web site is hosted by a computer somewhere on the interweb,
that never gets turned off, connected to a really thick pipe to the internet.
Its configured to accept a certain number of new people visiting its website(s)
at once.

We’ve now completely mitigated this attack (to the point, where at most now its drawing 50kbps [6.5KB per second]). Technically, we can mitigate (and sustain) a much more serious attack. This was basically a “Get Lost, and STOP POSTING ABOUT US” poke.

An expensive poke. A sustained 8Mbps transfer rate is expensive in bandwidth!

So far (*wanders off to check*) we’ve identified 23,265 ip addresses which have tried to attack us. That’s a lot of infected computers, but it could have been worse.

It appears the attack has been petering out, we are identifying one new bad ip (infected computer) once every 30-60 seconds. At its peak, we were picking up at least one new ip every second.

If we have another look at that graph of the attack:Encryptec ddos attack

The attack started at 1AM GMT, and ramped up to full power in about 20 minutes. That means that it takes the Nuwar / Storm botnet about 20 minutes for a command to filter down into its bots.

At the beginning of the attack, the pattern we were seeing was a bad request from one ip, then 3 different bad requests, then back to the first IP. Sometime during the attack, I think about 1400 or 1500 (2 - 3pm) they switched to hitting us repeatedly from one ip address, showing that someone was probably monitoring at least a small part of this attack, and had noticed that we’d started to block the attacks.
Now, this happened quite a while ago.

Encryptec DDOS attack - 4 week

So why haven’t I posted about it yet? Why has it taken me 2 weeks to blog about this?

Because, its only now that we feel that we are able to safely weather another attack, should the Zhelatin Gang decide to start poking us again. If they didn’t like me posting what I have, they’re not going to like me posting this.

A message to them: I do not like bullies. Go pick on someone your own size for a change.
Thanks to stopddos.org, for analysing the logs and identifying Storm as our attackers.

UPDATE 20/10/07: A little while ago I sent a part of our logs for geographical analysis to one of the nice guys at castlecops.com.
Here is the graph that resulted from that. This is the top 5 attackers from country, in a pie chart. As you can see, Germany (Country Code DE) was the biggest, closely followed by the US. If you want to see other attack graphs, go here: http://www.spamtrackers.eu/wiki/index.php?title=Botnet_hosting (ours is listed there as BB, moved around to match up with the others, and slightly tweaked.)

Storm Worm Analysis (Take 2)

I’ve read quite a lot in my search on information about the Storm worm.

Capacity

Apparently, a better estimate of the Storm Worm Botnet’ current number of zombie machines is about 10 Million. As such, I’ve redone all my calculations (bottom of the article) with the updated numbers, and I’ve also spent some more time finding other numbers to remove some of the estimations from calculations.

I estimate, that the botnet currently has access to about 15,000 THz of CPU power. The fastest super computer currently in existence, Blue Gene L has 91.8 THz. So, this has fallen with the re-calculations.

I managed to find this report on the state of broadband in the US, which says that the average upload speed (all I’m interested in really) is about 371kb/s. So, I’ve recalculated all of my bandwidth calculations, working from that figure, as outside the US, e.g. canada, Japan, are likely to have much higher upload speeds. Also, Britain is starting to move to 448/812kb/s

About 442GB/s. Which, is equivalent to 339 Million emails per second, or 604 CDROMs worth of data every second.

Use

So, what have the Zhelatin Gang (group of crackers behind the Storm Worm) been up to with all this data capacity?

This report says that they are currently selling distribution capacity, as well as as of the 13th of August, testing their DDoS capacity.

This report from spamnation.info estimates that they are currently attacking a number of Anti- Spam / Malware sites. In fact, a large number of malware sites have / are under attack, including 419eater, which was basically overloaded with about 450GBs an hour worth of traffic, taking it off-line. CastleCops.com, is currently weathering the same high-level of incoming traffic.

Here is a graph of the traffic hitting 419eater.com. The attack took 419eater offline for a number of days, and they’re only coming back online now. They are still under attack, but have moved hosts, to someone who can cope with a massive amount of data incoming.

419eater DOS attack graph

At 11:44, traffic stops, as the site is taken offline, because the guys who hosted their website could no-longer cope with the sheer amount of incoming traffic.

Self Defence

The storm worm is (unfortunately for us) quite clever. It detects when its being used on what is called a virtual machine, a tool that some security researchers use to keep their PC safe from the trojan/virus, whilst they are trying to disassemble it.

Also, the botnet will launch a DDoS attack at any computer that either:

  1. Downloads the virus too many times (Researcher)
  2. Scans an infected computer for the basic signs of infection

I hope all this information is useful. The storm worm has quite worried me recently, and the only real way to combat it now, would be for the ISP’s to take action. Which they are not going to anytime soon - it does not make economic sense to do so.

My calculations are below. If you have any more up-to-date information for me to base them on, I’d love to hear from you. Leave a comment, or send me an email. My address is in the “about” page, linked above.

Calculations

All calculations are in computer-style notation, so * for multiplication, and / for division.

Processing Capacity (Zombies)

Assume 10 Million infected computers. 10,000,000.

Assume an average of 1.5Ghz processor in each computer. (Its probably more like 2.5Ghz, but safe side it.) 15,000,000 Gigahertz (Ghz)

15,000,000/1000 = 15,000 Terahertz (Thz)

Processing Capacity (Blue Gene L Super Computer)

Blue Gene L, has 131,072 Processors, each running at 0.7 GHz (700 Mhz).

131072 * 0.7 = 91750.4 Ghz

91750.4 / 1000 = 91.7504 Thz

Round to 1 decimal place = 91.8 Thz

Data Transfer (Zombies)

Assume 10 Million infected computers. 10,000,000.

Assume that each computer has about 371kb/s upload rate. (Probably a bit higher, but thats the average for the US, so safe-side it. 10 million is still a lot of computers…)

Get the 371 Kilobits into KiloBytes. 1 KiloByte = 8 KiloBits, so:

371/ 8 = 46.375KB/s per bot.

10000000 * 46.375 = 463,750,000KB/s transfer rate. Ok, that’s too mind-boggling. Lets get the numbers to be more sensible.

1 MegaByte = 1024 KiloBytes, so:
463750000 / 1024 = 452,880.859375MB/s. Not readable yet. Again.

1 GigaByte = 1024 MegaBytes, so:
452880.859375 / 1024 = 442.266464233GB/s

Err… I did do these sums right… *checks*. Wow.

Round to 0 decimal places = 442GB/s

Emails per second with 442GB/s bandwidth.

Assume an Average spam email size of 11.76 KB from This article, and rough confirmation from spamnation.info

From our bandwidth calculations above, there is 463,750,000 KB/s bandwidth available. So:

463750000 / 11.76 = 339434523.80952381 emails per second.

Round to 0 decimal places = 339,434,524 emails per second.

Round to 3 significant places = 339,000,000

CDs per second with 442GB/s bandwidth.

CD-ROM total size : 750MB.

From bandwidth calculations above, 452880.859375 MB/s.

452880.859375 / 750 = 603.841145833 CDROMS worth of data transfer per second.
Round to 0 decimal places = 604 CDROMS data per second.

Thanks to

Those on the CastleCops DDoS forum who helped provide data.. and the rest of the DDoS forum guys, for putting up with me whilst I find out more about the Storm / Nuwar botnet.

Scary analysis of Storm Worm

The Storm Worm. A virus and set of malware that has been spreading across the internet since January 2007. According to this article, it is now estimated that it has turned up to 50 Million computers into bots (or Zombie Computers), and is more powerful than a supercomputer.

So, I thought, a fine time to do some number crunching, to see if we can see approximately how powerful this bot-net is. (See here for the full article, with calculations at the bottom, if you’re looking at this from the homepage.)
The bot-net will probably have a maximum of around 40,000 Terahertz (THz) at its disposal. To put that into perspective, the worlds fastest supercomputer (The Blue Gene L) has around 91.8 Terahertz.

Ok, that is a really, really impressive amount of processing power. But what use is that power without being able to get data (spam emails etc.) onto the internet?

61 GigaBytes a second.

You heard me right. The bot network, will have the estimated capacity to pump 61 Gigabytes of data onto the internet per second.

At 20KB an email, thats 3,200,000 (Three Million Two Hundred Thousand), emails per second.
Revised email calculations (see note below): at 11.73KB an email, thats 5,442,177 emails per second.

Or, at 750MB per CDROM, thats 83.3 discs per second.
I think the term “Houston, we have a problem” doesn’t even come close to showing the amount of pain these guys can cause. At that rate of data transfer, this cracker (”hacker” for mass-media) group will be able to take any website in the world off-line, with a Distributed Denial of Service attack. They may even be able to take the internet offline, with another Backbone attack.
Now, this has been a almost-worst-case scenario, but trust me when I say, this is not good. I think, something drastic may be in order.

Note: Checkout my math and assumptions by clicking the “Read More” (if you’re not already in the full post). If you see any problems, or have some more-up-to-date information, feel free to register, and add a comment. (Sorry about forcing you to register. Spammers have been causing problems ><.) Alternatively, my email address can be found on the “about” page above.”
Update: Looks like the bot-net is being brought into play to attack anti-spammer websites.
http://www.spamnation.info/blog/archives/2007/09/419eater_ddosd.html

Update #2: The webmaster of spamnation.info has confirmed (roughly) the analysis of this article, which says the average spam email is 11.76KB. As such, I have revised my email calculations. I’ll probably post a follow-up to this, if I can get any more accurate data to go on, and I may re-do all of the calculations at some point, with a more conservative bot number estimate base.

Calculations

All calculations are in computer-style notation, so * for multiplication, and / for division.

Processing Capacity (Zombies)

Assume 40 Million infected computers, (even though the article says 50,000,000, lets err more on the safe side…) 40,000,000.

Assume an average of 1Ghz processor in each computer. (Its probably more like 1.5, but safe side again.) 40,000,000 Gigahertz (GHz)

40000000/1000 = 40,000 Terahertz (THz)

Processing Capacity (Blue Gene L Super Computer)

Blue Gene L, has 131,072 Processors, each running at 0.7 GHz (700 Mhz).

131072 * 0.7 = 91750.4 Ghz

91750.4 / 1000 = 91.8 Thz (Rounded to 1 decimal place)

Data Transfer (Zombies)

Assume 40 Million infected computers. 40,000,000.

Assume that each computer has about 128kb/s upload rate. (Probably closer to at least 256kb/s, but lets err on the safe site. 40 million is a lot of computers…)

Get the 128 Kilobits into KiloBytes. 1 KiloByte = 8 KiloBits, so:

128 / 8 = 16KB/s per bot.

40000000 * 16 = 64,000,000KB/s transfer rate. Ok, that’s too mind-boggling. Lets get the numbers to be more sensible.

1 MegaByte = 1024 KiloBytes, so:
64000000 / 1024 = 62,500/s. Not readable yet. Again.

1 GigaByte = 1024 MegaBytes, so:
62500 / 1024 = 61GB/s (rounded to 0 decimal places). Err… I did do these sums right… *checks*. Wow.

Emails per second with 61GB/s bandwidth.

Assume an Average spam email size of 20 KiloBytes (This article says 11.76KB, but its out of date, and source is offline. Err on safe side.)

From our bandwidth calculations above, there is 64,000,000 KB/s bandwidth available. So:

64000000 / 20 = 3,200,000 per second.

Emails per second with 61GB/s bandwidth (Revised calculations).

The 11.76KB appears to be accurate, so lets revise these calculations to take that into account.

Bandwidth 64,000,000 KB/s.

64000000 / 11.76 = 5,442,177 emails per second (Rounded to 0 decimal places.)

CDs per second with 61GB/s bandwidth.

CD-ROM total size : 750MB.

From bandwidth calculations above, 62,500 MB/s.

62500 / 750 = 83.3 CDROMS worth of data transfer per second. (Rounded to one decimal point)




Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK: England & Wales
Creative Commons Attribution-NonCommercial-ShareAlike 2.0 UK: England & Wales