Archive for the ‘Life the Universe and Everything’ Category
Bind Logs – Top DNS Queries
This week at work we fancied figuring out what the top sites are on are network. So decided an easy way to work this out would be to enable logging on are bind server and then write a script to work out what the top DNS queries are.
To enable logging in bind add the following to your `/etc/bind/named.conf`
logging {
channel simple_log {
file "/var/log/named/bind.log" versions 10 size 50m;
print-time yes;
print-severity yes;
print-category yes;
};
category default {
simple_log;
};
category queries {
simple_log;
};
};
this will output log entries every time a query is done. We can then parse the log files with the following script to get out just the url and a count.
#!/bin/bash echo "Top 20 Domains" echo "" cat /var/log/named/bind.log* | grep 'queries' | cut -d '/' -f 3 | sed 's/www.//' | sort | uniq -c | sort -nr | head -n 20
this script will output the top 20 in descending order with a count of the number of queries next to the url.
I then like to run the script every 2 minutes using watch.
watch -n 120 './bindLogCheck.sh'
running this should give you output like this
Top 20 Domains
6782 example.net
950 api.del.icio.us
600 test.com
600 something.com
536 mail.google.com
527 site.de
526 alpha.com
526 delta.com
526 gamma.co.uk
526 digitalforensicsmagazine.com
386 pablumfication.co.uk
200 google.com
192 safebrowsing-cache.google.com
189 safebrowsing.clients.google.com
97 google-analytics.com
83 facebook.com
74 googleads.g.doubleclick.net
74 bbc.co.uk
70 uk.mg40.mail.yahoo.com
70 capa.org
Any questions or comments as usual I would love to hear them.
Research:
Get All Installed Packages on System in apt-get Format
Recently I needed to setup three almost identical web boxes. First of all I installed all three boxes with a base install of Ubuntu 10.04 server. I then installed all the required packages on one box and started looking for a way to replicate the installed packages on the other boxes.
I had a Google around and did not manage to find a way to do it so I decided to come up with a method by myself.
On the box where you have all your installed packages do
dpkg --list | grep "^ii" | cut -f3 -d ' ' | sed ':a;N;$!ba;s/\n/ /g' > installedOnSystem01
if you check the outputted file it should look something like this
adduser apache2 apache2-mpm-prefork apache2-utils apache2.2-bin apache2.2-common apg apt apt-utils aptitude base-files base-passwd bash bash-completion ...
once you know the file is ok, copy it to your second system using your preferred method for example scp
scp installedOnSystem01 username@system02:
then on the second system run the apt-get command in simulation mode
sudo apt-get -s install `cat installedOnSystem01`
this should give you output like this
... 0 upgraded, 30 newly installed, 0 to remove and 6 not upgraded. ...
if this looks ok remove the `-s` and run the command again to install your packages.
That’s it job done, the systems should now have the same installed packages. If anyone has any questions or can offer a way to shorten the process please leave a comment.
Ubuntu 10.04 – Fix Button Layout
For anyone running Ubuntu 10.04 the new default positioning of the minimize, maximize and close buttons can be quite annoying.
Previously I have been using `gconf-editor` to edit the settings manually whenever I change themes. But here are two quicker ways.
First of all you can just use `gconf-editor` running it from terminal and specifying the parameter of the key you want to edit. The GUI will then launch with the value you want to edit already selected.
gconf-editor /apps/metacity/general/button_layout
Now Double click on `button_layout`, change it to read ‘:minimize,maximize,close’, click ok and the change should take effect.
The second way is to use the `gconftool` which allows you to directly edit gconf repository values without using a GUI.
In a terminal session run
gconftool -g /apps/metacity/general/button_layout
It will return what the current button layout is set to. Which should read
close,minimize,maximize:
To edit this just do
gconftool -s /apps/metacity/general/button_layout -t string :minimize,maximize,close
Your button layout should now be correct.
WPA2 Hole196 Vulnerability
Last month the Wi-Fi Alliance started steps to put an end to WEP and TKIP. By January 2011 the Wi-Fi Alliance plans to disallow TKIP on access points, and disallowed its use on all WiFi devices by 2012. WEP unfortunately survives a bit longer, with the standard being banned on access points from 2013 and banned from all WiFi devices a year later. WPA2-Mixed mode which allows TKIP, will also go in 2014, leaving only WPA2-AES.
With that good news last month, bad news comes this month for the security of Wi-Fi standards.
AirTight Networks have uncovered a vulnerability that they call “Hole196″ ( The 196 referring to the last line of Page 196 in the IEEE 802.11-2007 specification ), this is a vulnerability in the WPA2 security protocol that exposes WPA2-secured Wi-Fi networks to malicious insiders. With WPA2 being vastly adopted as the most robust option for many environments due to its resilience to brute force dictionary attacks, this vulnerability will effect both corporate and public networks significantly.
By exploiting the vulnerability, an insider can bypass WPA2 private key encryption and authentication to sniff and decrypt data from other authorized users as well as scan their devices for vulnerabilities, and potentially allow attackers to compromise users devices. AirTight researcher, Md. Sohail Ahmad, will be demonstrating this vulnerability at the Black Hat Arsenal (July 29th) and at DEFCON18 (July 31st) in a presentation entitled “WPA Too?!”.
The “WPA Too” presentation will demo the vulnerability and explain how it can be exploited by a malicious user to attack and compromise a legitimate user.
For the people not lucky enough to attend either security conference, AirTight will present a public Webinar on August 4 at 19:00 GMT to detail its findings.
Once the details of the vulnerability are disclosed it will be time to determine what steps and countermeasures can be used to protect wireless network infrastructure. But for now all that can really be done is to break out the VPN tunnels whenever using Wi-Fi. This can at least protect against your data being intercepted but there is still the potential for the attacker to disrupt the targets network traffic.
References:
Credit where credit is due, these are the sites I read when looking into this vulnerability.
Goodbye, WEP & TKIP
WPA2 Hole196 Vulnerability
WPA2 finds itself in a “hole”! Vulnerable to insider attacks!
Black Hat ® Technical Security Conference: USA 2010 // Black Hat Arsenal
WPA2 Exposed with ‘Hole 196′ Vulnerability
Wi-Fi WPA & WPA2 Encryption Cracking Guide
Wi-Fi Alliance to dump WEP and TKIP … not soon enough
Researcher Hints 802.1X WPA2 Flaw
unrar All Files in Directory
For quite a while I have found it a pain that unrar does not allow you to extract multiple files at once.
The other day I finally figured out how to get past this. Simply use xargs with place holders.
This command will unrar all the files in the current directory and any subdirectories to the directory /home/user/directory/
find . -iname "*.rar" | xargs -i unrar x {} /home/user/directory/
htop & nmon
Whenever I run a screen session I always used to have top running in screen 0 by default.
This was till recently when a friend recommended `htop` to me which is a great enhancement on top and I now use this by default.
Just the other day I found another alternative to using `top` which is `nmon` although its not as customisable as `htop` it still has some good features, so I thought I would do a quick post on both and also share my configs which I think work well.
htop
htop is an advanced, interactive system monitor process viewer designed as a replacement for the top program.
Unlike `top`, htop provides a full list of processes running, instead of the top resource consuming processes. htop also uses colour and gives visual information about processor, swap and memory status.
htop is configurable to have extra information displayed such as battery level and other information that can’t be shown with top.
Also the colouring and formatting of the top like output can be configured to improve the usability.
To install under ubuntu its simply a case of
sudo apt-get install htop
To install under Gentoo
sudo emerge htop
Below is my htop configuration which I have tweaked a bit. You can give it a go with mine or just fire up htop yourself and have a mess around with the config till you find what works for you. One of the nicest things about htop is that if you make a configuration change it automatically saves on exit.
# Mark Davidson (mark@4each.co.uk) .htoprc
# Beware! This file is rewritten every time htop exits.
# The parser is also very primitive, and not human-friendly.
# (I know, it's in the todo list).
fields=0 48 17 18 38 39 40 2 46 47 49 1
sort_key=47
sort_direction=1
hide_threads=0
hide_kernel_threads=1
hide_userland_threads=0
shadow_other_users=0
highlight_base_name=1
highlight_megabytes=1
highlight_threads=1
tree_view=1
header_margin=1
detailed_cpu_time=0
color_scheme=0
delay=15
left_meters=AllCPUs Memory Swap
left_meter_modes=1 1 1
right_meters=Tasks LoadAverage Uptime
right_meter_modes=2 2 2
nmon
nmon is a tool designed to give a huge amount of information all on one screen. The tool can be used to get a wealth of performance statistics, allowing you to get the stats on one screen that would normally take 5 or 6 other tools.
I have only had chance to try it out recently as its only been available in the Ubuntu repository since 10.4 (Lucid Lynx).Also its not available under the Gentoo portage tree so this limits its availability quite a bit.
However it is easy to compile from source.
To install on Lucid Lynx simply `sudo apt-get install nmon`.
Once installed fire it up by just typing nmon then when the application launches press h to see the options for configuration.
I found it fairly annoying having to configure the application every time it launches. I could not find a configuration file to save the config but after reading the manual I found you can set the default behaviour using an enviromental variable.
To do so you just do `export NMON=cmknt` best to add this line to do your .bashrc to make it survive reboots etc.
Zend_Form apply Zend_Filter to all form elements
So recently at work I have been doing a lot of work with the Zend Framework, but I was having some issues with magic_quotes_gpc
being turned on in the PHP configuration of the server. The easy thing to do would be to disable magic_quotes_gpc in the php.ini and that solves the issue. However this was not an option in my case, as I didn’t want to fiddle with the configuration in case it broke anything else.
I had a look around for a solution and found one on Phil Brown’s Web Development Blog his post
Zend Framework Forms and magic_quotes_gpc which explains how to use Zend Filters to combat this issue. His solution works perfectly but you have to set the filter for each form element individually and I found this a bit of a chaw.
So I looked around to find a way of applying it to all elements within the form.
Here is the solution I came up with this is all the code you need to get it working. I am doing this for the Strip Slashes filter but will work with any standard or non standard filters.
First you want to add the filter to your library. So create in your library directory the folder `MoreCowbell` then inside that the folder `Filter` and finally create the file `StripSlashes.php` which contains the following.
<?php
//FILE: library/MoreCowbell/Filter/StripSlashes.php
class MoreCowbell_Filter_StripSlashes implements Zend_Filter_Interface {
public function filter($value)
{
return get_magic_quotes_gpc() ? $this->_clean($value) : $value;
}
protected function _clean($value)
{
return is_array($value) ? array_map(array($this, '_clean'), $value) : stripslashes($value);
}
}
?>
Next you need to create a class that extends Zend_Form. In my case I have created it and added it as part of my Pablumfication library.
<?php
//FILE: library/Pablumfication/Zend/Form.php
class Pablumfication_Zend_Form extends Zend_Form {
// Configure path to custom plugins
public $elementPrefixPaths = array('filter' => array(
'prefix' => 'MoreCowbell_Filter',
'path' => 'MoreCowbell/Filter'
));
// Shortcut to default element filters
public $elementFilters = array(
'StripSlashes'
);
public function __construct($options = null) {
parent::__construct($options);
$this->addElementPrefixPaths($this->elementPrefixPaths);
$this->setElementFilters($this->elementFilters);
}
}
?>
Once that’s done its simply a case of a ensuring that any forms you create extend this one. Then when the form is constructed it will automatically apply the correct element filters and element prefixs to each element within the form.
<?php
class Application_Form_Article extends Pablumfication_Zend_Form {
public function init() {
$this->setName('ArticleForm');
$this->setMethod('post');
$this->addElement('text', 'heading', array(
'required' => true,
'label' => 'Heading:'
));
$this->addElement('textarea', 'content', array(
'required' => true,
'label' => 'Content:'
'cols' => 55,
'rows' => 16
));
$this->addElement('text', 'photoS', array(
'required' => true,
'label' => 'Photo Small:'
));
$this->addElement('text', 'photoL', array(
'required' => true,
'label' => 'Photo Large:'
));
$this->addElement('submit', 'Submit');
}
}
?>
At the moment its only the strip slashes filter that is automatically applied but if for instance you wanted StringTrim to be applied to every form element you would just modify. The declaration of `elementFilters`.
public $elementFilters = array( 'StripSlashes' );
to
public $elementFilters = array( 'StripSlashes', 'StringTrim' );
GNU Binutils – strings
A really quick post this time to just take note of the strings tool which is part of the Binutils collection of binary tools
The strings tool allows you to print all printable characters in a given file or files. This can be very useful for a number of different tasks from extracting metadata from a file to retrieving a certain level of information from unknown executables such as what URLs it has contained within it.
So how to install it on your system.
Under Ubuntu or Debian
sudo apt-get install binutils
Under Gentoo
sudo emerge sys-devel/binutils
Now for a usage example lets say you have a PDF file if you do `strings mypdf.pdf` you will get output similar to this.
%PDF-1.3 1 0 obj /Pages 2 0 R /Type /Catalog endobj 2 0 obj /Type /Pages /Kids [ 3 0 R ] /Count 1 endobj 3 0 obj /Type /Page /Parent 2 0 R /Resources << /XObject << /Im0 8 0 R >> /ProcSet 6 0 R >> /MediaBox [0 0 1075 720] /CropBox [0 0 258 173] /Contents 4 0 R /Thumb 11 0 R endobj 4 0 obj /Length 5 0 R stream 258 0 0 172.8 0 0 cm /Im0 Do endstream endobj 5 0 obj endobj 6 0 obj [ /PDF /Text /ImageC ] endobj 7 0 obj endobj 8 0 obj /Type /XObject /Subtype /Image /Name /Im0 /Filter [ /RunLengthDecode ] /Width 1075 /Height 720 /ColorSpace 10 0 R /BitsPerComponent 8 /Length 9 0 R stream endstream endobj 9 0 obj 3096046 endobj 10 0 obj /DeviceCMYK endobj 11 0 obj /Filter [ /RunLengthDecode ] /Width 106 /Height 71 /ColorSpace 10 0 R /BitsPerComponent 8 /Length 12 0 R stream endstream endobj 12 0 obj 30105 endobj 13 0 obj endobj 14 0 obj 30105 endobj 15 0 obj endobj 16 0 obj 30105 endobj 17 0 obj /Title (My PDF) /CreationDate (D:20100310110120) /ModDate (D:20100310110120) /Producer (ImageMagick 6.4.5 2009-06-04 Q16 OpenMP http://www.imagemagick.org) endobj xref 0 18 0000000000 65535 f 0000000010 00000 n 0000000059 00000 n 0000000118 00000 n 0000000301 00000 n 0000000386 00000 n 0000000404 00000 n 0000000442 00000 n 0000000463 00000 n 0003096696 00000 n 0003096719 00000 n 0003096747 00000 n 0003126997 00000 n 0003127019 00000 n 0003127035 00000 n 0003127057 00000 n 0003127079 00000 n 0003127101 00000 n trailer /Size 18 /Info 17 0 R /Root 1 0 R startxref 3127298 %%EOF
As you can see you get quite a lot of information out of it follow that with a grep you can easily extract specific pieces of data. For example `strings mypdf.pdf | grep Title` will return.
/Title (My PDF)
Not terribly exciting I do appreciate but shows a quick example. Its one of these things that is damn useful when you need it and not terribly exciting the rest of the time.
Suddenly the Dungeon collapses!! – You die…
So I came into work the other day and one of the SSH sessions I had left open with a screen session had an interesting error.
Suddenly the Dungeon collapses!! - You die...
Now that is one of the weirdest errors I have ever seen come out of any program on any platform. Apparently the error is a throw back to the game nethack which is kind of cool.
My screen session seems to have died with some memory issue which is something I have never seen happen before
May 14 04:44:42 chronos view: *** glibc detected *** view: double free or corruption (!prev): 0x0000000000936d50 *** May 14 04:44:21 chronos kernel: [4894494.908208] screen.real[3934]: segfault at 7fffaa1d6a10 rip 4055ee rsp 7fffaa1d6a10 error 6
Just a post to note what this error is related to and that if it happens to you just check your syslog most likely a similar error will show up.
Gentoo eix
eix is a tool that allows for fast and highly flexible searching of the gentoo portage system. There is a guide of how to get it installed and basic usage instructions on the Gentoo Wiki.
In this post I would just like to point out the two most useful commands I have found while using eix.
First of all the format argument is one of the most powerful features of eix. I have found it to be very useful when using the available versions parameter. Which allows you to see all the available versions of a particular program, available on the portage tree.
Here is an example of using it to find all the avaliable versions of nmap within the portage tree
eix --format '<availableversions:NAMEVERSION>' -e nmap
Which will give you an output like
net-analyzer/nmap-4.76 net-analyzer/nmap-4.85_beta9 net-analyzer/nmap-4.90_rc1 net-analyzer/nmap-5.00 net-analyzer/nmap-5.00-r2 net-analyzer/nmap-5.10_beta1 net-analyzer/nmap-5.20 net-analyzer/nmap-5.21
Then if you wanted to install say version 5.20 of nmap you can just do
emerge =net-analyzer/nmap-5.20
Another really useful command is multi installed which as is implied allows you to find all packages that have multiple versions installed.
The command with it arguments for this is
eix -i
Since the output from this is quite big on my system I will give a different command and its output for example. This command does the same as mentioned above but is limited by category of ‘dev-lang’ and also used verbose output to make it a bit clearer.
The resulting command is
eix -C dev-lang -v -i
Which reveals that I have quite a few versions of python installed
Those are the quick two commands I wanted to point out for now. I highly recommend checking out eix as it makes a really good alternative to qsearch. When using eix checking out the manual is an absolute must as its over 50 pages long and gives you a true idea of the ammount of functionality avaliable.
