binfalse

I just installed a new server from HP, a ProLiant DL180 G6. Here are some notes about the setup.

To check the hardware status you need to install the ProLiant Support Package. Running a Debian/Ubuntu you should import the HP PSP mirror in your sources.list . It can be found here, you might include something like:

deb http://downloads.linux.hp.com/SDR/downloads/proliantsupportpack/Debian stable current/non-free

After an aptitude update you’ll find some new packages. I recommend to install hpaclui to speak to your raid-controllers and hp-health to interact with your hardware.

With hpaclui you can ask the raid-controllers for some information:

usr@srv % hpacucli ctrl all show status

Smart Array P123 in Slot 1
   Controller Status: OK
   Cache Status: OK
   Battery/Capacitor Status: OK

usr@srv % hpacucli ctrl slot=1 show config

Smart Array P123 in Slot 1                (sn: SOMESN  )

   array A (SAS, Unused Space: 0 MB)


      logicaldrive 1 (99.99 GB, RAID 1, OK)

      physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 99 GB, OK)
      physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS, 99 GB, OK)
      [...]

   array B (SAS, Unused Space: 0 MB)


      logicaldrive 2 (99.99 TB, RAID 5, OK)

      physicaldrive 1I:1:3 (port 1I:box 1:bay 3, SAS, 99 TB, OK)
      physicaldrive 1I:1:4 (port 1I:box 1:bay 4, SAS, 99 TB, OK)
      [...]

   Expander 250 (WWID: SOMESN, Port: 1I, Box: 1)

   Enclosure SEP (Vendor ID HP, Model SOMEMD) 248 (WWID: SOMESN, Port: 1I, Box: 1)

   SEP (Vendor ID SOMEVNDR, Model  SOMEMD) 249 (WWID: SOMESN)

So you get an idea of your storage.

The hp-health packages comes with a tool called hpasmcli . It’s used to query all the hardware states:

usr@srv % hpasmcli -s "SHOW"

Invalid Arguments
         SHOW ASR
         SHOW DIMM
         SHOW FANS
         SHOW HT
         SHOW NAME
         SHOW PORTMAP
         SHOW POWERMETER
         SHOW POWERSUPPLY
         SHOW SEL
         SHOW SERVER
         SHOW TEMP
         SHOW TPM
         SHOW UID

usr@srv % hpasmcli -s "SHOW POWERSUPPLY"

Power supply #1
        Present  : Yes
        Redundant: Yes
        Condition: Ok
        Hotplug  : Not supported
Power supply #2
        Present  : Yes
        Redundant: Yes
        Condition: Ok
        Hotplug  : Not supported

Both tools are very easy to use and give a great overview about the health. So I immediately developed a monitoring plugin that parses the output of those runs. I came to the point, that I wasn’t able to find some documentation about the hpasmcli tool. Most of its output was clear, but I don’t know what happens if a fan breaks. The output with working fans looks like:

usr@srv % hpasmcli -s "SHOW FANS"

Fan  Location        Present Speed  of max  Redundant  Partner  Hot-pluggable
---  --------        ------- -----  ------  ---------  -------  -------------
#1   SYSTEM          Yes     NORMAL  45%     Yes        0        No            
#2   SYSTEM          Yes     NORMAL  43%     Yes        0        No            
#3   SYSTEM          Yes     HIGH    100%    Yes        0        No            
#4   SYSTEM          Yes     HIGH    100%    Yes        0        No            
#5   SYSTEM          Yes     NORMAL  22%     Yes        0        No            
#6   SYSTEM          Yes     NORMAL  21%     Yes        0        No            
#7   SYSTEM          Yes     NORMAL  47%     Yes        0        No            
#8   SYSTEM          Yes     NORMAL  46%     Yes        0        No

So what if a fan is broken? Is it still Present and the Speed -string just changes to NONE or something like that? I send a support request to HP, but all they respond was a premium-rate number to call. Seems that my understanding of service differs from theirs. Since I don’t know how the output looks like in an error case (I don’t want to stick pencils into new machines) the plugin can’t decide whether the fans are OK. If you want to use my plugin you need to skip fan-checks until HP publishes a document with possible values. IMHO a public tool should be open source, so I can get those information on my own, or at least well documented!

Btw. HP if you read this, please include some permanent links to your web interface ;-)

Today a lot of great mails arrived at my inbox. In one of them I was reading I’ve just added your feed to the site.

Where did this mail come from?

The sender of the email was Tal Galili. He is a researcher in BioStatistics at the Tel Aviv University, very active around the internet. He also originated R-Bloggers and in this email he told me that I’m recruited ;-)

What is R-Bloggers

R-Bloggers is an aggregation of more than 200 bloggers writing about GNU’s R and some statistics/math/hacks that can be done using R. If you didn’t heard about R-Bloggers I strongly recommend to take a look at their website. I’m following this project since a while, it’s a great fusion of brainiacs! So I’m proud to have my modest R-related articles listed between them.

Btw. if you like R-Bloggers and/or have some R experiences yet you should also take a look at the ‘R’ programming Wikibook. Contributing your knowledge is greatly appreciated!

Let’s see what the future brings, happy hacking!

…sts (of course!). Yesterday I just installed a rating plugin, inspired by these stackexchange platforms.

Searching through the WordPress plugin directory didn’t make me happy. All existing plugins lack of desired features. After some tests I decided to modify UpDownUpDown of Dave Konopka. It’s a nice plugin, but still didn’t match my criteria. For example guests were not allowed to vote, there were some XHTML bugs and I didn’t like the style. So I created a patch (it’s attached..) and sent it to Dave (I don’t yet have a github account). He told me that he’ll take a look at it and might apply it to the official plugin, so if you also want to use this rating plugin with my additional features just keep the URL in mind and watch out for a new version.

The special version I’m using here right now has of course some more slight changes, to have it perfectly matched to my own blog. So you are now able to vote for articles, positive or negative, to give me a hint what my visitors like to read ;-)

I additionally installed a further page that lists my articles sorted by votes: top. So you can get a smart overview of best/worst content.

With this in mind: Happy voting! ;-)

Ever worried about these disgusting packets leaving your network interface!? Why not creating your own packets?

Of course it’s more than nonsense creating all packets on your own, but sometimes there might be a reason making you wish you could.. For ex. for my last article I searched for a possibility to modify some contents of a packet. First I thought about using iptables , but than I found a nice tool: scapy!

With scapy you can create your own packets, IP/TCP/UDP whatever! It is very heavy but comes with an user-friendly interface. Using Debian/Ubuntu you need to install python-scapy :

aptitude install python-scapy

To open the interface just run scapy . You can easily create an IP packet by typing something like this:

>>> ippacket=IP()
>>> ippacket.dst='binfalse.de'
>>> ippacket.ttl=12
>>> ippacket
<IP  ttl=12 dst=Net('binfalse.de') |>

So an IP packet is stored in the variable ippacket . This packet will be send to binfalse.de and has a ttl of 12 (if there are more than 12 network nodes between your machine and the target it will disappear and never arrive at the target). Let’s create some TCP stuff:

>>> tcpcrap=TCP()
>>> tcpcrap.sport=1337
>>> tcpcrap.dport=80
>>> tcpcrap
<TCP  sport=1337 dport=www |>

We stored some TCP information in tcpcrap . This packet will be send through your port 1337 and hopefully arrive at port 80 (in general a webserver is listening on port 80 ). That’s it for the networking part. Last but not least we will create some data to send:

>>> data='GET / HTTP/1.1 \\nHost: binfalse.de\\n\\n'
>>> data
'GET / HTTP/1.1 \\nHost: binfalse.de\\n\\n'

Combining all parts we’ll get a very nice packet, sending it will trigger my webserver to send the main page of my website (Sending exactly this packet won’t ever result in any website from my webserver. Why? Just think about…):

>>> whole=ippacket/tcpcrap/data
>>> whole
<IP  frag=0 ttl=12 proto=tcp dst=Net('binfalse.de') |<TCP  sport=1337 dport=www |<Raw  load='GET / HTTP/1.1 \\nHost: binfalse.de\\n\\n' |>>>
>>> send(whole)
.
Sent 1 packets.

Well done! Ok, that’s very much to do. But fortunately it’s just that much code for explanation, you can send the same packet in a single line:

>>> send(IP(ttl=12,dst='binfalse.de')/TCP(sport=1337,dport=80)/'GET / HTTP/1.1 \\nHost: binfalse.de\\n\\n')
.
Sent 1 packets.

Very smart, isn’t it? You can also sniff whooshing packets! But something like this I won’t explain, find out by yourself ;-)

I often here people saying something like

SSH to your home PC? Sitting behind a NAT? A snowball's chance in hell...

But is it really impossible?

What is a NAT?

NAT (network address translation) is a technique to cover multiple clients behind one router. Kristian Köhntopp explained the technology very well in his article NAT ist kein Sicherheitsfeature (GER). But let me summarize some things. Here is a small image to visualize the topology of an example network:

You see, the NAT represents something like a bridge between it’s clients (in network 10.0.0.0/24 ) and the rest of the world. The connections of the clients are translated by this router. Assuming client 10.0.0.3 wants to speak to my webserver 87.118.88.39 , he sends a packet containing, among others, the following information:

Source:      10.0.0.3:39478
Destination: 87.118.88.39:80

So all machines on the way from 10.0.0.3 to 87.118.88.39 know whom to send the packet next. When this packet arrives at the NAT, the NAT will rewrite it. The NAT stores a table for all recent connections. Each entry consists of a client IP, client port and a local port on its public interface. For our example the table entry for this example might look like:

The resulting port on the NAT is arbitary, it’s just one free port.. Each packet arriving on port 1234 of the public interface of the NAT is forwarded to 10.0.0.3:39478 . Our rewritten packet 10.0.0.3->87.118.88.39 now contains the following informations:

Source:      88.66.88.66:1234
Destination: 87.118.88.39:80

and is send to the next node in the world wide web. Nobody out of 10.0.0.0/24 will ever know that there is a machine 10.0.0.3 requesting a website from 87.118.88.39 . The webserver on 87.118.88.39 will send it’s answer to the pretended source, 88.66.88.66:1234 , and the NAT will forward the traffic according to its table entry to 10.0.0.3 .

Why do NAT’s exists? The solely plausible reason seem to be the lack of IPv4 addresses. With a NAT an ISP just need to offer a single IP address for a huge bunch of clients. Hopefully this will change in times of IPv6!

Why does it seem to be impossible?

Since the private network 10.0.0.0/24 is not known by the outer world (it is simply not route-able in the Internet, see wikipedia), you cannot connect from outside 10.0.0.0/24 straight to 10.0.0.3 . The WWW will only see 88.66.88.66 as source for all the clients. That means all clients in 10.0.0.0/24 have the same public IP for each machine that is not in 10.0.0.0/24 . So how to access 10.0.0.3 ? Speaking to 88.66.88.66 will result in crap, you don’t know which port will be forwarded to whom!? If it is forwarded at all…

How is it nevertheless possible?

Method one...

…is not very nice, if you are looking for a real solution please skip this paragraph and continue with solutions two and three ;-) Since there is no entry in the NAT table that specifies an outside target, you can send packets from any location to 88.66.88.66:1234 and the NAT will forward them to 10.0.0.3:39478 (according to my example). So to create a path from outside to 10.0.0.3 ‘s SSH server you just need to send a packet from 10.0.0.3:22 to any server outside that informs you about the source IP and source port that was reported by the NAT (it’s the address that will be forwarded to the client). If you immediately connect to this address, and if a SSH server is listening on 10.0.0.3:22 , you should be able to establish a SSH session. Simple isn’t it ;-) To get this working you could try something like repeating the following commands frequently:

# stop SSH to free the port
/etc/init.d/ssh stop

# send a packet to somewhere that is configured to notify you about the NAT settings
# the foreign side should instantaneously close the connection
# so the netcat command doesn't block your script
netcat -p 22 binfalse.de 1337

# restart SSH to get ready for connections
/etc/init.d/ssh stop

Of course you can also install some iptables rules to rewrite the TCP packets. So you can send the packets from some other ports than 22 , iptables will rewrite them so the target machine (and the NAT) thinks they came from :22 . With this setup you don’t have to stop SSH, because you don’t need the free port… But just hack it your way ;-)

Method two...

…is much more comfortable. You can set up a reverse SSH tunnel! Again you need another machine outside the NAT, that has a SSH server running and will act as your gateway. Just connect to it from your local machine behind the NAT:

ssh -R 1337:localhost:22 you@your.server

That will open the port 1337 on your.server . All packets arriving at this port are transferred through the SSH tunnel to your home PC. Run something like screen or top on the server to always transfer packets (otherwise the connection will be closed after some time), with -o ServerAliveInterval=XXX you can adjust the threshold for closing the SSH connection. Surround it with a while loop and you’ll reestablish closed connections (network errors or something like that):

while [ 1 ]; do ssh -o ServerAliveInterval=60 -R 1337:localhost:22 you@your.server; done

By default the opened port is just bound to 127.0.0.1 (the servers loopback interface), so you can only send packets from the server itself (or need some more network hacking). To have this hack listening to 0.0.0.0 (all interfaces) add the following to your /etc/ssh/sshd_config on your.server :

GatewayPorts yes

and restart the daemon.

Method three...

…might be the most elegant. Set up a VPN! But that’s too much for now, request some explanations from 3dfxatwork, he’s your OpenVPN guy!, and take a look at Dirty NAT tricks to get a VPN to work with clients also numbered in the private address space

So you see, no hasty prejudices ;-)