Challenge is over.

SEMS challenge
SEMS challenge

About 6 or 10 moths ago we were searching for a student to work with us in the SEMS project. In order to reduce the number of applications I started a challenge. To solve this challenge you had to show some understanding for basic techniques and programming languages, so we didn’t waste our time with people not able to write a single line of source code.

And what should I say? It was successful! We’re now a great team with three students :D

However, currently this challenge seems to spread over the internet. And lot’s of people try to solve it (and many submit a wrong answer^^). But even worse, some of you guys try to exploit it by submitting something like

"; SHOW TABLES;

In general I don’t care. It was just some lines of PHP that send me an email in case of a correct answer. There is no database and the worst that can happen is a full inbox, but now I decided to close this challenge and instead forward users to this article.

Thus, if you arrive here feel free to apply for a job! I guess all of my readers, even if they didn’t solve this challenge, are perfect fellows…

If you nevertheless want to give it a try you can download the challenge.

Extended MyTinyTodo

MyTinyTodo is a self-hosted todo-list which convinces by its simplicity. It allows to maintain several different lists, you can assign tags, priorities and due dates to certain tasks. I used it myself for a long time and decided to fork the project in order to implement some stuff I missed in the original version.

 Figure 1: MyTinyTodo Result
Figure 1: MyTinyTodo Result

I do not intend to talk about MyTinyTodo a great deal. Very tiny, does nothing that isn’t necessary. No Dropbox/Facebook/Instagram etc integration. I really like this kind of software :D

But I was missing an essential feature: Creating tasks via mail. Lucky us, MyTinyTodo is distributed under the terms of GPLv3 license. Thus, I hg clone d and extended the tool with desired functionality. And since the IDE was already opened I added a tiny authentication (now: username + password; previously: .htaccess ) and secured the API by introducing a signature. Nothing special or complex, but it had to be done.

Long story short: I’m now able to submit tasks via e-mail. That means, a mail containing the following:

To: todo@your.server.tld
Subject: My New TodoItem
some more text

to describe this todo item

priority:1
tags:someTag1,someTag2
duedate:nextweek
list:myNewList

will result in something similar to Figure 1. All possible attributes that are recognized in the mail body are listed at the wiki on GitHub.

Find out more on GitHub.

Integrating Tomcat with Apache

You can configure the Apache web server to forward requests to Tomcat. Thus, you can speak to both servers on ports 80 or 443 and get rid of the :8080 for your Tomcat applications. I’m somehow doing that very often, so here is small how-to for copy&paste purposes.

Install jk

As you might know, while Tomcat is Java stuff Apache is written in C. So in general it’s not that easy to get them talking to each other. The key to achieve an integration is called mod_jk (see The Apache Tomcat Connector). So first of all you need to install it:

aptitude install libapache2-mod-jk

If it is installed you can configure an AJP worker in /etc/libapache2-mod-jk/workers.properties :

# Defining a worker named ajp13_worker and of type ajp13
# Note that the name and the type do not have to match.
#
worker.ajp13_worker.port=8009
worker.ajp13_worker.host=localhost
worker.ajp13_worker.type=ajp13

As soon as this is done the bridge is ready to close the gap between Apache and Tomcat.

Configure Tomcat

We need to configure an AJP connector on port 8009 . So open /etc/tomcat7/server.xml and add another connector next to the other ones:

<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" address="127.0.0.1"/>

If you’re lucky there is already such a connector defined in the comments. So just remove the comment…

Configure Apache to speak through jk

Here I’ll show you how to setup a virtual host. For example, copy the following to /etc/apache2/sites-available/012-yourapp.conf :

<VirtualHost *:80>
      ServerAdmin some@body.tld
      ServerName yourapp.yourserver.tld
      ServerAlias ya.yourserver.tld

      RewriteEngine on
      RewriteRule ^/(.*)$ /YourApp/$1 [L,PT]

      JkMount /* ajp13_worker
</VirtualHost>

Ok, let me shortly explain what I did there.

  1. Everything that arrives at this vhost gets forwarded to our previously defined AJP worker (line 9)
  2. I assume your Tomcat webapp is running on server:8080/YourApp , therefor I configured a substitution of the URL to insert /YourApp (line 7). Of course you need to have mod_rewrite installed and enabled. (You may skip this line if you’re fine with having /YourApp in all your URLs)
  3. The rest should be clear. The vhost is available at http://yourapp.yourserver.tld , as well as at http://ya.yourserver.tld (lines 3&4). You can also use SSL, just configure line 1 to listen at *:433 and add the SSL stuff to the body of your vhost. (SSL exmaple)

Afterwards, enable the vhost to populate it:

a2ensite 012-yourapp

Give it a try

If this is done just restart everything:

service tomcat7 restart
service apache2 restart

Now Apache forwards all requests to http://yourapp.yourserver.tld to your Tomcat webapp at http://yourserver.tld:8080/YourApp .

Find all Text Files, recursively

Because I was thinking of something like that for a long time.

In bash/zsh (add it to your .rc ):

textfiles ()
{
    file $(find $*) | /bin/grep -E 'text|empty' | cut -d ':' -f1
}

Using this function it’s possible to open all text files of a project at once:

kate $(textfiles project/*)

Change Title of moderncv Document

Once again I had to prepare a CV for an application. I’m using the moderncv package to create the CV in and I was always bothered about the title of the document. Today I spend some time to fix that.

 moderncv produces an ugly title
moderncv produces an ugly title

Using moderncv you can produce really fancy CV’s with very little effort. But unfortunately, by default it produces an ugly title (see the screenshot taken from Okular). As you can see, there is some character that cannot be displayed by certain tools.

I guess most of my “CV-reviewers” don’t care about this little issue, if they recognize it at all, but it bothers me whenever I have to create a resumé. I already tried to override it using the hyperref package, but wherever I put the statement it seems to have no effect.

However, since moderncv is open source (yeah! lovit) I took a look at the code to see how they produce the title. It was quite easy to find the concerning statement (in my case /usr/share/texlive/texmf-dist/tex/latex/moderncv/moderncv.cls:96 , texlive-latex-extra@2012.20120611-2):

\AtEndPreamble{
  \@ifpackageloaded{CJK}
    {\RequirePackage[unicode]{hyperref}}
    {\RequirePackage{hyperref}}
    \hypersetup{
      breaklinks,
      baseurl       = http://,
      pdfborder     = 0 0 0,
      pdfpagemode   = UseNone,% do not show thumbnails or bookmarks on opening
      pdfpagelabels = false,% to avoid a warning setting it automatically to false anyway, because hyperref detects \thepage as undefined (why?)
      pdfstartpage  = 1,
      pdfcreator    = {\LaTeX{} with `moderncv' package},
%      pdfproducer   = {\LaTeX{}},% will/should be set automatically to the correct TeX engine used
      bookmarksopen = true,
      bookmarksdepth= 2,% to show sections and subsections
      pdfauthor     = {\@firstname{}~\@familyname{}},
      pdftitle      = {\@firstname{}~\@familyname{} -- \@title{}},
      pdfsubject    = {Resum\'{e} of \@firstname{}~\@familyname{}},
      pdfkeywords   = {\@firstname{}~\@familyname{}, curriculum vit\ae{}, resum\'{e}}}
  \pagenumbering{arabic}% has to be issued after loading hyperref
}

As expected the pdftitle contains a double-hyphen that is converted by latex to a dash. Apparently a problem for some programs. To fix this issue you could sudo:modify this file, but that’s of course messy. Better add something like the following to the end of the header of your document:

\AtEndPreamble{
\hypersetup{pdftitle={Your New Title}}
}

This will override the broken output of the package.

Check if certain Port is Open

Just needed to get to know whether something listens at a certain TCP port on a particular host.

Here is my workaround using Perl:

my $sock = IO::Socket::INET->new (
	PeerAddr => "1.2.3.4",
	PeerPort => 1337,
	Proto => "tcp",
	Timeout => 1
);
echo "closed" if !defined $sock;

Works at least for me. Any concerns or better solutions?

The Password Dilemma

Earlier this week I had a very small conversation with Pedro Mendes on twitter (well in terms of twitter it might be a long dicussion). It was initiated by him calling for suggestions for a password safe. I suggested better using a system for your passwords, which he thought was a bad idea. So lets have a look at both solutions.

You all know about these rules for choosing a password. It should contain a mix of lower and upper case letters, numerals, special characters, and punctuation. Moreover, it should be at least eight characters long and has to be more or less random. Since our brain is limited in remembering such things we tend to use easy-to-remember password (e.g. replacing letters using leet). But of course hackers are aware of that and it is quite easy to also encode such rules in their cracking algorithms. Equally bad is using one strong password for all accounts. So, how to solve this problem?

Using a Password Safe

The first good idea is using very strong passwords for every account and writing them down, so you don’t have to remember them. You probably often heard that writing passwords on a sheet of paper is a very stupid idea. And storing them in a document on your desktop is even worse. But there are lots of tools that help you with that problem, e.g. KeePass or KeePassX, just to name two open source solutions. You can organize your passwords and store them in an encrypted file. Thus, you just have to remember one single password to open this safe. These tools often include an excellent password generation functionality that helps you choosing passwords. And even if another website gets hacked, you just need to open your safe and replace the password with a new one. Very convenient.

Unfortunately, this solution also comes with some drawbacks. Since you cannot remember a single password you always need to have this safe with you. I usually use five different machines, so you have to distribute this file (at least to have a backup). And of course you want to have it in sync, so you might want to store it in a cloud or something. But every copy of this safe increases the chance that an attacker gets access to it. Moreover, you cannot put a subsafe containing only passwords for trivial accounts (like twitter) on your mobile phone (which I also do not trust). So, there are many weak points to get access to the safe (e.g. a design fail of the cloud, a bug in the cloud, an evil system administrator having access to your PCs at work, a law enforcements etc.). And as soon as the attacker has access to this file he just has to crack one human rememberable password to see the whole collection of your passwords. Probably including login names and links to the web sites. Worst case scenario.

Using a Password System

The second idea is using a system to generate passwords for each account. You have to choose a very strong password , and a function that creates a unique password for every account using and the (domain) name of the related service: . You just need to remember this very good and . Depending on your paranoia and your mind capabilities there are many options to choose . An easy might just put the 3rd and last letters of at the 8th and 2nd pos in (see example below). More paranoid mathematicians might choose an that ASCII-adds the 3rd letter of to the 8th position of , puts the at the 2nd position in , and appends the base64 representation of the multiplicative digital root of the int values of the ASCII letters of to . Here you can see the examples:

u:M~a{em0 twitter ur:M~a{eim0 u2.6:M~a{eW0Mi4yNDU2MjFlKzE0Cg==
u:M~a{em0 google ue:M~a{eom0 u2.4:M~a{e]0MS40MjU4MjNlKzEyCg==

So, you see if the password for twitter gets known the hacker isn’t able to log into your google account. To be honest, I guess that nobody will choose , but I think even is quite good and leaves some space for simple improvements.

However, as expected this solution also has some dramatic disadvantages. If one of your passwords gets compromised you need to change your system, at least choosing a different and maybe also an alternative for . As soon as a hacker is able to get two of these passwords he will immediately recognize the low entropy and it is not difficult to create a pattern for your passwords making it easy to guess all other passwords.

Conclusion

This is not to convince somebody to use one or the other solution, its more or less a comparison of the pros and cons. In my opinion the current password mechanism is sort of stupid, but we need to find the least bad solution until we have some alternatives. So what about creating a small two-factor auth system? You could combine the two above mentioned solutions and use a password safe in combination with a password system. So keep a short lock in mind which is necessary to unlock the passwords in the safe. Maybe something like 29A which you have to add to every password (on some position of your choice, e.g. just append it). Thus, if a hacker breaks into one service only a singe password is compromised and you just need to update this entry in your safe, and if your whole safe is cracked all passwords are useless crap. Of course you have to create a new safe and update all your passwords, but the guy who knows your old “passwords” doesn’t know how to use them. However, we are discussing on a very high level. The mentioned scenarios are more or less just attacks against a particular person. I am a sysadmin, so I would already be very glad if users won’t use passwords like mama123 and stop sending passwords in clear-text mails!

Supp: The Conversation

just for the logs (in twitter chronology: new -> old):

Pedro Mendes @gepasi at 1:13 PM - 30 May 13
@binfalse I agree, but using 30 character completely random ones seems to be the best.

martin scharm @binfalse at 5:40 PM - 29 May 13
@gepasi either using a password safe (which also has drawbacks) or a system with a strong p and a complex f.

martin scharm @binfalse at 5:39 PM - 29 May 13
@gepasi however, i support the attitude seeing every pw as compromised. so the most important rule is using unique pws for every service.

martin scharm @binfalse at 5:39 PM - 29 May 13
@gepasi even after reading this article i’d say that ur:M~a{eim0 is quite strong and i’d expect to find it within the 10% uncracked.

Pedro Mendes @gepasi at 1:18 PM - 29 May 13
@binfalse but thanks for the tip on KeePassX

Pedro Mendes @gepasi at 1:18 PM - 29 May 13
@binfalse a system is not recommended. Anything a human can remember is broken within 24h. Read http://arstechnica.com/security/2013/05/how-crackers-make-minced-meat-out-of-your-passwords/

martin scharm @binfalse at 1:03 PM - 29 May 13
@gepasi and even if someone breaks into twitter, your google passphrase (“ue:M~a{eom0”) isn’t compromised.

martin scharm @binfalse at 1:03 PM - 29 May 13
@gepasi quite easy to remember (when you know p), very hard to guess and brute-forcing the related hash really takes some time.

martin scharm @binfalse at 1:03 PM - 29 May 13
@gepasi e.g. p=”u:M~a{em0” and n=”twitter” would result in “ur:M~a{eim0” as a password for twitter.

martin scharm @binfalse at 1:02 PM - 29 May 13
@gepasi you just need to remember p and f, which may put the 3rd and last letter of n at the 8th and 2nd pos in p.

martin scharm @binfalse at 1:02 PM - 29 May 13
@gepasi choose a password p (as strong as possible) and a function f(p,n) that creates a unique password from p and a (domain) name n.

martin scharm @binfalse at 1:02 PM - 29 May 13
@gepasi afaik KeePassX is a good one. but i recommend to use a system!

Pedro Mendes @gepasi at 9:07 AM - 29 May 13
I need suggestions for a good password manager. Ideally only local storage (ie no cloud storage)

wp-login.php Brute Force Defense

Currently observing a lot of brute force attacks trying to get access to my WordPress installation. Fortunately, I’ve been aware of such cranks when I installed WordPress, and now I want to share my technique to prevent such attacks.

What's the problem?

There are some guys who try to get access to your website’s content to distribute even more spam and malware. Since they don’t have your credentials they need to guess them. Usually they randomly choose common login names (like admin or martin ) and popular passwords (like root123 or martin ) and try to log in to your web site. However, there are lot’s of possibilities and only a few will work, so they usually need a lot of attempts. To prevent an intrusion you should choose an uncommon user name and a strong password (not only for your WordPress installation!). Nevertheless, there is still a chance to guess the credentials, so you’ll sleep much better if you make sure that there’s no chance for an attacker to break into your site.

Deny access to wp-login.php

The idea is to reject any login from anyone but you. For instance, using apache (most common webserver) you may only allow the access to wp-login.php from defined IP adresses:

<Location /wp-login.php>
   ErrorDocument 403 /
   Order deny,allow
   Deny from all
   Allow from 1.2.3.4
</Location>

This piece of code, included in a vhost or in an .htaccess file, will only allow connections from 1.2.3.4 to your wp-login.php . All other requests will be forwarded to / . You need to have the module mod_access installed. For more information take a look at the documentation of the mod_access . Other web servers like nginx or lighttpd have similar solutions. (And I think hope even the microsoft crap is able to do such basic stuff without much configuration overhead, but I’m too busy to read microsoft documentations…)

Workaround for dynamic IPs

As long as you’re editing your articles using static IP everything is fine. But what if you’re cursed with an NAT? Indeed, it will be very annoying if you always have to adjust this config in order to log into your WordPress management interface! Fortunately, there is a small workaround if you have SSH access to that server. Simply restrict the access to the file to connections from the server’s own IP. Thus, only connections from the server itself are able to log in. In order to get access you need to setup a tunnel to your server using SSH providing a socks proxy:

ssh -D8765 you@your.web.server

This command will create a tunnel from your local system to your.web.server . Connections to port 8765 at your systems will be forwarded to your server, hence, connections to your wp-login.php through the tunnel will be allowed. From now on only users having access to the server (physically or via SSH) are allowed to access you wp-login.php :-) There’s only one restriction left: you need to SSH to your server and you have to configure your browser to use this socks proxy before you can access WordPress. I recommend to use FoxyProxy.

Testing

Ok, let’s ensure that our config works. Try to access wp-login.php from an IP which is not allowed to access this file, e.g. using curl :

usr@client % curl -I /wp-login.php
HTTP/1.1 302 Found
[...]
Location: /
[...]

Since I’m not allowed to access this page I got a 302 and am redirected to / . Ok, what happens if I connect from an allowed host?

usr@srv % curl -I /wp-login.php
HTTP/1.1 200 OK
[...]

Excellent, 200 == allowed! If you want to verify your proxy connections using curl pass another parameter -x socks5://127.0.0.1:PORT to the command:

usr@client % curl -x socks5://127.0.0.1:8765 -I /wp-login.php
HTTP/1.1 200 OK
[...]

Great, everything’s fine :D

More Security

Of course you can add similar rules for other web sites or scripts. For example to restrict the access to the whole admin interface of WordPress add another restriction to the vhost / .htaccess :

<LocationMatch ^/wp-admin>
   ErrorDocument 403 /
   Order deny,allow
   Deny from all
   Allow from 1.2.3.4
</LocationMatch>

I’m sure you’ll find even more reasonable rules.

KDE file type actions

Annoyingly, KDE’s PDF viewer okluar always opened links to websites with an editor presenting me the source code. But I just figured out how to change this behavior.

 kcmshell4 dialog to configure filetype-application-mappings
kcmshell4 dialog to configure filetype-application-mappings

KDE maintains a central config defining what to do with certain file types. Unfortunately, in my case the default application for HTML files was an editor (kwrite/kate). I don’t know who or what defined this stupid behavior, but there is a tool called kcmshell4 to edit the KDE configuration. That said, to edit the filetype-application-mapping hand it the parameter filetypes :

usr@srv % kcmshell4 filetypes

You’ll get a dialog to define a mapping for each known file type. In my case I had to configure okular to open links to HTML pages with konqueror. Hope that helps you to save some time ;-)

encfs: transparent crypto overlay

encfs is a cryptographic file system (encfs-website). The principle is very easy, you “bind-mount” one directory (containing the crypt stuff) to another directory (where it’s unencrypted). Thus, you can share the encrypted stuff and nobody but you can read your data. So this system is excellent applicable for cloud services like Dropbox, which trap you with some space in the cloud “for free”, but want you to share your private data with them. In some <p>’s we’ll see how to setup encfs for Dropbox, but let’s first take a look at encfs itself.

encfs

First of all you have to install encfs. Assuming you’re sitting in front of a Debian-based os:

root@abakus ~ # aptitude install encfs

Since encfs is fuse-based the user who wants to use encfs has to be member of the group fuse . You may use the groups command to make sure you belong to fuse :

martin@abakus ~ % groups
martin mail fuse

If you’re not yet member of that group edit the /etc/group file or use the useradd command (howto). To apply the changes you need to re-login or use newgrp (man newgrp).

That’s it, now the way to use encfs is parved. Let’s say we want to work with our data in /dir/clear , while the whole stuff is stored encrypted in /dir/crypt . It’s quite simple to setup this environment, just call encfs [crypt-dir] [decrypt-dir] :

martin@abakus ~ % encfs /dir/crypt /dir/clear
Creating new encrypted volume.
Please choose from one of the following options:
 enter "x" for expert configuration mode,
 enter "p" for pre-configured paranoia mode,
 anything else, or an empty line will select standard mode.
?>

Give it a p and choose a password. This command will install an encrypted container in /dir/crypt and immediately mount it to /dir/clear . Feel free to create some files in /dir/clear (you’re new working directory) and compare this directory with /dir/crypt . You’ll see that you are not able to understand the files in /dir/crypt , unless you’re a genius or the setup failed. Thus, it’s no issue if anyone might have access to the content in /dir/crypt .

To unmount the clear data use fusermount -u /dir/clear . To remount it just call again encfs /dir/crypt /dir/clear , it will just ask you for the password to decrypt the data.

Of course it’s not very convenient to mount the directory every time manually, hence there is a workaround to automount your encfs directories on login. You need to install libpam-mount and libpam-encfs :

root@abakus ~ # aptitude install libpam-mount libpam-encfs

To automatically mount an encfs on login the password for the crypt-fs has to be the same as the password for your user account! If that’s the case, add a line like the following to /etc/security/pam_mount.conf.xml :

<pam_mount>
    [...]
    <volume user="martin" fstype="fuse" path="encfs#/dir/crypt" mountpoint="/dir/clear" />
    [...]
</pam_mount>

On your next login this directory will automatically be mounted. Very smart!

Using encfs for the cloud

Ok, let’s get to the main reason for this article, winking towards Norway ;) . As you might have heard, there are some file hosting services out there, like Dropbox or Ubuntu One. They provide some space in the cloud which can be mounted to different devices, so that your data is sync’ed between your systems. Unfortunately, most of these services want to read your data. E.g. the Dropbox system wants to store a file used by multiple users only once. Even if they pretend to assure that nobody’s able to read your private data, you all know that this is bullshit nearly impossible! However, I strongly recommend to not push critical/private files to these kind of providers.

But, thada, you’ve just learned how to sync your files using the cloud while keeping them private! Let’s assume the directory /home/martin/Dropbox is monitored by Dropbox, you just need to create two more directories, like /home/martin/Dropbox/private (for encrypted files to be sync’ed) and /home/martin/Dropbox-decrypt (for decryption). Mount /home/martin/Dropbox/private to /home/martin/Dropbox-decrypt using encfs and work in /home/martin/Dropbox-decrypt . As explained above, feel free to setup an automount using pam ;-) Everything in /home/martin/Dropbox but not in the private directory will be sync’ed unencrypted, so you won’t loose the opportunity to share some open data with [whoever] e.g. via web browser.

Of course, this method comes with some drawbacks:

  • It is a bit more work to setup every client, before you can start working with your private data (fortunately the overhead is kept in reasonable limits)
  • You cannot access these files through the web browser, or using your mobile phone (unless your phone comes with encfs-support)

All in all, you need to decide on your own, how much you trust Dropbox (and alike) and which kind of data you commit to Dropbox unencrypted.