binfalse

Independent of your programming experiences, you should have learned that regular expressions are more or less write-only.

Write-only? What is he talking about!? Actually I revisited some Perl code with a relatively short reg-ex. Do you think I was able to understand what I’ve thought when I created that piece of code? Not in the slightest!

But there is a smart modifier, that enables you to comment your regular expressions: x. With /x all white-spaces are ignored and with an unescaped # the rest of the line is treated as a comment. I found a nice example, what do you think is this expression for:

/^1?$|^(11+?)\\1+$/

No idea? Don’t even bother, I’m also stumped… Here is the solution: It’s used to check for prime numbers ;-) Using the x-mod the explanation looks much more readable (via Neil Kandalgaonkar):

/
  ^1?$   # matches beginning, optional 1, ending.
         # thus matches the empty string and "1".
         # this matches the cases where N was 0 and 1
         # and since it matches, will not flag those as prime.
|   # or...
  ^                # match beginning of string
    (              # begin first stored group
     1             # match a one
      1+?          # then match one or more ones, minimally.
    )              # end storing first group
    \\1+            # match the first group, repeated one or more times.
  $                # match end of string.
/x

So you see, it’s really helpful to use the x-modifier. At least for your own understanding :-P

A bit more explanation can be found on Perl.com.

Today I played a bit with JRI as part of rJava, a Java-R-interface. Here you can learn how to setup for Debian/Ubuntu/akins.

Installation

Assuming you have a running version of Java and GNU’s R, you have to install r-cran-rjava :

aptitude install r-cran-rjava

Shell environment

To talk to R through Java you have to specify three more environmental variables. First of all you need to publish you R installation path, my R is found in /usr/lib64/R :

export R_HOME=/usr/lib64/R

If you didn’t or the path is wrong you’ll fall into trouble:

R_HOME is not set. Please set all required environment variables before running this program.

Second the $CLASSPATH needs to get an update. Precisely you have to add the archives JRIEngine.jar , JRI.jar and REngine.jar . In my case all of them can be found in /usr/lib/R/site-library/rJava/jri/ , so the $CLASSPATH should be set like that:

export CLASSPATH=.:/usr/lib/R/site-library/rJava/jri/

If the $CLASSPATH isn’t defined correctly you won’t be able to compile your Java code.

Last but not least you have to add the native JRI-library to your $LD_LIBRARY_PATH , by default this lib is located in the same directory like the jar’s:

export LD_LIBRARY_PATH=/usr/lib/R/site-library/rJava/jri/

If the $LD_LIBRARY_PATH isn’t proper you’ll experience errors like this:

Cannot find JRI native library!
Please make sure that the JRI native library is in a directory listed in java.library.path.

java.lang.UnsatisfiedLinkError: no jri in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1734)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1028)
        at org.rosuda.JRI.Rengine.<clinit>(Rengine.java:19)

To not always do the same you might write these export stuff to your .bashrc or .zshrc respectively.

Eclipse setup

Of course in Eclipse you’ll also have to define these three things. Where are the jar’s located? Add them to your libraries in Project > Properties > Java Build Path > Libraries. Instead of the $LD_LIBRARY_PATH you can set the java.library.path in Run > Run Configurations > Arguments. Add -Djava.library.path=.:/usr/lib/R/site-library/rJava/jri/ to the VM arguments (modify the path to match your criteria). The R_HOME can be published in Run > Run Configurations > Environment. Create a new variable with the name R_HOME and the value /usr/lib64/R (or an equivalent path). That’s it, see the section above to identify what went wrong if something fails.

Netbeans setup

Two of these three parts are also straight forward in Netbeans. First publish the location of the jar’s. Right-click on your project and choose Properties > Libraries. In the Compile-tab click Add JAR/Folder and search for the jar files. Next task is to adjust the library-path. Right-click on your project and choose Properties > Run. Add -Djava.library.path=.:/usr/lib/R/site-library/rJava/jri/ to the VM Options (modify the path to match your criteria). The third step is a little tricky. As far as I know there is no way to change the environment from within Netbeans, so you can’t create the variable R_HOME after Netbeans is started. In my opinion you have two options:

1. Export the variable before starting Netbeans:

   usr@srv $ export R_HOME=/usr/lib64/R
   usr@srv $ netbeans
   

you might want to write a wrapper script that does this step for you, or include the export in any of the resource files that are called before Netbeans starts (e.g. your .bashrc ).

1. Change the environment from within your project. At stackoverflow you can find a workaround, but I think this is a very lousy solution..

If you have further suggestions please let me know! Meanwhile George Bull published a setup guide for Netbeans on Windows hosts. Seems to be worthy to take a look at it ;-)

Testcase

If you defined your environment properly, you should be able to utilize the REngine. I have a small script for you to test whether all things are fine:

package de.binfalse.martin;

import org.rosuda.JRI.Rengine;

public class JRItest
{
  public static void main (String[] args)
  {
    // new R-engine
    Rengine re=new Rengine (new String [] {"--vanilla"}, false, null);
    if (!re.waitForR())
    {
      System.out.println ("Cannot load R");
      return;
    }

    // print a random number from uniform distribution
    System.out.println (re.eval ("runif(1)").asDouble ());

    // done...
    re.end();
  }

}

You should be able to compile and run it, afterwards you’ll see a random number from an uniform distribution. Congratulations, well done :-P

For more information see the JRI and rJava sites at RForge.net.

I have bad news for those of you trying to produce lucid code!

In his blog Radford M. Neal, Professor at the University of Toronto, published an article with the headline Two Surprising Things about R. He worked out, that parentheses in mathematical expression slow down the run-time dramatically! In contrast it seems to be less time consuming to use curly brackets. I verified these circumstances to be true:

> x=10
> f <- function (n) for (i in 1:n) 1/(1*(1+x))
> g <- function (n) for (i in 1:n) (((1/(((1*(((1+x)))))))))
> system.time(f(10^6))
   user  system elapsed 
  2.231   0.000   2.232 
> system.time(g(10^6))
   user  system elapsed 
  3.896   0.000   3.923 
> 
> # in contrast with curly brackets
> h <- function (n) for (i in 1:n) 1/{1*{1+x}}
> i <- function (n) for (i in 1:n) {{{1/{{{1*{{{1+x}}}}}}}}}
> system.time(h(10^6))
   user  system elapsed 
  1.974   0.000   1.974 
> system.time(i(10^6))
   user  system elapsed 
  3.204   0.000   3.228

As you can see adding extra parentheses is not really intelligent concerning run-time, and not in a negligible way. This fact shocked me, because I always tried to group expressions to increase the readability of my code! Using curly brackets speeds up the execution in comparison to parentheses. Both observations are also surprising to me! So the conclusion is: Try to avoid redundant parentheses and/or brackets!

To learn more about the why you are referred to his article. He also found a interesting observation about squares. In a further article he presents some patches to speed up R.

Ok, R is very well-considered in certain respects, but there are also some things annoying me… This time it’s scoping…

Let’s have a look to the following code:

fun=function()
{
	if (runif(1) > .5)
		x = 1
	x
}

First it looks damn unspectacular. But wait, whats that:

> x=0
> fun()
[1] 1
> fun()
[1] 0

Taking a closer look to the function shows that the returned value is randomly chosen from local ( runif(1) > .5 ) or global scope ( runif(1) <= .5 ). So you can’t expect a result from this function. Nasty, especially while debugging external code, isn’t it? :-)

> sum(sapply(1:10^6, function (null) fun()))/10^6
[1] 0.499681

So again my advise: Think about such specific features! This won’t happen in any sensible language…

Sitting on an almost well configured host, I experienced some authentication issues the last few days…

So for example I’m using xtrlock as default X locking mechanism, but if I try to run it on this machine I got the following error:

/tmp % xtrlock
password entry has no pwd
1 /tmp %

Mmh, that is crap. My workaround to temporarily avoid this problem: Connecting to another host via SSH, running xtrlock within a GNU screen session ;-) But that’s no solution for a longer time… So I started debugging. First of all I grabbed the sources from the apt repository and searched for this error message. Turned out to be this piece of code (beginning with line 94 of xtrlock.c ):

errno=0;  pw= getpwuid(getuid());
  if (!pw) { perror("password entry for uid not found"); exit(1); }
#ifdef SHADOW_PWD
  sp = getspnam(pw->pw_name);
  if (sp)
    pw->pw_passwd = sp->sp_pwdp;
  endspent();
#endif

  /* logically, if we need to do the following then the same 
     applies to being installed setgid shadow.  
     we do this first, because of a bug in linux. --jdamery */
  setgid(getgid());
  /* we can be installed setuid root to support shadow passwords,
     and we don't need root privileges any longer.  --marekm */
  setuid(getuid());

  if (strlen(pw->pw_passwd) < 13) {
    fputs("password entry has no pwd\\n",stderr); exit(1);
  }

Ok, seems that the provided password(-hash) is shorter than 13 characters… Going on debugging, the content of pw comes from getpwuid(getuid()) and seems to be ok (matches my users profile like it can be found in /etc/passwd ). At this time (line 1) pw->pw_passwd contains only an single x , more information can’t be retrieved from the passwd -file.. Next the code checks whether SHADOW_PWD is defined, means whether we use an additional shadow -file. Since thats the case this code is executed and the variable sp gets the broken-out fields of the record in the shadow password database that matches the username pw->pw_name (validated, my user). Checking this sp variable I recognized that it is null ! So pw->pw_passwd won’t be updated and still contains the single x from the passwd entry… First I thought about a bug in the getspnam () function, such things might happen due to the Debian unstable release I’m using, but after some further thoughts I checked the shadow file itself:

/tmp % l /etc/shadow
-rw-r----- 1 root root 2673 Feb 16 15:49 /etc/shadow

In comparison with other systems with working xtrlock instances I figured out, that this file shouldn’t only be owned by root. Instead the group has to be shadow! So here is the solution to this issue:

/tmp % chgrp shadow /etc/shadow

And everything is working fine again. Have no idea what or who changed the permissions for the shadow-file…

Update: By the way, afterwards I tried to use Xscreensaver instead of xtrlock, but I wasn’t able to unlock the screen when the shadow rights are wrong. The /var/log/auth.log held messages like that:

Feb 17 10:14:32 HOST xscreensaver: pam_unix(xscreensaver:auth): conversation failed
Feb 17 10:14:32 HOST xscreensaver: pam_unix(xscreensaver:auth): auth could not identify password for [USER]

But this is just for google-searchers ;-)