CUViper

The diff command is an essential piece of any programmer’s or administrator’s toolbox, to show the difference between two files.  Or if not that tool exactly, there are many like it.  But sometimes I want to compare the output of two commands, often really the same command operating on different inputs.  You could save the output of each to separate files and diff that, or in a shell like bash you can use “<(cmd)” fed directly to diff.  For example, when I was recently dealing with an elfutils bug with prelinked binaries, I ran a command like this:

diff <(eu-readelf -hlS foo) <(eu-readelf -hlS foo.prelink)

That works just fine, though it’s a little redundant, but it gets really bad if the command gets much more complex.  And if you need to adjust the command, you have to do it in both places, or the diff falls apart.  So after getting annoyed by this, I decided to scratch that itch and write this “pdiff” script:

#!/bin/bash

CMD=
CMDARG=
DIFF=
OPTS=$(getopt -o d:gvc:I: -n pdiff -- "$@") || exit
eval set -- "$OPTS"
while true; do
    case "$1" in
        -d) DIFF="$2"; shift 2;;
        -g) DIFF=gvimdiff; shift;;
        -v) DIFF=vimdiff; shift;;
        -c) CMD="$2"; shift 2;;
        -I) CMDARG="$2"; shift 2;;
        --) shift; break;;
         *) echo "Internal error!" >&2; exit 2 ;;
    esac
done

function run() {
    local cmd
    test -n "$CMDARG" && cmd="${CMD/$CMDARG/$@}" || cmd="$CMD $@"
    echo "# $cmd"
    eval $cmd
}

case $# in
    2) exec ${DIFF:=diff -u -p} <(run $1) <(run $2);;
    3) exec ${DIFF:=diff3} <(run $1) <(run $2) <(run $3);;
    4) test -n "$DIFF" &&
       exec $DIFF <(run $1) <(run $2) <(run $3) <(run $4);;&
    *) echo "Unable to diff $# args" >&2; exit 1 ;;
esac

It even has option processing, like -v to open in Vim, -g for gVim, or -d for a custom diff command.  Then -c gives a shared command, and -I lets you specify a replacement string within the command (rather than placing arguments at the end).  Any remaining arguments get fed to separate invocations of the command for comparison.  That’s well beyond the one-liner I started with, but now I can reuse this to make future comparisons easier.  The first command I gave is now more simply:

pdiff -c 'eu-readelf -hlS' foo foo.prelink

That command can now get more complex, with compound pipelines or whatever else I need, without being such a pain to compose.

I did a little searching afterward and found that there are already other tools called pdiff, like here and here, so if I ever want to publish this more formally I should find a new name.  But for now, it meets my needs, and maybe it can be useful for someone else too.

I recently read an LWN article on David A. Wheeler’s essay, “Fixing Unix/Linux/POSIX Filenames.”  The gist is that he thinks the filename rules are too permissive — we have ‘/’ as the path separator, and a raw 0 terminates the path, but anything else is fair game.  On the surface, this has a certain beautiful simplicity to it.  However, there are characters that have special meaning depending on the context, so almost any code that actually tries to interpret a filename will have to add a lot of complexity to be robust.  The essay delves into many ways that things can go wrong.

Filenames have been this way in for a long time though, and I don’t expect that this will change officially anytime soon.  Still, my day job now is developing SystemTap, and this sort of problem is one of many sorts that SystemTap can address.  Here’s a script to show how a system administrator could patch the kernel with their own addendum to the filename rules:

#!/usr/bin/stap -g
# badname.stp
# Prevent the creation of files with undesirable names.

# return non-zero if the filename should be blocked
function filter:long (name:string)
{
  return euid() && isinstr(name, "XXX")
}

global squash_inode_permission
probe kernel.function("may_create@fs/namei.c")
{
  # screen out the conditions which may_create will fail anyway
  if ($child->d_inode || $dir->i_flags & 16) next

  # check that the new file meets our naming rules
  if (filter(kernel_string($child->d_name->name)))
    squash_inode_permission[tid()] = 1
}
probe kernel.function("inode_permission@fs/namei.c").return !,
      kernel.function("permission@fs/namei.c").return
{
  if (!$return && squash_inode_permission[tid()])
    $return = -13 # -EACCES (Permission denied)
  delete squash_inode_permission[tid()]
}

The script starts by defining a filter function.  It first check whether the effective user ID is non-zero, so the root user can bypass the filter.  Then, for the prude admins out there, I’ve chosen to block filenames that contain the string “XXX”.  I intentionally kept this part small for this example, but you could easily write a function covering all of the new rules that Wheeler suggests.

After that is a probe on the may_create function, which is what the kernel calls to validate permissions for new files.  We can call our filtering function from here to see if the filename is OK, but since may_create is an inline, we don’t have a direct way to influence its result.  The last thing may_create does though is copy the result of inode_permission (or permission in earlier kernels), which we can override.  So, we save the filtering decision in a global, and then in a return probe on inode_permission, we can change the successful $return code to our own error value.  Now, any attempt to create a file that doesn’t pass our rules will get an error of “Permission denied”.

This sort of script is really just a band-aid, and it doesn’t do anything to deal with files that already have “bad” names.  Still, I hope this is an interesting example of how easily one can modify kernel behavior with SystemTap.  This script can be a starting point to define and try out your own filename rules, and changes can be reloaded and tested without ever having to reboot.  Once your policy has been decided, you can configure the script to load as soon as the system boots, so you’re always running with your improved filename rules, even across kernel upgrades.

It’s powerful stuff, but don’t let it get to your head…

I made an upgrade on my netbook today.  Can you spot the difference?

If you guessed that it’s now 100% open-source compatible, you are correct!

Even though I ordered the Linux package, Dell cheaped out with a Broadcom wireless card that doesn’t have very good Linux support.  Broadcom does have drivers available, but you have to download and compile the wrapper yourself.  Downloading a new driver and all of the kernel-devel packages is a little harder when your network is not connected…

So to replace the Broadcom, I ordered an Intel 3945ABG card, because it is well supported in Linux.  The kernel has the right drivers already included, so Fedora works right out of the box, even booting off of a live cd.  And not only are the drivers open-source, but they even work better.  My connection time shrunk from 20-30 seconds down to about 5 seconds.  I can live with that!