Posts for the month of January 2008

I wanna rsync all night long...

I use rsync a lot on my PC boxes (Cygwin) and Linux servers. I keep forgetting what I do where, so usually I have a file called "command" so I can just ". command" with bash.

Anyway, here are a few so I remember what I did. Maybe you will find them helpful too:

Backup music drive (M:, ext3) to external USB drive (X:\!Music, NTFS)

cd /cygdrive/x/\!Music
rsync -av --size-only --exclude command --exclude '!Books on MP3' --exclude '!FTP' /cygdrive/m/ ./

Backup external USB drive (X:, NTFS) to external FireWire drive (I:, FAT32)

(Yes, I backup my backup, long story...)

cd /cygdrive/i
rsync -av --size-only --exclude command --exclude *iso --exclude '/System Volume Information' /cygdrive/x/ ./

Keep my Cygwin mirror up to date on Mjolnir (Linux server)

cd /share/shared/Cygwin_mirror
rsync -va --stats --delete --exclude command --exclude /mail-archives --progress rsync://mirrors.xmission.com/cygwin/ ./ && touch date
wget http://www.cygwin.com/setup.exe
mv -f setup.exe ../

Getting Tcl/Tk stuff to work on Fedora 8

I have a third party app that has an installer that is written in Tcl/Tk. Apparently, my Fedora 8 install has a busted install of wish, the shell it uses. Here's the ass-backwards way I got it to run.

The error was "Application init failed ... This probably means that Tcl wasn't installed properly."

First, I ran tclsh manually to figure out where it was looking - puts $auto_path shows me. It was looking in /build/xfndry8/J.36/env/TDS/Tcl/dist/export/lib instead of, oh, I dunno, maybe /usr/share/tcl8.4/tcl8.4 . However, Tk looks at /usr/lib/tk8.4 instead of /usr/share/tk8.4 ...

mkdir -p /build/xfndry8/J.36/env/TDS/Tcl/dist/export/lib
cd /build/xfndry8/J.36/env/TDS/Tcl/dist/export/lib
ln -s /usr/share/tcl8.4/tcl8.4 /build//J.36/env/TDS/Tcl/dist/export/lib
cd /usr/lib
mv tk8.4 tk8.4_orig
ln -s /usr/share/tk8.4 tk8.4
cp tk8.4_orig/pkgIndex.tcl tk8.4

Note: This is all scratched on the back of a Dilbert calendar sheet so there may be a typo or two.


Follow-up

Apparently, one of my yum updates broke it. I needed to re-run the mv and ln commands. For the record, I'm now at:

tcl-8.4.17-1.fc8
tk-8.4.17-2.fc8

I decided to file a bug...


Follow-up 2

I should've known - when all else, fails, blame Xilinx. If I had paid attention, I would've realized that Xilinx tools version 8 would be xfndry8 and J.36 is their internal build number (IIRC 9.2SP4 is J.40).

Reminder email and wiki page for trac

I wrote this a long time ago, never had a chance to update it, and somebody asked on the trac mailing list today...

You can get ADMMailHandler in the ADMLogger package (sorry, too lazy today).

You need to configure a few places too. Like I implied, not quite ready for public consumption, but at my rate never will be. Please feel free to "upstream" me any changes.

Also, if there is a trac project that matches the user name (that's how my company does trac sandboxes) then it will also create a page MyTickets with a dump of links.

#!/usr/bin/perl -w
use strict;

# Script to send email reminders in a cron job.
# Email trac at revragnarok <.> com

# I should've made it DBI compliant - but I'm not getting paid for this ;)

BEGIN {
   push @INC, "/usr/src/my_scripts/mail_reminder"; # Where ADMMailHandler is hiding
}

use Data::Dumper;
use ADMMailHandler;
use DirHandle;

use constant URLbeforeproj => 'http://webmail.revragnarok.com/';
use constant URLafterproj => '/ticket/';
use constant Nodomain => [email protected]';
use constant Subject => 'Outstanding tickets';

chdir("/var/www/trac");

my (%proj, %user);
my @projs;
# Look in directory for project names
my $dh = DirHandle->new('.') or die "Can't open trac directory! : $!\n";
while (defined ($_ = $dh->read)) {
  # Does it have a trac DB?
  if (-e "$_/db/trac.db") {
    push @projs, $_;
  } # -e
}
foreach my $projname (@projs) {
  # Open each one
  open(SQL,"/usr/bin/sqlite3 $projname/db/trac.db \"select id,status,owner,summary from ticket where status != 'closed'\"|");
  while (<SQL>) {
    chomp;
    my ($id, $status, $owner, $summary) = split /\|/;
    #print "Proj: $projname, ID: '$id', status: '$status', owner: '$owner', summary: '$summary'\n";
    next if ($owner eq '');
    $user{$owner}{"$projname-$id"} = $summary;
  }
  close(SQL);
}
foreach my $username (sort keys %user) {
  # Verify it is valid
  my $fullname = $username;
  $fullname.= Nodomain if ($fullname !~ /@/);
  # Put together the email
  my $mh = ADMMailHandler::new($fullname, 1, Subject);
  $mh->append(qq{<TABLE WIDTH=95% BORDER=1><TR>
<TD><B><U>Project</U></B></TD>
<TD><B><U>Ticket Summary</U></B></TD></TR>});
  foreach my $ticket (sort keys %{$user{$username}}) {
    my ($proj, $num) = split(/-/, $ticket);
    # print "Processing (proj = $proj) (num = $num) (summary = $user{$username}{$ticket})\n";
    $mh->append(qq{<TR><TD>$proj #$num</TD><TD><A HREF="}.URLbeforeproj.$proj.URLafterproj.$num.qq{">}.$user{$username}{$ticket}."</A></TD></TR>\n");
  }
  $mh->append("</TABLE>");
  $mh->sendit();
  # Now we'll make a special 'MyTickets' wiki page if the person has their own sandbox
  my ($tryname, undef) = split(/@/, $fullname);
  if (scalar grep(/^${tryname}$/, @projs)) {
    # There is a match
    open(WIKI, ">/tmp/$$");
    print WIKI "== My Tickets ==\n||'''__Project__'''||'''__Ticket Summary__'''||\n";
    foreach my $ticket (sort keys %{$user{$username}}) {
      my ($proj, $num) = split(/-/, $ticket);
      print WIKI "||$proj !#$num||[$proj:ticket:$num $user{$username}{$ticket}]||\n";
    }
    close WIKI;
    `/usr/bin/trac-admin $tryname wiki remove MyTickets > /dev/null 2>/dev/null`;
    `/usr/bin/trac-admin $tryname wiki import MyTickets /tmp/$$`;
    unlink "/tmp/$$";
  } # Sandbox or not
} # Each username

Xilinx BIT files and the Linux/Unix/BSD "file" command

The attached file will ID a Xilinx BIT file and tell you when it was compiled, the original NCD file name, and most importantly the chip it is for. It doesn't give a speed grade, but it gives all the other stuff.

All credit goes to the FPGA FAQ Question 26.

To install on a machine that already has file installed (yours probably does) you need to find your magic file. I will present what I did on a Cygwin box as an example, season to taste:

  1. cd /usr/share/file/
  2. rm -rf magic.mgc
  3. cat /tmp/xilinx-magic >> magic
  4. file -C

The last command "compiles" the magic file into magic.mgc. To make sure it all worked, you can grep -i xilinx magic* and see a few spots.

Example output:

[email protected] ~/projects/ss/trunk/vhdl
$ file */*bit
BenADDA/benadda.bit:                Xilinx BIT file - from BenADDA.ncd - for 2v6000ff1152 - built 2007/ 6/27(13:19:26) - data length 0x23d014ff
BenADDAV4/benadda.bit:              Xilinx BIT file - from BenADDA.ncd - for 4vsx55ff1148 - built 2008/01/07(15:37:49) - data length 0x1f3928ff
BenADDAV4_Clock/mybenaddaclock.bit: Xilinx BIT file - from MyBenADDAClock.ncd -for 2v80cs144 - built 2008/01/11(14:18:37) - data length 0x1652cff
BenDATAV4/bendatadd.bit:            Xilinx BIT file - from BenDATADD.ncd - for 4vlx160ff1148 - built 2008/01/11(17:53:27) - data length 0x4cf4b0ff
BenNUEY/bennuey.bit:                Xilinx BIT file - from BenNUEY.ncd - for 2vp50ff1152 - built 2008/01/10(17:14:41) - data length 0x2447c4ff

This file has been submitted to the maintainer of the file command so some day may come with a default build.

Makefile dependencies

So I inherited a project that apparently had Makefiles written by people who had no idea how Makefiles are supposed to work.

When I edited one of the header files and reran make, nothing happened. So I read some stuff online about making gcc auto-create dependency files. However, they seemed to be outdated pages that referred to using sed and stuff when gcc seems to automagically dump the proper format already. And this is a RHEL3 box, so not exactly cutting edge.

Anyway, I went with a hybrid approach of a few different pages I found, so figured I would share it here.

These reply on the "main" Makefiles defining OBJS as the target files (.o, e.g. SocketHandler.o).

In the main Makefile, I redefined their CLEAN_CMD to add $(OBJS:.o=.d) to the $(RM) command. Then, after the OBJS was defined, I include Makefile.dep.

Also note that this is a little convoluted because the project actually combines a bunch of .o files into .a library files. For non-library compilations, the targets would just be %o and the gcc command line would be slightly tweaked. I just wanted to show the newer-than-any-webpage-I-saw -MF command. No more sed or redirects needed.

Makefile.dep:

# This Makefile will generate .d files for every source file found.
# This allows make to handle header files properly.

# This file must be included AFTER $OBJS has been defined, not in
# the global Makefile

# Include existing .d if they are present:
-include $(OBJS:.o=.d)

# Override default cpp and c compiling instructions:
$%(%.o) : %.c 
        $(CC) -MM -MP $(CPPFLAGS) $(CFLAGS) $*.c -MT "$@($*.o)" -MF $*.d
        $(CC) -c $(CPPFLAGS) $(CFLAGS) $*.c -o $*.o
        $(AR) cr $@ $*.o

$%(%.o) : %.cpp
        $(CXX) -MM -MP $(CPPFLAGS) $(CXXFLAGS) $*.cpp -MT "$@($*.o)" -MF $*.d
        $(CXX) -c $(CPPFLAGS) $(CXXFLAGS) $*.cpp -o $*.o
        $(AR) cr $@ $*.o

Obnoxious VMWare console beeps

The VMWare machines on my work laptop would chime whenever I did things like filename completion. The problem was, it was BIOS-level super obnoxious beep that was likely pissing off my cube-neighbors (no volume control nor mute worked). I had turned off all sounds in the Windows host OS, which seemed to be the problem. So if your console beeps are out of hand, set the Windows' "Default Beep" to something. That will have Windows re-intercepting the beep and then the system volume control / mute will work again.

Mail relaying fun

For the second time, my damned ISP has blocked port 25. No real reason, I have secure Linux boxes as the only things sending email; I would get 5-6 server reports a day. That's all. But they blocked me again, and I am sick of arguing with the scripted Indian morons.

Main machine

Anyway, on the main Linux box, ragnarokjr (named so because it is a VM box), I was able to finally get qmail to send all mail thru TLS:

  1. Unpacked netqmail
  2. Applied the full netqmail TLS patch to get base64.h and base64.c
  3. Erased it all and re-unzipped
  4. Copied in the base64.h and base64.c
  5. Applied the remote auth patch only
  6. "make qmail-remote"
  7. Copied qmail-remote over /var/qmail/bin/qmail-remote
  8. Edited /var/qmail/control/smtproutes to include name and password:
    :smtp.isp.net:587 username password
  9. Made smtproutes owned by "qmailr" and chmod og-r so it's kinda secured

So now qmail-remote will use TLS on port 587 as needed to get around the stupid block...

Other machines

One of my other machines runs CentOS which uses exim instead of qmail and it took me a while to find this FAQ.

/etc/exim/exim.conf, in the "routers" section:

send_to_gateway:
  driver = manualroute
  transport = remote_smtp
  route_list = * ragnarokjr.revragnarok.com
  no_more

And of course, /etc/init.d/exim restart

Hopefully, this can help somebody else. I was searching all kinds of terms like "exim upstream" (lots of Debian stuff saying what changed from standard) and it took a bit...