Posts in category trac

Migrated Again

Wow. I've had this blog since 2002. Waaay back in the day, it was some proprietary format, and I migrated it 13 years ago to trac.

At that time, it was on a dedicated Red Hat box that also acted as my firewall.

At some point since then, I migrated it to vmware - see that topic for some of the problems.

Originally that VM image ran on my CentOS server (cf. ​https://bugs.centos.org/view.php?id=3884 ) and some point it was migrated to my Windows 7 desktop.

Since it was in a VM, I could always snapshot and really beat on it. I had files as far back as 2001 and GPG signatures for my RPMs from Red Hat OS before it was Fedora/RHEL split. Over the years, I've managed to beat it into submission to the point I had it running Fedora 31; of course it's built-in now with dnf system-upgrade. But that's not the point. Fedora 32 broke Python2, and trac isn't there yet. (Side note - the VM has been called webmail for years, but I uninstalled SquirrelMail and moved to Google-hosting many years ago.)

With the COVID-19 quarantine, I decided to migrate this blog to containers so I can just use a pre-defined trac container and go on my merry way. Hopefully less maintenance in the future.

So, that's where it is now. As I tested the site from my wife's iPad (on cellular) I just had to marvel at how data travels to get this post out of my house: 

(you) <=> (Cloudflare) <=> OpenWrt <=> Win10 Pro <=> Hyper-V Docker VM <=> Container [Ephemeral] <=> Docker Volume

Reminder email and wiki page for trac

I wrote this a long time ago, never had a chance to update it, and somebody asked on the trac mailing list today…

You can get ADMMailHandler in the ​ADMLogger package (sorry, too lazy today).

You need to configure a few places too. Like I implied, not quite ready for public consumption, but at my rate never will be. Please feel free to "upstream" me any changes.

Also, if there is a trac project that matches the user name (that's how my company does trac sandboxes) then it will also create a page MyTickets with a dump of links. 

#!/usr/bin/perl -w
use strict;

# Script to send email reminders in a cron job.
# Email trac at revragnarok <.> com

# I should've made it DBI compliant - but I'm not getting paid for this ;)

BEGIN {
   push @INC, "/usr/src/my_scripts/mail_reminder"; # Where ADMMailHandler is hiding
}

use Data::Dumper;
use ADMMailHandler;
use DirHandle;

use constant URLbeforeproj => 'http://webmail.revragnarok.com/';
use constant URLafterproj => '/ticket/';
use constant Nodomain => '@yourdomain.com';
use constant Subject => 'Outstanding tickets';

chdir("/var/www/trac");

my (%proj, %user);
my @projs;
# Look in directory for project names
my $dh = DirHandle->new('.') or die "Can't open trac directory! : $!\n";
while (defined ($_ = $dh->read)) {
  # Does it have a trac DB?
  if (-e "$_/db/trac.db") {
    push @projs, $_;
  } # -e
}
foreach my $projname (@projs) {
  # Open each one
  open(SQL,"/usr/bin/sqlite3 $projname/db/trac.db \"select id,status,owner,summary from ticket where status != 'closed'\"|");
  while (<SQL>) {
    chomp;
    my ($id, $status, $owner, $summary) = split /\|/;
    #print "Proj: $projname, ID: '$id', status: '$status', owner: '$owner', summary: '$summary'\n";
    next if ($owner eq '');
    $user{$owner}{"$projname-$id"} = $summary;
  }
  close(SQL);
}
foreach my $username (sort keys %user) {
  # Verify it is valid
  my $fullname = $username;
  $fullname.= Nodomain if ($fullname !~ /@/);
  # Put together the email
  my $mh = ADMMailHandler::new($fullname, 1, Subject);
  $mh->append(qq{<TABLE WIDTH=95% BORDER=1><TR>
<TD><B><U>Project</U></B></TD>
<TD><B><U>Ticket Summary</U></B></TD></TR>});
  foreach my $ticket (sort keys %{$user{$username}}) {
    my ($proj, $num) = split(/-/, $ticket);
    # print "Processing (proj = $proj) (num = $num) (summary = $user{$username}{$ticket})\n";
    $mh->append(qq{<TR><TD>$proj #$num</TD><TD><A HREF="}.URLbeforeproj.$proj.URLafterproj.$num.qq{">}.$user{$username}{$ticket}."</A></TD></TR>\n");
  }
  $mh->append("</TABLE>");
  $mh->sendit();
  # Now we'll make a special 'MyTickets' wiki page if the person has their own sandbox
  my ($tryname, undef) = split(/@/, $fullname);
  if (scalar grep(/^${tryname}$/, @projs)) {
    # There is a match
    open(WIKI, ">/tmp/$$");
    print WIKI "== My Tickets ==\n||'''__Project__'''||'''__Ticket Summary__'''||\n";
    foreach my $ticket (sort keys %{$user{$username}}) {
      my ($proj, $num) = split(/-/, $ticket);
      print WIKI "||$proj !#$num||[$proj:ticket:$num $user{$username}{$ticket}]||\n";
    }
    close WIKI;
    `/usr/bin/trac-admin $tryname wiki remove MyTickets > /dev/null 2>/dev/null`;
    `/usr/bin/trac-admin $tryname wiki import MyTickets /tmp/$$`;
    unlink "/tmp/$$";
  } # Sandbox or not
} # Each username

Using SVK for Roaming SVN Users

I have a Trac/SVN server on my work laptop (in a VMWare box). Others are needing access to the files more, so I needed a way to have two way merging. Of course, others have had this problem already, and svk was the result. However, there are certain aspects of svk that I'm not too fond of. Mainly, I didn't want to lose my TortoiseSVN capabilities or all my subversion know-how. However, I'm going to exploit the fact that an svk "depot" is under the hood a svn repository.

Here's what I did:

  1. I needed to get svk running under Cygwin. That was a real PITA, but luckily, somebody was nice enough to put all the instructions on ​this wiki page.
  2. Now I need to get a local copy of the entire svn repository under svk:
    • svk mkdir svnrepo
    • svk mirror http://svnserver/svn/path/to/repo //svnrepo/reponame
    • svk sync -a (This took FOREVER)
    • svk mkdir local
    • svk cp //svnrepo/reponame //local/reponame

OK, so now, we have a local svk "depot" which has in it /svnrepo/ and /local/ but it is all kept in a single svn repository on your hard drive. Now, the magic: we check out from that file using TortoiseSVN to create a subversion working copy. Using TortoiseSVN, I checked out "from" file:///E:/cygwin/home/user/.svk/local/local/reponame - you'll note that the first part is my cygwin home directory (username of 'user'), and the double local is not a typo - the first is a "real" directory on my E: drive, the second is at the root level of the repository (that we made above).

Now, when I'm offline, I can just use my local working copy, and am able to check in as much as I want without any worries. Another use for this I read about was if you want to check in a lot more than your coworkers do and want to keep the "master" repository "clean."

To perform the actual sync with the master server:

  • svk pull //local/reponame (This makes sure the local svk depot is in sync)
  • svk push --verbatim //local/reponame
    • The verbatim flag prevents svk from inserting its own header which was causing problems with trac by pointing to revision numbers in the future which just made no sense.

Drawbacks

  • One of the files I tried to push was locked on the master repository, but that information doesn't seem to be propagated properly, so the push failed until I unlocked the file manually on the master server.
  • Need to do the push and pull manually.
  • svn's keyword substitution now replaces info with local information, like revision number of the file in the local svk depot, not the master repository (which means printouts aren't going to match). - It seems that all svn properties may be iffy.

Resources

Very odd subversion issue

So, I had a file that refused to be checked in. Seems the file had previously been checked in, but then deleted with the TortoiseSVN checkbox "Keep Locks" checked. So it wouldn't go away. Anyway, the solution was svnadmin rmlocks /data/svn_repos/vhdl_bn /bn.bit

​http://paste.lisp.org/display/49343

​http://colabti.de/irclogger/irclogger_log/svn?date=2007-10-17,Wed;raw=on

(not really "trac" but I don't have a "svn" tag…)

Using Trac for Teaching

This is something I wrote up a while back (Trac 0.9). Now it's even easier since 0.10 doesn't need the svn backend.

Using Trac in a Teaching Environment

Obviously, some paths and usernames/groups etc may need to be changed…

  1. Install ​trac. I used 0.9.6.
  2. Create the repository (fake)
    1. svnadmin create /var/www/svn/teacher
    2. TMPDIR=`mktemp -d`
    3. svn import "${TMPDIR}" file:////var/www/svn/teacher -m "initial setup"
    4. rm -rf "${TMPDIR}"
    5. chown -R apache.apache /var/www/svn/teacher
  3. Create the trac project
    1. trac-admin /var/www/svn/teacher initenv "Mrs. M Web World" ​sqlite:db/trac.db /var/www/svn/teacher /usr/share/trac/templates
    2. chown -R apache.apache /var/www/trac/teacher
    3. trac-admin /var/www/svn/teacher permission add "Admin" "TRAC_ADMIN"
  4. Set up Apache as needed…
    1. htpasswd -cm /var/www/trac/teacher/.htpasswd Admin
    2. htpasswd -m /var/www/trac/teacher/.htpasswd Teacher
    3. htpasswd -m /var/www/trac/teacher/.htpasswd TestStudent [pwd: test]
    4. Create/modify trac.conf in Apache's conf.d directory (see below)
  5. (Test login at this point)
  6. Install some needed plugins
    1. ​WebAdmin
    2. ​TicketDeletePlugin
  7. Now we need to edit trac itself, to make it look nicer for student interface:
  1. Edit /usr/lib/python2.3/site-packages/trac/ticket/web_ui.py
    1. Search for "New Ticket" within a util.Markup() call and change it to "New Question"
  2. Edit /usr/lib/python2.3/site-packages/trac/ticket/roadmap.py
    1. Search for "Roadmap" within a Markup() call (with accesskey 3) and change it to "Syllabus"
  1. Edit trac.ini as needed:
    1. I changed the [header_logo] with an image of the county school system and associated link.
    2. Make sure the various plugins are enabled/disabled in [components] section of trac.ini (see below)
  2. Login again (as 'Admin'). You should have an 'Admin option'.
  3. Permissions. I made two groups "@teachers" and "@students".
    1. @students get TICKET_APPEND, TICKET_CREATE, TICKET_VIEW, ROADMAP_VIEW, MILESTONE_VIEW, WIKI_VIEW
    2. @teachers get TICKET_ADMIN, WIKI_ADMIN, MILESTONE_ADMIN, ROADMAP_VIEW
  4. At this point, we're done with the main server stuff (until the term starts and you need to add more users as above). Now the teacher can customize with that login (recommended).
  5. My recommendations:
    1. Milestones:
      1. These should be based on the syllabus (that's what the students see it as). I assume each major section of the class would get one. The "due date" could be the Monday of the Quiz week or something similar.
    2. Ticket types: General Question and Clarification Request
    3. Components: None (removed them all)
    4. Priorities: None (removed them all)
    5. Severities: None (removed them all)
    6. Versions: None (removed them all)

httpd.conf

Alias /trac/ "/usr/share/trac/htdocs/"

<Location /teacher>
   SetHandler mod_python
   PythonHandler trac.web.modpython_frontend
   PythonOption TracEnv /var/www/trac/teacher
   AuthType Basic
   AuthName "Mrs. M's Web World"
   AuthUserFile /var/www/trac/teacher/.htpasswd
   Require valid-user
</Location>

trac.ini excerpts

[components]
webadmin.* = enabled
trac.wiki.macros.macrolistmacro = disabled
trac.versioncontrol.web_ui.log.logmodule = disabled
trac.about = disabled
trac.search.searchmodule = disabled
ticketdelete.* = enabled
trac.versioncontrol.web_ui.changeset.changesetmodule = disabled
trac.ticket.web_ui.updatedetailsfortimeline = disabled
trac.mimeview.patch.patchrenderer = disabled
trac.mimeview.rst.restructuredtextrenderer = disabled
trac.wiki.macros.pageoutlinemacro = disabled
trac.attachment.attachmentmodule = disabled
trac.wiki.macros.imagemacro = disabled
trac.versioncontrol.web_ui.browser.browsermodule = disabled
trac.mimeview.php.phprenderer = disabled
trac.timeline.timelinemodule = disabled

Automated diffs in trac

Somebody on the mailing list for trac wanted to make a daily wiki page with the diffs for a build. He already had the scripts to do a nightly build, so I suggested this, and tried it and it worked: 

echo {{{ > /tmp/tempo
echo '#!diff' >> /tmp/tempo
svn diff -r70:76 >> /tmp/tempo
echo }}} > /tmp/tempo
trac-admin /var/www/trac/personal wiki import TodaysDiff /tmp/tempo

The new blog is ready!

So I have now competed moving all my stuff from the bblog engine to trac.

To inaugurate the new blog, my notes on how I did it:

  1. By hand, slowly copied and tagged every entry. Made sure the page name was the proper date (ignored timestamp) from the old system.
  2. Wrote a script to fool trac into thinking the pages were actually written on that date (see below).
  3. In the directory with trac.db database:
    1. apachectl stop
    2. sqlite3 trac.db '.dump wiki' > wiki_in.txt
    3. ./trac_blog_datefix.pl < wiki_in.txt > wiki_out.txt
    4. sqlite3 trac.db 'drop table wiki'
    5. sqlite3 trac.db < wiki_out.txt
    6. apachectl start

The script: 

#!/usr/bin/perl -w
use Date::Parse;
while (<>) {
        if ($_ !~ /insert into/i) {
                print;
                next;
        }
        if (m|(\d{4}/\d{1,2}/\d{1,2})/|ios) {
                print STDERR "Found date of $1: ";
                my $time1 = str2time($1);
                print STDERR $time1, "\n";
                # Now replace the current timestamp with a newone.
                my $str = $_;
                $str =~ s/\d{10}\.?\d{0,2}/$time1/;
                print $str;
        } else {
                # Standard wiki page (non-blog)
                print;
        }
}

Some interesting httpd rewriting with perl 

<VirtualHost *>
  ServerName svn.whatever
  ServerAlias svn
  <Perl>
#!/usr/bin/perl
my $svn_path = "/var/svn";
my $svn_location = "";
my $trac_path = "/var/trac";
opendir(SVN_ROOT, $svn_path) or die "Cannot open $svn_path";

while (my $name = readdir(SVN_ROOT)) {
  if ($name =~ /^[[:alnum:]]+$/) {
    $Location{"$svn_location/$name"} = {
      DAV => "svn",
      SVNPath => "$svn_path/$name",
      AuthType => "Basic",
      AuthName => "\"Subversion login\"",
      AuthUserFile => "$trac_path/access.user",
      AuthGroupFile => "$trac_path/access.group",
      Require => "group $name",
    };
  }
}

closedir(SVN_ROOT);

__END__

  </Perl>
</VirtualHost>