NacreData L.L.C.

Showing events from multiple public Google Calendars using your own custom CSS

2010-04-21T17:51:00.003-04:00

Today I hacked some code to produce a result I know others are looking for, so I thought I would share. So far what I have is the result of of an hour or so exploring the code to see if this was possible. In other words, there is room to make this much more elegant and I hope to do so at some point.

The seed of the issue is the inane inability to cusomize the color of the frame when embeding a Google Calendar. That really, really ought to be fixed by Google.

One solution to this problem which I found (yes, by Googling it...) is "calvis". The project home page is here: http://code.google.com/p/calvis/

Calvis uses the Google Data API to pull down calendar events and display them in on screen using JavaScript.

Unlike the embeded calendar, Calvis does not support displaying multiple Google Calendars. There is some discussion of this issue on the site's issue tracker (issue #3). This is the problem for which I now have a working solution.

Note that this solution has only been tested for the cases where all calendars used are public and will probably not work in its current form for private calendars.

Here's what I did:

The majority of the changes are within calvis-core.js and you can download the diff here.

1) I don't think it matters, but for what it's worth I replaced the projects jQuery 1.3.2 with the latest 1.4.2

2) I added a new variable, calList to the variables defined in the main() javascript funtion which is defined in-line on the calendar page. This variable holds an array of the calendars to be combined. Here's what my declaration looks like:


  var calList = [ 'google@communitygreenguide.org', 
    'b0prga519c0g0t3crcnc0g9in0@group.calendar.google.com' ];

3) Also within the main() javascript function on the calendar page, I set the calList variable within the Calendar object, using a new setCalList() method defined in calvis-core.js (more on this in a minute).


  calendar.setCalList(calList);

4) In the Calendar constructor, create a calList variable set to null:


  this.calList = null;

5) Defind the setCalList() method:


  calvis.Calendar.prototype.setCalList = function(calList) {
    this.calList = calList;
  }

6) Modify the existing getFeedUrl() method so that it returns an array of calendars and not just one calendar feed:


  calvis.Calendar.prototype.getFeedUrl = function() {
    
    feedUrlArray = new Array();
    calList = this.calList;
    
    for( var i=0; i < calList.length; i++ ) {
      feedUrlArray[i] = ['http://www.google.com/calendar/feeds/', 
        calList[i], '/', this.visibility, '/full'].join('');
    }
         
    return feedUrlArray;    
  };

7) Change some code in the initLoginControl() method so that it can deal with getting an array of calendars. I don't believe this code will actually serve any purpose since we're only using public calendars, so I just use the first calendar in the array.


  var feeds = calendar.getFeedUrl();
  var scope = feeds[0];

8) In the two places overlayGData() is called, modify the calling code so that it calls overlayGData() once for each calendar, passing in the feedUrl as an attribute. This code appears near the bottom of the updateWeekView() method and at the bottom of the updateMonthView() method:


  var feedUriArray = this.getFeedUrl();
  calendar = this;
  for( var i=0; i< feedUriArray.length; i++ ) {
    calendar.overlayGData(firstDate, lastDate, feedUriArray[i]);
  }

9) Change the method signature of overlayGData so that it will acced the feedUri as a parameter:


  calvis.Calendar.prototype.overlayGData = function(startDate, endDate, feedUri)

10) Remove the line in the existing overlayGData() method that retrieved the feedUri from the object:

var feedUri = calendar.getFeedUrl();

Uploading Files to a webserver from CKEditor under mod_perl

2009-11-24T14:32:00.003-05:00

Just got a request to elaborate on this topic which I had mentioned in a recent tweet. So here's some more details. Note that there is no file-type checking here. In my case the uploads are only being allowed from logged-in administrators, so I can assume they are not uploading malicious files.

1) Of course, remember to use enctype="multipart/form-data" in the HTML form. Obvious but I've forgotten it more than once over the years...

2) To tell CKEditor what URL to send the uploaded file to, I pass the target URL to the CKEditor command which replaces the form <textarea> with a rich text editor:

CKEDITOR.replace( 'form_textarea', { filebrowserUploadUrl : '/upload_file' } );

In my case I did this inside a jQuery "ready" handler:


$(document).ready( 
  function() {
    CKEDITOR.replace( 'form_textarea', { filebrowserUploadUrl : '/upload_file' } );
  }
);

3) Map the URL to a mod_perl handler

I'm using object oriented perl handlers, so the relevant portion of my httpd.conf file looks something like:


<Location ~ "^/upload_file$">
  SetHandler perl-script
  PerlHandler NacreData::Controllers::FormHandler->upload_file
</Location>

This will look different if you're using a procedural handler or mod_request or some such.

4) Require/Use a perl module to do the uploading

The one I got to work is Apache::Upload::Slurp

I installed this with CPAN and used a PerlModule directive in httpd.conf to make sure it was loaded at runtime.

5) Write the mod_perl handler

Here's a cleaned up version of what mine looks like:


  sub upload_file( $$ ) {
    my ( $self, $r ) = @_;
    
    use Apache::Constants qw( :common );
    my $apr = Apache::Request->instance( $r );
    $r->content_type('text/html');
    $r->send_http_header;

    require Apache::Upload::Slurp;
    my $Uploader = new Apache::Upload::Slurp;
    my $uploads  = $Uploader->uploads();
    my $upload = $uploads->[0];
    my $dir    = '/upload_files'; # directory where the uploaded files will go
    return unless $upload;
    return unless $upload->{data};

    # make this into a nice filename, no special chars, no spaces, etc.
    my $filename = $upload->{filename};
    $filename    =~ m{([^\\/]+$)};
    $filename    = $1;
    $filename    =~ s/[^.\w]//g;
    $filename    =~ s/__+/_/g;
    $filename    =~ s/^_//;
    $filename    =~ s/_$//;
    $filename  ||= 'file';

    # make sure we don't overwrite another file by the same name
    while( -f $dir.'/'.$filename || -d $dir.'/'.$filename ) {
      my $ext = '';
      if( $filename =~ m/(\.[^.]+$)/ ) {
        $ext = $1;
        $filename =~ s/\.[^.]+$//;
      }
      my $d = 0;
      if( $filename =~ m/(\d+)$/ ) {
        $d = $1;
        $filename =~ s/\d+$//;
      }
      $d++;
      $filename = $filename . $d . $ext;
    }

    my $fh;
    $filename =  $dir.'/'.$filename;
    open ( $fh, '>', $filename ) || die "can't open $filename";
    print {$fh} $upload->{data};
    close $fh;

    # now we return the information about the uploaded file to CKEditor.

    # reference passed in from CKEditor. We use this to call back, letting
    # the editor know we've don the upload.
    my $funcNum = $apr->param('CKEditorFuncNum'); 

    $filename =~ s|^/htdocs||; # path under document root to uploaded file
    # you probably need ot change this line -- in my case, since I'm 
    # running under chroot, the webroot is just "/" and docroot /htdocs

    my $hostname = $r->server->server_hostname;
    $hostname =~ m{(\w+\.\w{2,4})$};
    my $url = 'http://' . $hostname . $filename;

    print "<script>parent.CKEDITOR.tools.callFunction( $funcNum, '$url' );</script>";
  }

BBEdit 9.0 -- First Impressions

2008-09-02T00:09:00.003-04:00

After a little more than one working day using the newly released BBEdit 9.0, my impressions is positive.

First of all, I am glad to see that a trusted tool works largely as expected as it keeps pace. This is to me much more important than flashy new interfaces or features. The search interface is cleaner, which is nice, but over all I had no problem at all getting right back to work after the upgrade.

My favorite new feature is the variable auto-completion I noticed this evening while working on an approximately 400-line perl script with many, many variables. The auto-completion gave me great assurance without any split screens or searching that I had the spelling and capitalization of previously used variables correct.

Thanks, BareBones for the continued good work.

Some links about current DNS attack

2008-08-02T00:48:00.000-04:00

Is your ISP using a correctly patched DNS server?

Or are you open to attack? Could the websites you see not really be what they appear to be?

Listen to a podcast with a very complete explanation of DNS and the attack.

Product Review: Tone 12 GB USB 2.0 Mini Hard Drive

2008-08-01T22:51:00.002-04:00

Very cute, casing matches my MacBook Pro. Works reliably so far. Took formatting to Mac (Extended, case-insensitive, journaled) no problem. Slow as molasses, though. The speed really cuts into it's utility for my intended use in loading virtual machines.

Launch VMWare and open a suspended WinXP virtual machine:

* Over the network from G5 dual 1.8GHz PowerPC to MacBook Pro: 58 seconds
(54 seconds to suspend, saving state back to disk)

* Same virtual machine from Tone 12 GB attached to MacBook Pro directly: 1:02 minute. Saving state: 1:41 - 3:09 minutes.

* Same virtual machine loading directly from MacBook Pro HD: 17 seconds; suspend: 4.4 seconds

Green Business Leadership

2008-07-05T18:43:00.002-04:00

We'd heard that some rental car companies are renting hybrids. Calling around today, my wife was frustrated to learn that the options are few and far between in the Triangle area. Seems that just like the car manufactures, most rental companies are followers, looking behind, instead of business leaders looking forward.

Which gets me thinking—how can NacreData LLC do a better job of being a green business leader? Feel free to leave your suggestions in the comments.

Our transportation costs are largely non-existent — I work in a home office and everyone else currently doing NacreData work is also a home-based.

We're not a bulk hosting company, but I've read a few times (see http://blogs.business2.com/greenwombat/2007/02/photo_originall.html for instance) that server farms are power hogs. We're beginning to move more of the sites we manage onto virtual-machine based servers, where several server environments can share the resources of one physical machine.

Getting SpamAssassin's spamc to work with Qmail

2007-12-28T12:58:00.000-05:00

One of the recommended ways of setting up SpamAssassin to work with Qmail is to move the binary "qmail-queue" to a different name, say qmail-queue.orig, then create a shell script named qmail-queue which passes the email through SpamAssassin then on to the original qmail-queue.

One problem with that set up which I've been noticing is that it passes every email through the spam filter, even though the vast majority of email coming into the server is addressed to non-existent users. The time taken to scan all those emails was noticeably slowing down the system.

So what I really wanted to do was call the spam filter from the .qmail files for real users.

Qmail's default delivery mechanism, set to contain just "#" in .qmail-default, will then drop all emails sent to non-existent users without calling SpamAssassin.

I wanted to use the spamc utility, written in C, as a filter to call the already running spamd deamon from the user's spamc with the "-c" switch so that non-spam messages will return 0 (zero) and thus pass on to delivery. One small problem, though, is that the exit value for an email determined to be spam is 1. Qmail will interpret an exit value of 1 as a temporary failure and keep trying to deliver the message. This will create a backlog of undelivered messages and slow down the email server.

To get around this, I changed the exit value for spam emails in spamc. First I downloaded the latest SpamAssassin and untarred the download file. In the "spamc" subfolder I found a file named "libspamc.h". On or around line 85 is the line:

#define EX_ISSPAM 1

I simply changed this to

#define EX_ISSPAM 99

and then completed the installation steps:

perl Makefile.PL

make

sudo make install

which installed SpamAssasissin including spamc with the altered exit value.

Lastly, from my user .qmail files, I call spamc. For example for .qmail-devin:

|/usr/local/bin/spamc -c

./devin/Maildir/

and now SpamAssassin is run only for email coming to real users and all messages determined to be spam are silently dropped. The return value of 99 tells Qmail that the message has been completely processed and no further delivery steps need be taken.

MySQL: To Good to Be True

2007-08-09T18:24:00.000-04:00

In the back of my mind for some time now I've planned to move all NacreData projects away from MySQL to a more truly open source solution (almost surely PostgreSQL), and today's news pretty much cinches it.

Painting myself into a corner and working back out...

2007-07-29T23:20:00.000-04:00

Upon leaving my friend's place in Indiana recently (see previous post) I brought back home with me an old iMac, a "blueberry", one of the blue-and-white cute cpu-and-monitor triangle shaped deals. It has a G3 processor and had a broken installation of OS 9. I recovered the important data and transferred it to a couple other media at my friend's place, then started tinkering.

Having noticed the PPC disk in the OpenBSD distributions I decided to give it a go. Being the last to RTFM, as they say, I blithely blew away the machine's partitions and installed the usual set I'm used to with OpenBSD. The install went fine, but the machine could not be booted off the new OS at all.

So I read. Don't delete the Mac's boot partition it says. The Mac ROM needs to see a HFS boot partition or it won't go nowhere. Drat. The first many resources I found all describe how to set up the installation presuming you knew this and presuming you were intending to make a dual-boot machine with Mac on the other side. Finally I did find a couple links which provided the clues needed about how to rectify the situation. Use pdisk to create an HFS ("Apple_HFS") partition of at least 1MB size named "boot", along with the other partitions usually used (/, swap, /tmp, /var/, /usr, /home). Then re-do the install. Then mount the OpenBSD install CD and the HD's boot partition and copy the open firmware program "ofwboot" to the "boot" partition. Finally, remove all CD's and boot with Command-option-o-f and issue the command

boot hd:,ofwboot /bsd

Voilà, it works and I'm off and running :-)

PC Eudora to Mac Mail Conversion

2007-07-24T19:26:00.000-04:00

I had a wonderful few days visiting friends in southern Indiana and setting up a network at their rural property where there are outbuildings used for conferences. I went with the latest D-Link b/g/n router over the Apple Airport for cost and the flexibility of a web-based configuration interface for the wireless component of the system.

We also moved over 103,000 emails from Eudora 7 on a Win XP machine to Mac Mail on a new MacBook Pro. This was a bit of a pain. Our original plan was to move to Mac Eudora 6.2 for consistency, but the Eudora help for this process was entirely inadequate, and in places just flat wrong. Eudora 6.2 for OS X still uses OS 9 line endings (CR) for some reason, rather than the LF used by all UNIX based systems. Overall Eudora 6.2 felt like an outdated OS 9 piece of software I thought. The Mail importer, however, worked reasonably well (though slow) on the Eudora message files. Multiple levels of nested mailboxes were retained.

The third-party Eudora Mailbox Cleaner did a good job with the multiple address books, but the Filter->Rules import only imported the name of the filters, not any of the content, which was disappointing.

Updated site, new email gateway

2007-06-11T00:43:00.000-04:00

Eva's design work is up now at http://guide.BuyLocalCA.org/ and it looks rather nice I think. This is the next version of the site, which I've been working with for a couple years, which formerly looked pretty much like http://www.BuyLocalPA.org/ (designed by Design for Social Impact). NacreData's role in the new CA site, aside from hosting and getting the existing database to display information in the new design, was the integration of Google Maps into the search display. What do you think? I went with the "bounding coordinates" method for the zip code search -- calculate the lat/long coordinates corresponding to the bounding points of map X miles from the center of the chosen zip code and search the database for lat/long values falling within those bounds. This was quicker than comparing the distance of every listing in the database.

I also set up a new email gateway, OpenBSD 4.1 / Postfix to handle the primary flow from the National Campaign for Sustainable Agriculture. The qmail server on the web server machine was getting inundated and the old machine serving as secondary was failing, so time to upgrade. Seems to be working fine, though it seems such a shame for a nice fast machine to spend 99% of its time refusing emails sent to non-existent email address...

Updated Sites

2007-05-07T09:53:00.000-04:00

I should mention a couple of the sites I've been working which have recently been posted. First, the Heartwood website has had a long-overdue complete overhaul. See see the article on the site for a full list of attributions and thanks to all who contributed!

Also at the first of May we launched the wholesalers portion of the Philadelphia area online local food guide. The local food guide for Charlottesville, Virginia is also freshly live, though I have some adjustments to make to the categories on the site. Should be done today.

Lastly, I love my new office at our new home. I feel more productive already :-)

On second thought...

2007-03-10T00:05:00.000-05:00

You know all those things you're not supposed to have to worry about with high-level language? You know, like memory management and declaring variables?

Yeah, right.

Then you read books on security and run into crashing database server hung on aborted connections and, well...

Worry about those little things.

for the want of a "6"

2007-02-03T22:19:00.000-05:00

More interesting chroot challenges today.

First the challenge of perl modules which want to use Dynaloader to load shared C libraries at run time. Not gonna happen if the path isn't the same under the chroot, which it surely isn't. I got around this one with mod_perl and a startup perl script, so that the perl module is run as root, loading the libraries, before the chroot happens.

Then LWP returned an obscure error about tcp being a bad protocol. Hm. Google groups search to the rescue, and we find there is a table I never noticed before relating named protocol names to numeric values which also couldn't be found. Simply copied this table into the chroot with read-only permissions.

SSL on alternative ports

2007-01-23T12:10:00.000-05:00

There are penalties to trying to be too clever.

Having already set up an SSL VirtualHost for one client on my main PHP webserver, I decided to set up a second under a port other than the standard :443. This worked, and allowed for more than one SSL certificate to be used on that server without using additional IP's.

Unfortunately I soon began getting complaints from a few folks that the pages in question wouldn't load for them. Turns out several of them were coming from the same public library and another from a University account. So it appears that there are a significant number of people accessing sites from behind restrictive firewalls where SSL on an alternative port won't work.

Now I'll have to use additional IP's, something name-based VirtualHosting and HTTP 1.1 was designed to get around. Well, I tried. :-)

Net::UPS Patch

2007-01-12T21:02:00.000-05:00

Perl is such a great environment because when you need to do something like develop an interface to the UPS XML services, and you start mucking around with LWP and wonder why their service doesn't expect POST to have key=>value pairs, you suddenly realize someone has written Net::UPS and the problem has already been solved.

The 0.4 version of Net::UPS, however, doesn't deal with the new "your prices may vary" warning UPS attaches to every query as a warning. The following simple patch, similar but somewhat simpler and more general, than one already submitted to the module's author, works to remedy the situation:

--- /usr/local/lib/perl5/site_perl/5.8.8/Net/UPS.orig Fri Jan 12 20:56:28 2007
+++ /usr/local/lib/perl5/site_perl/5.8.8/Net/UPS.pm Fri Jan 12 20:54:10 2007
@@ -307,7 +307,10 @@
KeyAttr => [],
ForceArray => ['RatedPackage', 'RatedShipment']);
if ( my $error = $response->{Response}->{Error} ) {
- return $self->set_error( $error->{ErrorDescription} );
+ $self->set_error( $error->{ErrorDescription} );
+ if ( $error->{ErrorSeverity} eq 'Hard' ) {
+ return $self->set_error( $error->{ErrorDescription} );
+ }
}
my @services;
for (my $i=0; $i < @{$response->{RatedShipment}}; $i++ ) {

One last OpenBSD/qmail thought for the night...

2007-01-09T00:44:00.000-05:00

As mentioned here http://forum.qmailrocks.org/showthread.php?t=3205 and probably other places I didn't find when searching, there is an nosuid entry which prevents setuid programs from operating under the /var directory. This could be disabled; I got around it by moving the entire /var/qmail directory

mv /var/qmail /usr/local/qmail
ln -s /usr/local/qmail /var/qmail

Also useful...

2007-01-08T22:22:00.000-05:00

cr.yp.to seems to be unreachable tonight. A couple mirror links for application downloads:

http://blogs.sun.com/BableOn/entry/virtual_email_server_qmail_vpopmail

http://public.www.planetmirror.com/pub/djbdns/daemontools/?fl=

had to stick this somewhere to remember it...

2007-01-08T22:17:00.000-05:00

Shell script to set up qmail users in OpenBSD 4.0

#!/bin/sh

groupadd nofiles
useradd -G nofiles -d /var/qmail/alias -s /nonexistent alias
useradd -G nofiles -d /var/qmail -s /nonexistent qmaild
useradd -G nofiles -d /var/qmail -s /nonexistent qmaill
useradd -G nofiles -d /var/qmail -s /nonexistent qmailp
groupadd qmail
useradd -G qmail -d /var/qmail -s /nonexistent qmailq
useradd -G qmail -d /var/qmail -s /nonexistent qmailr
useradd -G qmail -d /var/qmail -s /nonexistent qmails

DBD::mysql done

2006-12-23T22:52:00.000-05:00

Installing the perl module DBD::mysql is notoriously plagued by glitches. Getting it going on my MacBook Pro proved no different. I found some references to others having issues, but none the same as mine. Perhaps this is because I have installed perl 5.8.8 (placing it in /usr/local/bin, archiving the existing install (5.8.6), and symbolically linking /usr/bin/perl to the new install). Or maybe it's the phase of the moon.

The error I got at the "make" stage listed two /tmp/.out files and told me they were both i386 architecture files and couldn't be bundled together. The Makefile.PL uses the output form mysql_config to set the -cflags directive by default, so I took a look at this. It was:

-I/usr/local/mysql/include -Os -arch i386 -fno-common

removing the "-arch" flag worked. All I had to do was:

mysql -u root -p
password: *****
> grant all on test.* to ''@localhost
OK
> quit

make clean
perl Makefile.PL -cflags="-I/usr/local/mysql/include -Os -fno-common"
make
make test
sudo make install

Wow

2006-12-17T23:07:00.001-05:00

What a wonderful birthday weekend it's been. I feel very loved. Thanks everyone.

:-)

Ginkgo Biloba

2006-12-08T09:42:00.000-05:00

Well, cool. The nice gentleman at the apple store took the memory and the MacBook Pro in the back and came back out with it working. Not sure what he did that I didn't, but that's ok.

More memory is nice since I've been running more applications at once since I downloaded Desktop Manager

Bad Memory

2006-12-07T17:16:00.000-05:00

New memory module for MacBook Pro came by FedEx. Install seemed easy enough, but the machine won't even turn on with it in. Damn. Appt. at Apple store this evening, we'll see what they have to say about it.