March 2009 Archives

Last week, I got an email message from a nice guy at my domain registrar, Network Solutions.  Turns out, NetSol enjoyed the blog post on my Network Solutions WHOIS plugin/search provider for Firefox and IE.  In fact, they liked it so much that they recently added my browser plugin to the Network Solutions Labs homepage!  Here's the screen cap; check out the "WHOIS Browser Search Plugin" ...



Add to your Firefox or IE7:  Click here to add my Network Solutions WHOIS search provider to your Firefox or IE7.

If you'd like to learn how I built this plugin, take a look at the HOWTO here.  Full screen shot can be found here.  NetSol Labs can be found at http://www.networksolutions.com/labs/index.jsp.

Network Solutions is my one and only domain registrar.  I would highly recommend them for all of your domain needs.

For the curious webmaster, I published my referrer log data for February through March of '09.  Of course, I crawled through the log to remove all of the referrer SPAM, blogs (the ones I don't like) and other worthless sites.  Even so, I saw a significant (~200%) increase in web-traffic at kolich.com over the last several months.

Here's the plain-text log.  Here's the HTML (clickable) log.


What is a referrer log?

As described by Wikipedia, "when visiting a webpage, the referrer or referring page is the URL of the previous webpage from which a link was followed."  In other words, a referrer to kolich.com is the page you came from to get to my blog.  The referrer might be another blog, a Google search, Twitter, or no page at all (e.g., if you simply entered http://kolich.com in the URL address bar of your browser).

I recently purchased a new APC BR1500LCD UPS (battery backup) for my home data center.  I ran some tests as I described in this post, to determine what type of battery backup unit I needed.  The APC BR1500LCD is perfect; exactly what I was looking for.

It's a very nice unit, I'm definitely enjoying the added power protection.  The UPS weighs about 30 lbs (13.6 kg).  However, it's slightly bigger than I thought: 5-inches wide, 8.5-inches tall, and about 13-inches long.  I had to rearrange a few things in my data center to accommodate it.  Not bad though for a unit that gives me roughly 25-30 minutes of battery up time with a 220 W load.

I'm also thoroughly enjoying the front LCD panel.  The LCD panel shows you the current load (in watts and as a %), input voltage, output voltage, input hertz, and the estimated battery run time in minutes.  The LCD panel is bright red; luckily it automatically dims itself after about 60 seconds.

For a moment, ignore the fact that self-hosting involves more than simply keeping a box turned on and connected to the net.  Other than data-storage costs (replacing a failed disk), cooling costs, ISP costs, security costs (routers, firewalls, etc.), and power backup costs (a UPS), imagine if the only thing you had to pay for when self-hosting at home is the electricity.  I've always been curious what percentage of my total monthly utility bill goes directly to keeping this system online 24/7.  So, like any curious software engineer, I decided to do a little research (and math) to find out.  Here are some questions I want to answer:

• What percentage of my total monthly utility bill goes to keeping kolich.com online 24/7/365?  My monthly utility bill includes gas, and electric.
  
  
• In terms of electricity costs only, how much does it cost me per month to keep kolich.com running 24/7?
  
  
• If I only turned on the server when I was actually in my home, how much would I save per month?  This doesn't make sense for a web-server, but most folks have some type of desktop computer they only turn on when they are actually using it.
  
  
• If I "upgraded" to an equally powerful yet more energy efficient box, how much would I save per month?
  

Continue reading to find out ...

This past weekend, heavy wind knocked out the power to my home for over an hour or so.  Unfortunately, the system that powers my blog failed to stay on during the outage.  Come to find out, the UPS (uninterruptible power supply, a.k.a., a battery backup) unit it was plugged into is pretty much dead; I suspect the battery is completely trashed.  Once the power returned, I rebooted my server and decided to test the UPS.  I booted up the box, and pulled the plug out from the wall.  The UPS kept my system online for all of about 4-seconds.  So much for being uninterruptible!

Given the extremely poor up time of my existing UPS, I decided that it's time for me to invest in a new one.  However, the last time I bought a UPS, I winged it.  I didn't have any clue how much power the system actually consumed and therefore I had no idea what kind of UPS I actually needed.  I basically guesstimated using the worst case scenario, which was fine, but this time around I wanted to do it right (or at least be a little more accurate).

When dealing with AJAX, you might need to configure Apache to return a HTTP 204 (No Content).  This is useful when your AJAX scripts need to "ping" the server, but you don't want the server to actually return any data in the response (e.g., just acknowledge the request).  The server might do something behind the scenes though (like log the request) before it returns a 204.  As I understand it, the only difference between a 200 and a 204, is that a 204 response means that "the server has fulfilled the request but does not need to return an entity-body".

I tried to figure out how to configure Apache to return a 204 No Content using one of the built in modules, like mod_actions, mod_alias, or mod_headers.  Surprisingly, there does not appear to be a module that can intercept a request and blindly return a response of your choice.  The best solution seems to involve using a small a Perl/CGI script.  So, I hacked some things together to get this to work. Andrew Grangaard alerted me that you can configure RedirectMatch to return an HTTP 204 No Content with mod_alias.

If you'd like to generate your own self-signed SSL certificates for use with Apache, the openssl command makes it easy.  At home, I run a few HTTPS development Apache instances that use my own self-signed SSL certificates.  Granted these certificates are not signed by a legitimate Certificate Authority (like Verisign, Thawte, or Network Solutions), but they get the job done if you want quick and cheap SSL security.  Keep in mind that if you use a self-signed certificate, a web-browser will probably complain as shown here.  So, you probably shouldn't use this in a real production environment.  However, for development stuff at home, this is perfect.

Here's how you can generate your own self-signed SSL certificates:

openssl genrsa 1024 > example.com.key
openssl req -new -key example.com.key -x509 -days 365 -out example.com.crt

Now that you've got a key and certificate (a .crt file), you can integrate them into Apache.  This involves using the SSLCertificateFile and SSLCertificateKeyFile directives in your Apache configuration file that defines an HTTPS VirtualHost.  You need to configure these directives to point to your certificate and key files, respectively.  In my environment, this configuration goes into /etc/httpd/conf.d/ssl.conf ...

##
## SSL Virtual Host Context
##

<VirtualHost _default_:443>
 ...
 SSLCertificateFile /path/to/crt/file/example.com.crt
 SSLCertificateKeyFile /path/to/key/file/example.com.key
 ...
</VirtualHost>

Remember, your private key (your key file) is important.  You should keep it in a secure/private place on your server, and not in a publicly readable directory.

I've been going key crazy over the last several days working on digital signing in PHP, and now Java.  It's not hard, but what's really confusing is all of the key generation and manipulation stuff before you even get to the code (the point where you want to actually use the key to sign some data).  PHP wants a plain RSA key in DER format.  Java seems to prefer a PKCS#8 base64 encoded RSA key in DER format with no password.  I know, this stuff seems like it's all over the place.

This post is an attempt to document what worked for me.  There are definitely other signing methods out there, but I finally got PKCS#8 with an RSA key in DER format to work in Java.  Here's how ...

I've been banging my head against my cube for a while on this one, and finally gave into the fact that I couldn't get SHA1withDSA signing in PHP to work.  In Java, SHA1withDSA signing appears to be a no-brainer.  However, in PHP, it's a disaster; if you know how to get SHA1withDSA working in PHP, please contact me.  Bottom line, I finally decided to give up on DSA and started using SHA1withRSA to digitally sign a request in PHP.  I need to do this because I have a PHP web-app that is communicating with a REST interface which requires each request to be digitally signed.  The whole point of the digital signature is so that the REST API can validate that the request really is coming from the sender; in other words, the sender really is who they say they are.

Continue reading or the gory details ...

At work, I sit behind a cluster of BlueCoat 800-2 Web Cache proxy appliances.  For security and efficiency, all of our outbound web-traffic is sent through this cluster.  This is fine, except that the BlueCoat appliances in this cluster do NOT obey the configuration guidelines in my robots.txt.  And, they unnecessarily ping many of the requested resources sent through them.  In other words, if you load a web-page using this web-proxy cluster, the cluster will continuously "ping" the site and its resources over and over again at a fixed interval.  I assume this is the cluster checking the site to see if anything has changed.  From a system administrators standpoint, this is slightly irritating because the extra "ping" adds to the load of the server and unnecessarily consumes bandwidth (the cluster appears to repeatedly request the same resources with a GET).

So, I took matters into my own hands and tweaked my root .htaccess file to block these pings with a 403 Forbidden.  Strangely enough, BlueCoat appliances appear to use an HTTP user-agent of "Mozilla/4.0 (compatible;)".  This is definitely not a legit user-agent string; the user-agent should identify the device that's making the request.  Lucky for me, this simplifies my .htaccess configuration a bit:

SetEnvIf User-Agent "^Mozilla\/4\.0 \(compatible\;\)$" block=1
Order allow,deny
Allow from all
Deny from env=block

And that, takes care of that.

This weekend I added a custom Google Translate widget to my blog.  Even though I'm an extremely patriotic American, I still appreciate the rest of the world and the many languages it speaks.  As a result, I've tried to make it somewhat convenient for foreign visitors to translate my blog into their native language.  I implemented the functionality of the translate widget in JavaScript; using jQuery of course.  You can find the full JavaScript here.  But, here are the important pieces (the functions) that you might care about if you're trying to implement your own Google Translate widget:

var gT = "http://translate.google.com/translate?u=%u&sl=en&tl=%tl";

function isKolichDotCom(){
  return window.location.href.indexOf("http://mark.koli.ch") == 0;
}

function setupTranslate(){
  $("p.translateflag").click(function(e){
    var href = encodeURIComponent(window.location.href);
    var tl = $(this).attr("tl");
    gT = gT.replace("%u",href);
    gT = gT.replace("%tl",tl);
    window.top.location.href = gT;
  });
}

$(document).ready(function(){
  if(isKolichDotCom()){    
    setupTranslate();
  }
});

I'd love to turn this post into a HOWTO and explain more about how everything works, but I need to take care of some other things around the house.  Plus, my right wrist is recovering from a brutal week @work, so I need to keep computer use to a minimum right now (carpal tunnel).  Enjoy.

Happy Friday the 13th.

You can install a Mozilla Firefox extension two ways.  First, if you have a .xpi file (a zip'ed, pre-packaged Firefox extension file), you can simply launch Firefox then select "File->Open File".  Select the XPI, and click Open.  This will kick off the extension installation in the browser.  This is neat, but it isn't as exciting as letting a user install your Firefox extension with a single "click here to install" link, like the Mozilla Firefox Add-ons portal.  So, a second option is to write up some slick JavaScript to call InstallTrigger.install() which triggers the Firefox extension installation.  Of course, there's no better way to do this than with jQuery.

Using jQuery to trigger the Mozilla Firefox extension installer is a snap.  You can find a demo, the JavaScript code, and other details here.  Enjoy.

Just for grins, I decided to run a few tests to determine the max size of an UNSIGNED BIGINT(20).  I use several databases where Hibernate's OBJ_VERSION field is defined as an UNSIGNED BIGINT(20), and I was curious to see how MySQL behaves with very large numbers.  As it turns out, the max size of an UNSIGNED BIGINT(20) is "18,446,744,073,709,551,615" ...

mysql> UPDATE colors SET c_name = 'Blue',
        OBJ_VERSION = 18446744073709551615
         WHERE c_code = 'BLU';
Query OK, 0 rows affected (0.00 sec)
Rows matched: 1  Changed: 0  Warnings: 0

mysql> UPDATE colors SET c_name = 'Blue',
        OBJ_VERSION = 18446744073709551616
         WHERE c_code = 'BLU';
ERROR 1264 (22003): Out of range value for column 'OBJ_VERSION' at row 1

I thought that MySQL might "wrap the value" around back to zero if you try to UPDATE with a value greater than the column's max.  But, it definitely does not.  In my case, this is probably not too important because in order to reach 18,446,744,073,709,551,615 Hibernate would have to pound on one specific database record over a quintillion (10^18) times.

It seems a little presumptuous, but I guess Hibernate operates on the assumption that a real database would never reach this limit?  Anyways, I thought this was interesting.  If you'd like to read more about Hibernate's version'ing scheme (that OBJ_VERSION thing I'm talking about) check this out.

Yesterday I posted a HOWTO that explained how to use Apache's mod_rewrite RewriteMap to create your own tiny URL alias redirect service (like bit.ly, tinyurl.com, etc.).  A few hours after I posted this HOWTO, I realized that using mod_rewrite's RewriteMap with the "txt:" map-type doesn't always work the way you want it to.  Come to find out, when you use RewriteMap with a "txt:" map file, Apache uses its own internal URL escaping mechanism (Adrian Sutton) that tweaks the URL you're trying to redirect to.  For example, a URL that contains ...

"© Mark Kolich" becomes "%C2%A9%20Mark%20Kolich"
or
"^DJI" becomes "%5eDJI"

This is OK, except that a lot of web-apps I'm trying to redirect to don't properly un-escape the URL before processing it (like Yahoo's interactive charts).  I did a little more research on this and realized the only solid alternative was to write a Perl script that mapped a key to a URL using mod_rewrite's "prg:" map-type.  Hence, avoiding Apache's own internal URL escaping mechanism.

I recently started using Twitter, and noticed that a lot of folks (myself included) like to use URL shortening services when posting a Tweet.  Since Twitter only lets you post Tweets 140-characters or shorter, services like bit.ly, tinyurl.com, and twurl.cc are critical when you want to Tweet a page with a long URL (like this blog post).  Just for fun, I started to look into how these URL shortening services work.  After a few minutes, I decided I wanted to make my own (a tinyurl.com, bit.ly, twurl.cc clone).  Not surprisingly, it's absolutely trivial to make your own URL alias service using Apache's mod_rewrite.

FOLLOWUP 3/21/09:  There's a better way to do this, as I described here.

Here's another irritating Linux vs. Windows difference ...

On Windows, a file with a name of "README.txt" is the same as "readme.txt".  On Linux, these are two completely separate files.  This can make things tricky when developing Java code on Windows, and deploying on Linux.  If your code references a resource (a file of some kind) with mixed capitalization in the file name, you must remember that referencing that resource may work on Windows but probably not on Linux.

Let's say you have "myresource.txt" in a directory.  In Java ...

new File("MyReSoUrCe.tXt").exists() returns true on Windows.

new File("MyReSoUrCe.tXt").exists() returns false on Linux.

Granted, most people wouldn't put something ridiculous like "MyReSoUrCe.tXt" in their code.  A more likely example is "MyResource.txt".  But, you get the idea.  BTW, I have no significant personal preference with regards file name case sensitivity.  However, I suspect Microsoft intentionally left this file name case convention in place.  Therefore, Windows would be backwards compatible with older DOS applications that expect "README" to be the same file as "readme".

Bottom line, just use an all lower case naming convention for any resources referenced by your code; regardless of what platform you are developing/deploying on.  Doing so will save you many hassles in the long run.

I spent a few cycles this afternoon integrating my Twitter timeline with my Movable Type blogging software.  Surprisingly, things worked out great with no issues in the first version.  The Twitter API is a well-written REST service that made it incredibly easy to combine my lifestream with my Tweets.  BTW, I hate the word "Tweet" but that's the terminology everyone in Twitter land seems use so I will too.

To start, I snagged the php-twitter library from Google Code.  The only php-twitter functions I really cared about were process() and userTimeline(), which use libcurl to read your Tweets in JSON or XML format.  Once I had the code I wanted, I created my own PHP class named TwitterCacher to handle all of the API calls and response caching.  The idea behind TwitterCacher is to work around a rate limiting problem of the Twitter API.  Twitter limits the number of API requests you can make in an hour or so, so if you use the Twitter API you have to be conscious of this limitation (otherwise, Twitter will blacklist you).  As a result, the developer using the Twitter API has to implement some type of caching mechanism to prevent hitting Twitter too often.  To solve this caching issue, my TwitterCacher saves the JSON or XML response from the Twitter API to a flat-file on your web-server.