July 2009 Archives

Unless you've been living under a rock for the last few years, you've probably heard of the great doomsday scenario surrounding IPv4.  Someone even wrote a cute iPhone app to count down to IPv4 doomsday: the day the world runs out of IPv4 addresses.  Fact is, we're running out of IPv4 addresses; those super convenient and useful ID's used to uniquely identify each attached device/computer/whatever on the Internet.  For grins, I decided to use ARIN's public WHOIS database to determine who owns the major Class A (CIDR /8) IP address blocks.

Over the last week or so, I received numerous direct messages from several blog readers regarding my TwitterCacher library.  Folks want to know if it's possible to use TwitterCacher with the Twitter Search API.  The answer is no, and you shouldn't have to.  Using the Twitter Search API, you can write code to read a Twitter feed, or search Twitter, without having a Twitter account.  If you want to write an app that simply monitors Twitter, or a feed from a specific user, you might find these examples useful.  As shown below, note that you can replace .atom with .json if you want the response in JSON format.

• Read a user's feed (get all of their tweets), use from:username ...
  
  XML:  http://search.twitter.com/search.atom?q=from%3Amarkkolich
  JSON:  http://search.twitter.com/search.json?q=from%3Amarkkolich
  
  
• Find all tweets containing a specific hashtag, use tag=[tag] ...
  
  XML:  http://search.twitter.com/search.atom?tag=hp
  JSON:  http://search.twitter.com/search.json?tag=hp
  
  
• Replies to a user (e.g., tweet starts with @markkolich) ...
  
  XML:  http://search.twitter.com/search.atom?q=to%3Amarkkolich
  JSON:  http://search.twitter.com/search.json?q=to%3Amarkkolich
  
  
• Mentioning a user (e.g., tweet contains with @markkolich) ...
  
  XML:  http://search.twitter.com/search.atom?q=%40markkolich
  JSON:  http://search.twitter.com/search.json?q=%40markkolich
  
  
• Referencing a user but not from that user ...
  
  XML:  http://search.twitter.com/search.atom?q=markkolich+-from%3Amarkkolich
  JSON:  http://search.twitter.com/search.json?q=markkolich+-from%3Amarkkolich
  
  

I'll leave the reading and parsing of the results up to you.  Enjoy.

I decided to change the URL pattern to each blog post on my mobile blog.  Until this morning, the URL to each blog post looked something like this:

http://mark.kolich.mobi/m.php?a=1

But, in the spirit of REST'ful API's I changed the URL pattern to something a little cleaner, and maybe a little more user friendly (easier to remember and recognize):

http://mark.kolich.mobi/m.php/1

Most folks would never even remotely care about this, but I'm somewhat of a perfectionist and maybe even a slight minimalist.  So, if this new pattern saves me 2-characters in the URL, then I'm happy.  The trick through was updating the PHP that powers my mobile blog so that it gracefully recognizes the old and new style URL.  Many search engines and other users have cached links to articles on my mobile blog, so if they use the old style URL, I want to gracefully redirect them to the new one.  Here's the PHP that let's me handle both styles, old and new:

// The article ID can only contain numbers, so if we get anything else
// use preg_replace() to remove all non-digits.
$a = (isset($_GET['a'])) ? preg_replace("/\D/","",$_GET['a']) : null;

// Check if $_GET['a'] really exists and make sure it's a number
// between 1 and 20. If it doesn't exist, or doesn't meet our criteria,
// then look for an article number in the new style URL. BTW, PHP's intval()
// returns zero (0) if the number couldn't be parsed for whatever reason;
// which works fine for us here.
if(empty($a) || preg_match("/^\d{1,2}+$/",$a)===0 ||
    intval($a)<1 || intval($a)>20){

  // Try to get the article number from the REQUEST_URI string.
  $uri = (isset($_SERVER['REQUEST_URI'])) ? $_SERVER['REQUEST_URI'] : null;
  $path = explode("/",$uri,4);
  $a = (isset($path[2])) ? preg_replace("/\D/","",$path[2]) : null;

  // See if we found anything useful. If not, then bail. Looks like the
  // user sent us some junk we don't understand.
  if(empty($a) || preg_match("/^\d{1,2}+$/",$a)===0 ||
      intval($a)<1 || intval($a)>20){
    header("Location: http://mark.kolich.mobi");
    return;
  }

}
else {

  // Tell search engines that the URLS for the mobile
  // posts have changed by sending them a 301.
  header("HTTP/1.1 301 Moved Permanently");
  header("Location: ".$_SERVER['PHP_SELF']."/".$a);
  return;

}

// If you get here, the value of $a is loaded
// and validated from *.php/x

If you're feeling adventuresome, and you're running PHP 5.2 or later, you could use PHP's filter_input_array() to validate the input as I described here.

Even so, this works well.  If the user visits http://mark.kolich.mobi/m.php?a=1 then I'll redirect them to the new style URL http://mark.kolich.mobi/m.php/1.  If the user requests a bogus article, or gives me junk instead of a valid number between 1 and 20, I'll throw them out to http://mark.kolich.mobi.

If you find any problems with this code, please let me know. Enjoy.

If you've ever looked at the code for some of the more popular mobile web-portals, you might have noticed that many of them include base-64 encoded binary image data in their cascading style sheets.  For example, Google's iPhone portal uses this content delivery mechanism to deliver image content to mobile browsers.  Here's an example using an inline style sheet to define the background-image of a <div>.  Note that the GIF image data itself is embedded inside of the CSS:

<style type="text/css">
 div.wrapper {
   background-image: url(data:image/gif;base64,R0lGODdhAQAgAMIIABstehsv
      fRwwgR4zhB81iiA3jSE6kvf6/ywAAAAAAQAgAAADDji21U1wSClqDRjow08CADs=);
 }
</style>

FWIW, I'm line-wrapping the base-64 encoding binary data above.  However, normally you don't want to line-wrap binary data when including it in an inline CSS declaration.  I'm simply wrapping above for display purposes. Here's a more complete, yet quick example.

As defined by the RFC 2397, the accepted syntax/format of data URI's are as follows:

data:[<mediatype>][;base64],<data>

Continue reading for the details ...

I released Gagawa PHP 1.2-beta on Google Code this afternoon. This version contains a fix for an important enhancement which was requested here on June 24th.  Prior to this release, the Doctype class was not properly declaring the correct HTML/XHTML document type.  Gagawa PHP 1.2-beta has been enhanced to support Doctypes defined by the user.  I've also added a Document class, that lets the user build a complete HTML document like so:

<?php

require_once("gagawa.php");

// A set of all common Doctypes are defined as const's
// in the new Doctype class.  This example is using
// HTMLStrict.
$d = new Document( Doctype::HTMLStrict );

$div = new Div();
$div->setId("mydiv")->setCSSClass("myclass");
$div->appendChild( new Text("some text in a div") );

$title = new Title();
$title->appendChild( new Text("Page title!") );

// The new Document class defines two public class
// members, a head_ and body_.  These Head() and Body()
// objects let the user build a complete HTML page.
$d->head_->appendChild( $title );
$d->body_->appendChild( $div );

echo $d->write();

?>

Special thanks to @namibcoder for reporting this missing piece of important functionality to us on our Google Code homepage.

Hot-linking (a.k.a., inline linking) of images and other resources is a common problem for system and network administrators across the web.  In a nutshell, hot-linking involves folks using your site/blog as the hosting mechanism for images and other resources on their sites or forums.  Wikipedia has a nice write up on hot-linking.  For example, when a user posts a message on a forum that includes an <img> tag sourcing an image hosted on kolich.com:

<img src="http://mark.koli.ch/american-flag.png" />

This is irritating to system and network administrators because it wastes bandwidth and unnecessarily consumes server resources.  Unless you're running a dedicated static content server, why would you want to use your web-server and network bandwidth to host images for forums or other blogs?  In most cases, you don't.

Luckily, Apache's mod_rewrite module makes it easy to stop hot-linking by blocking requests for images and other content that don't originate from your domain.  Here's my Apache configuration (in httpd.conf) that stops the hot-linking of my images.  Note, you can also put this in an .htaccess file assuming your hosting provider gives you permission to do so:

## Prevents hot-linking of my images on other sites.
## Will send back a 49x49 transparent pixel in place
## of hot linked images sourced from other sites.
RewriteCond %{HTTP_REFERER} !^http://(.+\.)?kolich\.(com|mobi)/ [NC]
RewriteCond %{HTTP_REFERER} !.*twitter\.com/ [NC]
RewriteCond %{HTTP_REFERER} !^$
RewriteRule .*\.(jpe?g|gif|bmp|png)$ /pixel.gif [L]

Using RewriteCond, I can check the HTTP referrer header to ensure that the request for an image originated from a user visiting my blog (under *.kolich.[com,mobi]).  I also allow requests originating from *.twitter.com given that I often post images and other stuff to my Twitter feed.  If the request for an image did not originate from *.kolich.[com,mobi] (e.g, on a forum or another blog) I simply send back a 1x1 transparent pixel (49-bytes) instead of the image itself.

Here's a request in my log files showing that someone tried to hot-link to my "case of the monday's" image in this forum thread.  Note that I'm sending back the 1x1 transparent pixel (49-bytes) instead of the actual JPG image the user tried to source:

173.34.220.243 - - [11/Jul/2009:11:40:18 -0700]
   "GET /2009/01/05/case-of-the-mondays.jpg HTTP/1.1" 200 49
   "http://forums.vwvortex.com/zerothread?id=2130343&page=63"
   "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; GTB5)"

Enjoy.

While working on MySQL triggers this afternoon, I encountered a strange problem that dealt with using a trigger to tweak records in a table after an UPDATE.  Specifically, I wanted to write a trigger that would automatically loop over all records in a table, and then mark any "expired" if they were added more than X days ago.  I added a trigger using the UPDATE mechanism I thought would work, but nope, I kept getting this error after an UPDATE:

ERROR 1442 (HY000): Can't update table 'pass' in stored
   function/trigger because it is already used by statement
   which invoked this stored function/trigger.

Looks intimidating.  According to the MySQL forums, a fairly large number of folks have reported this same problem.  Sadly, I wasn't able to find a good solution to the problem, and according to MySQL's documentation, what I'm trying to do is damn near impossible with triggers.

This post is an attempt to document a strange occurrence when using the Apache Commons HTTPClient to establish an HTTPS connection through a web-proxy.

In my travels at work, I recently came across an interesting situation using the Apache Commons HTTPClient library.  For a project, I'm using the Commons HTTPClient to open a TCP based tunnel through a web-proxy.  Unfortunately, I have to use a proxy because my employer forces me to use one for all outgoing HTTP traffic.  This means that when I want to establish a secure tunnel to a web-server outside of my employer's corporate firewall, the Apache Commons HTTPClient must open a TCP tunnel through my employer's web-proxy.  In other words, when I need the HTTPClient to connect to a secure web-site with HTTPS, it needs to open a TCP tunnel through the proxy.

In doing so, I see a ton of these casual INFO messages from the HTTPClient library in my log files:

INFO: Response content length is not known Apr 10, 2009 1:12:26 AM
  org.apache.commons.httpclient.HttpMethodBase readResponseBody
INFO: Response content length is not known Apr 10, 2009 1:12:29 AM
  org.apache.commons.httpclient.HttpMethodBase readResponseBody

It looks like the warning is coming from the readResponseBody() method of the HttpMethodBase class.  What gives, man?

Happy July!  I apologize for my recent lack of blogagge bloggage (is that a word?); work and life have been keeping me plenty busy and I haven't had much time to sit down and write up any lengthy blog posts.  Well, tonight is the exception.

In the last week, I stumbled across a few mobile sites that hide the iPhone's Safari browser toolbar on page load.  In other words, the little address bar at the top of the browser disappears (automatically scrolls up) once the page finishes loading.  I did a little research to figure out how this worked, and discovered that a simple JavaScript one-liner can take care of this for you:

window.scrollTo(0,1);

According to this post on iPhone Microsites, window.scrollTo(0,1) is the officially documented method for hiding the iPhone toolbar.  On my mobile blog portal at http://mark.kolich.mobi I added this JavaScript one-liner, and sure enough, it does work!  Here's a before and after screenshot (yes, this is a screenshot from an emulator, not an actual iPhone).

Continue reading for some additional HOWTO details ...