eHow Earnings Tracker : A New Direction

One of the major challenges with doing software development is that you need to be constantly evaluating the direction that you’re heading in with both the code and the end product.  You can’t just decide to do something and then put your head down and do it.  The landscape changes as you develop a product – you learn better ways to do things, roadblocks come up, and your vision of the end product (and code) becomes more refined and  clear.  If you don’t pay attention to these things as you go along then inevitably you’ll wind up miles away from where you should be in the end.

On the other hand, you can’t be changing direction with the wind either.  This is a trap that many programmers fall into.  If you don’t have a solid starting vision for the product or you constantly change the requirements for the product or how it is implemented then you’ll wind up with a big mess or, even worse, the product will never get finished.

The balancing act of changing things that need to be changed and not changing things too often is one of the things that makes software development as much like an art or craft as an engineering or science.  Complicating things further is the fact that what’s best to do for the product often conflicts with what’s easiest to do in the code.

I was working on the eHow Earnings Tracker last night when I had one of those “aha!” moments where I knew that I needed to change the direction I was heading in.

My original vision for the tracker was that it would automatically detect eHow earnings pages as the user browsed the web.  When an earnings page was detected, the tracker would read the current page of earnings, save them to a file, and compare them to the previous update.  The user would be able to see which articles had increases in earnings from update to update and they would have a history of earnings that could be exported.  This  seemed like it would be fairly straightforward to implement and would be a low friction solution for the user.

As I worked on the code, a number of implementation difficulties arose and it became apparent that there would be some usability issues with the end product.

Implementation difficulties (code):

  1. Firefox can have multiple tabs open at once which means that, in theory, multiple earnings pages could be loading at the same time.  Writing code to handle this sort of scenario is very difficult in a Javascript based environment.
  2. Due to the fact that the code is only parsing one page of articles at a time and the article order is not guaranteed, the code can’t just blindly read & write the entire contents of the current earnings file stored on disk.  Each time an earnings page is viewed, the file needs to be read in and then its data needs to be compared to the current page of earnings to see whether article entries need to be added or modified.
  3. The code would be executing every time an earnings page was viewed, even if it didn’t have to.

Usability issues:

  1. The user would have to manually click through each page of articles (which can get tedious since articles are only displayed 10 at a time).
  2. Since the code was only seeing one page of articles at a time, it would have no way to inform the users of any articles that were added or removed since the last update.

My “aha!” moment was in realizing that the better way to do this would be give the user an “Update Earnings” button that could be activated whenever they were on their earnings page.  That button will trigger the tool to automatically navigate through all of the users’ earnings pages collecting information.  At the end of the process, an HTML page can be displayed showing a summary of all the earnings data.  This fixes all of the implementation and usability issues in one shot.  It’s a rare case where I can simplify the code and improve the product at the same time.

How to Track eHow


Since I now have a Firefox plugin that will properly update the browser’s status bar with the URL of the web page being viewed, it’s time to write code that will specifically detect an eHow earnings page.  For those of you following along at home, take the code from the previous post and add the following function just after the updateStatusBar function :

var update = function()
{
    var host = Application.activeWindow.activeTab.uri.host;
    var path = Application.activeWindow.activeTab.uri.path;

    m_eHowMsg = ""

    if(host.search(/www.ehow.com/i) == 0)     
    {         
        // eHow earnings URLs are in the form "http://www.ehow.com/members/user-p#-articles.html" or  "http://www.ehow.com/members/user-articles.html"         
        var userAndPageNumPattern = new RegExp("\/members\/\(.+)-(.+)-articles.html");         
        var userAndPageNumResult = userAndPageNumPattern.exec(path);          
        var userPattern = new RegExp("\/members\/\(.+)-articles.html");         
        var userResult = userPattern.exec(path);          

        if(userAndPageNumResult && (userAndPageNumResult.length < 1))         
        {             
            m_eHowMsg = userAndPageNumResult[1];         
        }
        else if(userResult && (userResult.length &lt; 1))         
        {             
            m_eHowMsg = userResult[1];         
        }     
    }      
    updateStatusBar(m_eHowMsg);
}

This function looks for URLs that match eHow earnings pages and then updates the status bar with the eHow user name that it finds.  To hook this function up, just replace the calls to updateStatusBar in onTabSelect() and onTabLoad() with calls to update().

Now when we navigate to a URL that contains eHow article earnings data we see the username in the status bar (the status bar will be blank for other URLs):

ehow username in statusbar

A Wrench in the Works – Multiple Tabs in Firefox

The code from my previous post worked great with a single tab open in Firefox.  However, using multiple tabs exposed a problem – all tabs in a single Firefox window share the same status bar.  Another issue is that the event “DOMContentLoaded” gets fired any time a page finishes loading, regardless of what tab you’re on.  This means that the information in the status bar may or may not actually match the tab that you’re on.

I also found some better techniques for placing my code in a namespace/module that are described here and here.  It’s also worth noting that Mozilla’s Developer Center has documentation for their Javascript APIs and objects.  Unfortunately, the documentation usually isn’t very good and I wind up spending an awful lot of time looking for simple pieces of information.

After a bit of work, I came up with the code below to do what I wanted.  This code only works for Firefox 3.5 because that’s when event data was added for the TabOpen, TabClose, and TabSelect events.  In the code below, event.data is a BrowserTab object.

// Create a global variable for the addon
var g17of26 = {};

g17of26.eHowEarningsTracker = function ()
{
    // Private functions
    var updateStatusBar = function(msg)
    {
        var statusBar = window.document.getElementById("ehow_earnings_tracker_statusbar");

        if(statusBar)
        {
            statusBar.setAttribute("label", msg);
        }
    }

    return {  // This brace must stay on the same line as the return - it's a Javascript quirk

        // This function is called when the browser is first launched
        onBrowserLoad : function()
        {
            Application.activeWindow.events.addListener("TabOpen", g17of26.eHowEarningsTracker.onTabOpen);    
            Application.activeWindow.events.addListener("TabClose", g17of26.eHowEarningsTracker.onTabClose);
            Application.activeWindow.events.addListener("TabSelect", g17of26.eHowEarningsTracker.onTabSelect);

            Application.activeWindow.activeTab.events.addListener("load", g17of26.eHowEarningsTracker.onTabLoad);
        },

        // This function is called when the browser quits
        onBrowserUnload : function()
        {
            window.removeEventListener("load", g17of26.eHowEarningsTracker.browserLoad, false);
            window.removeEventListener("unload", g17of26.eHowEarningsTracker.browserUnload, false);        

            Application.activeWindow.events.removeListener("TabOpen", g17of26.eHowEarningsTracker.onTabOpen);
            Application.activeWindow.events.removeListener("TabClose", g17of26.eHowEarningsTracker.onTabClose);
            Application.activeWindow.events.removeListener("TabSelect", g17of26.eHowEarningsTracker.onTabSelect);
        },

        // This function is called when a tab is opened
        onTabOpen : function(event)
        {
            event.data.events.addListener("load", g17of26.eHowEarningsTracker.onTabLoad);
        },

        // This function is called when a tab is closed
        onTabClose : function(event)
        {
            event.data.events.removeListener("load", g17of26.eHowEarningsTracker.onTabLoad);
        },

        // This function is called when a tab is selected
        onTabSelect : function(event)
        {
            updateStatusBar(Application.activeWindow.activeTab.uri.spec);
        },

        // This function is called when a page finishes loading in a tab
        onTabLoad : function(event)
        {
            if(event.data.uri.spec == Application.activeWindow.activeTab.uri.spec)
            {
                updateStatusBar(event.data.uri.spec);
            }
        }
    };
}(); // the parens here cause the anonymous function to execute and return

// Hook up events
window.addEventListener("load", g17of26.eHowEarningsTracker.onBrowserLoad, false);
window.addEventListener("unload", g17of26.eHowEarningsTracker.onBrowserUnload, false);

Software Spotlight : TreeSize

Ever wonder where all of the space on your hard drive has gone?  I certainly do from time to time, and for years I’ve been using a great tool called TreeSize Free from JAM Software to find out.  TreeSize Free is a lightweight program (~1.55MB)  that hooks itself into the context menu in Windows Explorer.

Once it’s installed, you can just right click on any directory or drive and select “TreeSize Free” from the context menu.  TreeSize will then scan the selected directory or drive and give you a summary of all the space taken up by the files and subdirectories.

TreeSize Free has a variety of options that let you customize the display.  You can choose the units (KB, MB, GB, or mixed) to use in the display or instead you can view by percentage of space occupied.  You can even tell it to show you the number of files in each directory.  Overall, it’s a very useful tool that’s simple to use.

tree sizeSo that’s where my hard drive space went!

If you need some serious hard drive analysis and reporting, they have a professional version available.

Making a Firefox Addon Do Something

As it turns out, hooking up Javascript to XUL so that an addon can actually do something is remarkably similar to linking HTML and Javascript together when doing web development.  The first step is to create a Javascript file and include it it in the XUL file much like you would include a Javascript file in an HTML file:

<overlay id="ehow_earnings_tracker_overlay" xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul">
    <script type="application/x-javascript" src="chrome://ehow_earnings_tracker/content/browser_overlay.js" />
    <statusbar id="status-bar">
        <statusbarpanel id="ehow_earnings_tracker_statusbar" label="" />
    </statusbar>
</overlay>

Now we can add code in browser_overlay.js to actually do something.  In this case, we will update the status bar text with the URL of HTML page that was loaded:

// Creates a global variable called "ehow_earnings_tracker"
var ehow_earnings_tracker =
{
    init:
        function()
        {
            // Grab the contents of the browser
            var appcontent = document.getElementById("appcontent");
            if(appcontent)
            {
                // Get notified when the page is done loading
                appcontent.addEventListener("DOMContentLoaded", ehow_earnings_tracker.onPageLoad, true);
            }
        },

    onPageLoad:
        function(aEvent)
        {
            var doc = aEvent.originalTarget; // doc is document that triggered "onload" event
            var statusBar = document.getElementById("ehow_earnings_tracker_statusbar");

            if(!statusBar)
            {
                return;
            }

            statusBar.setAttribute("label", doc.location.href);
        }
};

// Hook into the window load event
window.addEventListener("load", ehow_earnings_tracker.init, false)

Now, when I run Firefox with my plugin enabled it dynamically updates the status bar with the URL of the loaded page:

url in statusbar

The Search for a WordPress Plugin to Display Code

A big part of this blog is going to be posting code snippets from various languages so I needed to find a way to display code with syntax highlighting in WordPress posts.  Finding a WordPress plugin to display code was a bit overwhelming at first.  There are a lot of plugins out there that display code and do syntax highlighting and it was difficult to know which ones were any good and what versions of WordPress they would work on (some of the last modified dates were over a year ago).  Many of them require disabling the WSIWYG editor, which is annoying.

My searching brought me to this article that listed 12 plugins.  Unfortunately, I didn’t like the look of most of the plugins on the list and the one that I did try (WP-SynHighlight) didn’t work quite right under WordPress 2.8.  At this point I decided to see if I could figure out what StackOverflow uses since I really like the syntax highlighting there.  Turns out that they use Google’s Prettify and there’s a WordPress plugin implemented with it.  That plugin also did not work well under WordPress 2.8 (the posted version is from well over a year ago).

Not to be defeated, I continued my search and found a great article from someone who actually installed and tested 8 syntax highlighting plugins.  I went right to the top of the list and installed Syntax Highlighter Plus.  I installed it and it actually worked!  I’m not thrilled with the styling on the code snippets so I’ll probably tweak it a bit, but it looks solid and it works.

Here’s the list of languages that it supports:

* Bash — `bash`, `sh`
* C++ — `cpp`, `c`, `c++`
* C# — `c#`, `c-sharp`, `csharp`
* CSS — `css`
* Delphi — `delphi`, `pascal`
* Diff — `diff`
* Groovy — `groovy`
* Java — `java`
* JavaScript — `js`, `jscript`, `javascript`
* Perl — `perl`, `pl`
* PHP — `php`
* Plain text — `plain`, `text`
* Python — `py`, `python`
* Ruby — `rb`, `ruby`, `rails`, `ror`
* Scala — `scala`
* SQL — `sql`
* VB — `vb`, `vb.net`

* XML/HTML — `xml`, `html`, `xhtml`, `xslt`

If you are going to use Syntax Highlighter Plus then you’re also going to want to grab Visual Code Editor and this article explains why.  However, if you decide to use Visual Code Editor then you need to use version 0.18 of Syntax Highlighter Plus instead of the latest version.

Creating a Firefox Addon

Firefox addons are implemented using a combination of XUL, Mozilla’s proprietary XML based language, and Javascript.  For anyone who’s done development with HTML and Javascript, learning to write a Firefox addon is pretty straightforward.  All you need is a decent tutorial to get started.  Fortunately, finding tutorials on how to create addons for Firefox is pretty easy.  Some of the ones I found were pretty old, but as far as I could tell the information was still accurate. If you need to learn Javascript, there are plenty of tutorials over on W3Schools.  If you prefer books, I highly recommend David Flanagan’s Javascript: The Definitive Guide.

I decided to start with a tutorial from Mozilla which shows you how to create a classic “Hello World” program by displaying “Hello, World” in the browser’s status bar.   I went through the tutorial quickly and easily by following the exact steps that were outlined in the tutorial.  “Hello, World” showed up my the status bar without  a hitch.  The flaw with this tutorial is that it doesn’t show you how to make the addon actually do something, which is the topic of my next post.

Another useful article I found was this article on Lifehacker that gives some good tips for getting set up to develop Firefox addons.

Development Project : eHow Earnings Tracker

Last fall I was looking for a way to make some additional income and my wife, whose career currently consists of writing online, pointed me toward eHow.  I’ve never been much of a writer but I thought I’d do ok with writing How-To articles about programming and other computer related topics.  I browsed around the site and found that most of the computer related content was just awful.  Almost all of it was poorly written, out of date, or just completely inaccurate.  I thought to myself “I can do better than this!”.

So in October I created an account and wrote a couple of articles about programming with WPF.  I got distracted with other things for a while the two articles earned next to nothing over the next six months.  In late April I decided to give it another shot and I’ve written 25 more articles since then.  I still haven’t earned much of anything, but from reading the eHow forums I’ve concluded that you really need to reach a critical mass of article count and a certain threshold of article age before you really start seeing any sort of real money.  Even just going from 2 articles to 27 over the course of two months I’ve noticed a significant acceleration in the rate that my earnings are increasing.  Of course, creating this blog has completely stalled my eHow article writing :) .

eHow writers earn money through some secret earnings algorithm that seems to be primarily based on people clicking ads on your articles.  The official eHow line is that the earnings are based on a variety of factors but the primary driver really seems to be  ad clicks.  Fortunately for me, computer related ads seem to do very well in terms of CPC (cost per click – the rate that advertisers pay for ad clicks).

The previous two paragraphs are background info for the topic of this post – tracking eHow earnings is a royal pain in the butt.  eHow gives you very limited data for your earnings.  For each article in your library, you are shown lifetime views and lifetime earnings.  The only other numbers you get are lifetime views for all articles and total earnings for the month.  This means that you have no way to track article views or earnings over time unless you manually record them every day.  To make matters worse, you can only see data for 10 articles at a time.  For a few days I tried tracking the earnings manually using a Firefox addon called TableTools to copy the earnings data and paste it into Excel.  That just sucked and I was determined to create a better way.

My first thought was to create a Windows application that could make HTTP requests to snag the earnings data since it would involve using technology that I was already familiar with.  I opened up Firebug and watched the HTTP traffic as I logged into eHow and viewed my earnings pages.  I immediately realized that trying to decipher all of the traffic to figure out what requests I’d need to make was going to be a total nightmare.

It then dawned on me that this might be a good thing to implement using a Firefox addon.  From my experience using other Firefox addons, I knew that they were capable of doing everything that I would need it to do.  Perfect!  I would get an opportunity to learn how to write Firefox addons, I would be creating something that I would want to use on a regular basis and something that other people would most certainly want to use.  I was sold – off to learn how to write a Firefox addon!

Go, Go WordPress Widgets!

Every WordPress theme has one or more sidebars in it (in the case of this site, the sidebar is that gray section on the right hand side of the screen).  Sidebars contain widgets, which are modules that show specific types of information such as Archives, Links, or Categories.

The next step after choosing a theme is to configure these widgets.  This is done by going to the Appearance->Widgets screen.  You can drag and drop widgets in and out of the sidebar.  You can also reorder the widgets in the sidebar by dragging them around.  Most widgets have some sort of configurations settings (as show in the screenshot below).

wordpress widgets

My Kingdom for a Good WordPress Theme

One of the first things I wanted to do when I got my blog up and running was to pick a new theme. I had a few requirements that I wanted for my theme that really limited my choices. The first thing is that I wanted was a fluid layout that would resize the content to fit the viewer’s browser window. My monitor runs at a resolution of 1680×1050 and I get really annoyed at websites that run at a fixed width of around 800 pixels – it takes up less than half the width of my screen!  Another thing that I wanted was a horizontal menu just above the main content.  Finally, I wanted a theme that would display categories and tags for each post (surprisingly, not all themes do this).  Fortunately, I found one theme I liked that had all three.

WordPress makes it really easy to search for themes and install them.  From the Appearance menu, go to Themes.  There you will see the currently active theme along with a list of inactive installed themes.  You can activate, preview, or delete any of the inactive themes.  To install more themes, click on “Install Themes” at the upper right corner of the screen.  This will give you an interface where you can search for new themes.  I found the search functionality to be a bit clunky (you only get one page of “Recently Updated” themes) and the Feature Filter didn’t seem to work quite right.  I had better luck searching for themes on WordPress.org.  I would search there to find a theme that I liked and then enter the theme name into the search box on the “Install Themes” page of my website.  From there, it’s a single mouse click to install the theme.  Once it’s installed you can then activate it from the Appearance->Themes page.

wordpress theme managementInstalling themes is easy – finding one you like is another matter