17 of 26

So, it’s done – the initial release of the eHow Earnings Tracker is available from the Mozilla Addons site.  Head on over to my downloads page for more details on obtaining the addon.  The tracker is currently classified as “experimental” which is just Mozilla’s way of saying that they haven’t reviewed it and placed their stamp of approval on it.  Apparently there are over 400 addons in the review queue so that won’t be happening any time soon .

I must say that overall, it was a very smooth process to get my addon hosted by Mozilla.  I just had to create an account, fill out a few fields, and upload the addon.  The hardest part was deciding which software license I wanted to release my code  under.  The legalese was overwhelming and I just picked the simplified BSD license.  I have no idea if it was the right choice or not.

The tracker is currently pretty barebones, but it is functional from the limited testing I did with it.  A few fellow eHow writers (Carol McKenzie, Dianne Cass, and knyc) were kind enough to volunteer their services in testing it out for a few days before I start broadcasting its availability to the eHow community.

What it Does

As I originally mentioned, eHow gives writers very limited earnings data.  Currently there is no way to writers to figure out which articles get income on a day to day basis without doing a whole lot of manual work.  The tracker collects data each time that it is run and keeps a history so that writers can see earnings increase over time on a per article basis.  Since earnings updates only occur at most once per day, the tracker keeps data on a per day basis.  This means that if the tracker is run multiple times during a single day, it only keeps the last set of data for that day.

Using the tracker is very easy.  Upon installation, it adds submenus to both the main Tools menu and the context menu.  The user simply needs to navigate to one of their article earnings pages and select the menu option “Update Earnings”.  The tracker will then read all of the earnings pages and generate an HTML report.  The HTML report is currently very barebones and ugly but the plan is to fix that up.

eHow Earnings Tracker context menu

How it Works

The tracker works by first examining the URL of the web page that’s currently in the browser.  eHow has very specific URL formats for the earnings pages so the tracker can figure out whether or not it can be run.

Once it is run, the tracker examines the HTML of the page and extracts data based on HTML elements and CSS class names.  It can also figure out how to navigate to the other earnings pages by using the “Next” anchor link at the bottom of the page and gathers information off each page as it loads.

HTML structure of the eHow article earnings pages

The tracker stores its data in an XML file in the Firefox profiles directory.  If you’re curious, you can track it down by looking in the same directory as the URL that appears in the HTML report after the tracker is run.  The file is called eHowEarningsTracker_17of26_earnings_[ehow user name].xml.

Caveats

The tracker requires Firefox 3.5 because I needed to use some functionality that only became available in 3.5.

The tracker can only collect data when it is explicitly run.  Since it can only read what’s in the rendered HTML pages, it can’t know anything that the user doesn’t see.  This means that if the tracker will never have data for days that it wasn’t run.

This tool is completely tied to eHow’s HTML code.  If eHow does anything to change the HTML layout of their earnings pages then the tracker will not be able to successfully read any more data until I update it to handle the new HTML layout.  However, all of the past collected data would still be there.

I have no idea how this thing is going to handle large amounts of data.  Some people have hundreds of articles and tracking earnings data over weeks and months is going to get big.

Upcoming Changes

There’s really only two things I need to work on at this point.  The first one is to do some serious work on the HTML report.  It needs some nice styling and I really want to add some functionality to it (such as being able to sort by column).

The other thing that I want to do is to provide a way to export/backup the data in CSV or XML format.

One of the major challenges with doing software development is that you need to be constantly evaluating the direction that you’re heading in with both the code and the end product.  You can’t just decide to do something and then put your head down and do it.  The landscape changes as you develop a product – you learn better ways to do things, roadblocks come up, and your vision of the end product (and code) becomes more refined and  clear.  If you don’t pay attention to these things as you go along then inevitably you’ll wind up miles away from where you should be in the end.

On the other hand, you can’t be changing direction with the wind either.  This is a trap that many programmers fall into.  If you don’t have a solid starting vision for the product or you constantly change the requirements for the product or how it is implemented then you’ll wind up with a big mess or, even worse, the product will never get finished.

The balancing act of changing things that need to be changed and not changing things too often is one of the things that makes software development as much like an art or craft as an engineering or science.  Complicating things further is the fact that what’s best to do for the product often conflicts with what’s easiest to do in the code.

I was working on the eHow Earnings Tracker last night when I had one of those “aha!” moments where I knew that I needed to change the direction I was heading in.

My original vision for the tracker was that it would automatically detect eHow earnings pages as the user browsed the web.  When an earnings page was detected, the tracker would read the current page of earnings, save them to a file, and compare them to the previous update.  The user would be able to see which articles had increases in earnings from update to update and they would have a history of earnings that could be exported.  This  seemed like it would be fairly straightforward to implement and would be a low friction solution for the user.

As I worked on the code, a number of implementation difficulties arose and it became apparent that there would be some usability issues with the end product.

Implementation difficulties (code):

1. Firefox can have multiple tabs open at once which means that, in theory, multiple earnings pages could be loading at the same time.  Writing code to handle this sort of scenario is very difficult in a Javascript based environment.
2. Due to the fact that the code is only parsing one page of articles at a time and the article order is not guaranteed, the code can’t just blindly read & write the entire contents of the current earnings file stored on disk.  Each time an earnings page is viewed, the file needs to be read in and then its data needs to be compared to the current page of earnings to see whether article entries need to be added or modified.
3. The code would be executing every time an earnings page was viewed, even if it didn’t have to.

Usability issues:

1. The user would have to manually click through each page of articles (which can get tedious since articles are only displayed 10 at a time).
2. Since the code was only seeing one page of articles at a time, it would have no way to inform the users of any articles that were added or removed since the last update.

My “aha!” moment was in realizing that the better way to do this would be give the user an “Update Earnings” button that could be activated whenever they were on their earnings page.  That button will trigger the tool to automatically navigate through all of the users’ earnings pages collecting information.  At the end of the process, an HTML page can be displayed showing a summary of all the earnings data.  This fixes all of the implementation and usability issues in one shot.  It’s a rare case where I can simplify the code and improve the product at the same time.

Since I now have a Firefox plugin that will properly update the browser’s status bar with the URL of the web page being viewed, it’s time to write code that will specifically detect an eHow earnings page.  For those of you following along at home, take the code from the previous post and add the following function just after the updateStatusBar function :

var update = function()
{
    var host = Application.activeWindow.activeTab.uri.host;
    var path = Application.activeWindow.activeTab.uri.path;

    m_eHowMsg = ""

    if(host.search(/www.ehow.com/i) == 0)     
    {         
        // eHow earnings URLs are in the form "http://www.ehow.com/members/user-p#-articles.html" or  "http://www.ehow.com/members/user-articles.html"         
        var userAndPageNumPattern = new RegExp("\/members\/\(.+)-(.+)-articles.html");         
        var userAndPageNumResult = userAndPageNumPattern.exec(path);          
        var userPattern = new RegExp("\/members\/\(.+)-articles.html");         
        var userResult = userPattern.exec(path);          

        if(userAndPageNumResult && (userAndPageNumResult.length < 1))         
        {             
            m_eHowMsg = userAndPageNumResult[1];         
        }
        else if(userResult && (userResult.length &lt; 1))         
        {             
            m_eHowMsg = userResult[1];         
        }     
    }      
    updateStatusBar(m_eHowMsg);
}

This function looks for URLs that match eHow earnings pages and then updates the status bar with the eHow user name that it finds.  To hook this function up, just replace the calls to updateStatusBar in onTabSelect() and onTabLoad() with calls to update().

Now when we navigate to a URL that contains eHow article earnings data we see the username in the status bar (the status bar will be blank for other URLs):

The code from my previous post worked great with a single tab open in Firefox.  However, using multiple tabs exposed a problem – all tabs in a single Firefox window share the same status bar.  Another issue is that the event “DOMContentLoaded” gets fired any time a page finishes loading, regardless of what tab you’re on.  This means that the information in the status bar may or may not actually match the tab that you’re on.

I also found some better techniques for placing my code in a namespace/module that are described here and here.  It’s also worth noting that Mozilla’s Developer Center has documentation for their Javascript APIs and objects.  Unfortunately, the documentation usually isn’t very good and I wind up spending an awful lot of time looking for simple pieces of information.

After a bit of work, I came up with the code below to do what I wanted.  This code only works for Firefox 3.5 because that’s when event data was added for the TabOpen, TabClose, and TabSelect events.  In the code below, event.data is a BrowserTab object.

// Create a global variable for the addon
var g17of26 = {};

g17of26.eHowEarningsTracker = function ()
{
    // Private functions
    var updateStatusBar = function(msg)
    {
        var statusBar = window.document.getElementById("ehow_earnings_tracker_statusbar");

        if(statusBar)
        {
            statusBar.setAttribute("label", msg);
        }
    }

    return {  // This brace must stay on the same line as the return - it's a Javascript quirk

        // This function is called when the browser is first launched
        onBrowserLoad : function()
        {
            Application.activeWindow.events.addListener("TabOpen", g17of26.eHowEarningsTracker.onTabOpen);    
            Application.activeWindow.events.addListener("TabClose", g17of26.eHowEarningsTracker.onTabClose);
            Application.activeWindow.events.addListener("TabSelect", g17of26.eHowEarningsTracker.onTabSelect);

            Application.activeWindow.activeTab.events.addListener("load", g17of26.eHowEarningsTracker.onTabLoad);
        },

        // This function is called when the browser quits
        onBrowserUnload : function()
        {
            window.removeEventListener("load", g17of26.eHowEarningsTracker.browserLoad, false);
            window.removeEventListener("unload", g17of26.eHowEarningsTracker.browserUnload, false);        

            Application.activeWindow.events.removeListener("TabOpen", g17of26.eHowEarningsTracker.onTabOpen);
            Application.activeWindow.events.removeListener("TabClose", g17of26.eHowEarningsTracker.onTabClose);
            Application.activeWindow.events.removeListener("TabSelect", g17of26.eHowEarningsTracker.onTabSelect);
        },

        // This function is called when a tab is opened
        onTabOpen : function(event)
        {
            event.data.events.addListener("load", g17of26.eHowEarningsTracker.onTabLoad);
        },

        // This function is called when a tab is closed
        onTabClose : function(event)
        {
            event.data.events.removeListener("load", g17of26.eHowEarningsTracker.onTabLoad);
        },

        // This function is called when a tab is selected
        onTabSelect : function(event)
        {
            updateStatusBar(Application.activeWindow.activeTab.uri.spec);
        },

        // This function is called when a page finishes loading in a tab
        onTabLoad : function(event)
        {
            if(event.data.uri.spec == Application.activeWindow.activeTab.uri.spec)
            {
                updateStatusBar(event.data.uri.spec);
            }
        }
    };
}(); // the parens here cause the anonymous function to execute and return

// Hook up events
window.addEventListener("load", g17of26.eHowEarningsTracker.onBrowserLoad, false);
window.addEventListener("unload", g17of26.eHowEarningsTracker.onBrowserUnload, false);

As it turns out, hooking up Javascript to XUL so that an addon can actually do something is remarkably similar to linking HTML and Javascript together when doing web development.  The first step is to create a Javascript file and include it it in the XUL file much like you would include a Javascript file in an HTML file:

<overlay id="ehow_earnings_tracker_overlay" xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul">
    <script type="application/x-javascript" src="chrome://ehow_earnings_tracker/content/browser_overlay.js" />
    <statusbar id="status-bar">
        <statusbarpanel id="ehow_earnings_tracker_statusbar" label="" />
    </statusbar>
</overlay>

Now we can add code in browser_overlay.js to actually do something.  In this case, we will update the status bar text with the URL of HTML page that was loaded:

// Creates a global variable called "ehow_earnings_tracker"
var ehow_earnings_tracker =
{
    init:
        function()
        {
            // Grab the contents of the browser
            var appcontent = document.getElementById("appcontent");
            if(appcontent)
            {
                // Get notified when the page is done loading
                appcontent.addEventListener("DOMContentLoaded", ehow_earnings_tracker.onPageLoad, true);
            }
        },

    onPageLoad:
        function(aEvent)
        {
            var doc = aEvent.originalTarget; // doc is document that triggered "onload" event
            var statusBar = document.getElementById("ehow_earnings_tracker_statusbar");

            if(!statusBar)
            {
                return;
            }

            statusBar.setAttribute("label", doc.location.href);
        }
};

// Hook into the window load event
window.addEventListener("load", ehow_earnings_tracker.init, false)

Now, when I run Firefox with my plugin enabled it dynamically updates the status bar with the URL of the loaded page:

Firefox addons are implemented using a combination of XUL, Mozilla’s proprietary XML based language, and Javascript.  For anyone who’s done development with HTML and Javascript, learning to write a Firefox addon is pretty straightforward.  All you need is a decent tutorial to get started.  Fortunately, finding tutorials on how to create addons for Firefox is pretty easy.  Some of the ones I found were pretty old, but as far as I could tell the information was still accurate. If you need to learn Javascript, there are plenty of tutorials over on W3Schools.  If you prefer books, I highly recommend David Flanagan’s Javascript: The Definitive Guide.

I decided to start with a tutorial from Mozilla which shows you how to create a classic “Hello World” program by displaying “Hello, World” in the browser’s status bar.   I went through the tutorial quickly and easily by following the exact steps that were outlined in the tutorial.  “Hello, World” showed up my the status bar without  a hitch.  The flaw with this tutorial is that it doesn’t show you how to make the addon actually do something, which is the topic of my next post.

Another useful article I found was this article on Lifehacker that gives some good tips for getting set up to develop Firefox addons.

Last fall I was looking for a way to make some additional income and my wife, whose career currently consists of writing online, pointed me toward eHow.  I’ve never been much of a writer but I thought I’d do ok with writing How-To articles about programming and other computer related topics.  I browsed around the site and found that most of the computer related content was just awful.  Almost all of it was poorly written, out of date, or just completely inaccurate.  I thought to myself “I can do better than this!”.

So in October I created an account and wrote a couple of articles about programming with WPF.  I got distracted with other things for a while the two articles earned next to nothing over the next six months.  In late April I decided to give it another shot and I’ve written 25 more articles since then.  I still haven’t earned much of anything, but from reading the eHow forums I’ve concluded that you really need to reach a critical mass of article count and a certain threshold of article age before you really start seeing any sort of real money.  Even just going from 2 articles to 27 over the course of two months I’ve noticed a significant acceleration in the rate that my earnings are increasing.  Of course, creating this blog has completely stalled my eHow article writing .

eHow writers earn money through some secret earnings algorithm that seems to be primarily based on people clicking ads on your articles.  The official eHow line is that the earnings are based on a variety of factors but the primary driver really seems to be  ad clicks.  Fortunately for me, computer related ads seem to do very well in terms of CPC (cost per click – the rate that advertisers pay for ad clicks).

The previous two paragraphs are background info for the topic of this post – tracking eHow earnings is a royal pain in the butt.  eHow gives you very limited data for your earnings.  For each article in your library, you are shown lifetime views and lifetime earnings.  The only other numbers you get are lifetime views for all articles and total earnings for the month.  This means that you have no way to track article views or earnings over time unless you manually record them every day.  To make matters worse, you can only see data for 10 articles at a time.  For a few days I tried tracking the earnings manually using a Firefox addon called TableTools to copy the earnings data and paste it into Excel.  That just sucked and I was determined to create a better way.

My first thought was to create a Windows application that could make HTTP requests to snag the earnings data since it would involve using technology that I was already familiar with.  I opened up Firebug and watched the HTTP traffic as I logged into eHow and viewed my earnings pages.  I immediately realized that trying to decipher all of the traffic to figure out what requests I’d need to make was going to be a total nightmare.

It then dawned on me that this might be a good thing to implement using a Firefox addon.  From my experience using other Firefox addons, I knew that they were capable of doing everything that I would need it to do.  Perfect!  I would get an opportunity to learn how to write Firefox addons, I would be creating something that I would want to use on a regular basis and something that other people would most certainly want to use.  I was sold – off to learn how to write a Firefox addon!