Namespaces in JavaScript

Namespaces are one honking great idea. For years I've used namespaces in JavaScript, and I keep returning to my go-to script for handling this. It's pretty straightforward, and a quick search will return you dozens of alternatives. Regardless, here you go, enjoy:

https://gist.github.com/1440502

Basic LTI Consumer in PHP

A recent small project at work called for using WordPress as a Basic LTI Tool Consumer. Basic LTI is a dirt-simple, and quite handy, way to integrate web applications and is especially popular in the education industry.

We ended up creating a simple WordPress plugin. The plugin has some proprietary bits, but I've extracted the core logic into a gist for your enjoyment. Hopefully this is helpful for you PHP hackers out there, or anyone else needing a simple example of how Basic LTI works.

https://gist.github.com/1171921

Multitasking: a commitment to avoid that which is most important

Multitasking, by definition, removes attention from that which is most important. In a fantasy world of fixed tasks and infinite schedule, this works great. Fortunately for me, I live in the real world of priorities and deadlines.

A simplistic, but useful model:

1) Determine the most important task
2) Focus on the most important task
3) Complete the most important task
4) Go to step 1

Almost all interruptions will force a context switch, and all context switches force an implicit and natural restart at step #1: "Where was I at on that task again? Hrmm, oh yeah..."

Whenever I have problems with the above model, it's usually for a very small number of reasons:

The task was too large
I was unable to maintain focus for a sufficient period of time to complete the task. Natural interrupts, like sleep, result in context switches. This is usually not all that bad. Stopping for dinner, or for the night, gives a chance for my brain to rewire itself around the problem. The problem occurs when I restart at step #1 and determine that the most important task has changed.

The task was too complex
I was unable to keep the entire problem in my head at the same time. This causes internal multitasking and context switches within the scope of the "task". Solution: Keep tasks small, and try not to work on a problem I can't fit in my head.

External interruptions
Solution: Turn off email, phones, and IM for a while and focus on the task at hand. Let people know I'm focusing on the priority items, and for the next few hours, they are not my priority.

Whenever I hear someone brag about their multitasking skills, this is what I usually think:
- They apparently have no problem avoiding their high-priority tasks
- Or, they have no high priority tasks
- Or, they have a lot of menial tasks with low overhead/context switching costs
- Or, they're just full of shit

No Cucumber for Python?

Edit: ylluminate posted a comment about freshen, which seems to be exactly what I'm looking for.

--

It would seem that nobody in the Python world is much interested in Cucumber. I've given thought many times to this and wondered, how hard is it, really, to implement Cucumber in Python?

Tonight I made an attempt to create a reasonably Pythonic way of writing test steps. Here is what I have so far.

First, here are a couple example steps:

@step(r'Given I set variable "(.*)" to "(.*)"')
def set_var(args):
  context[args[0]] = args[1]

@step(r'When I request "(.*)"')
def urlopen(args):
  context['response'] = urllib.urlopen(args[0])

And the 'step' decorator:

def step(pattern):
  def outer(f):
    @wraps(f)
    def wrapper(statement):
      match = re.match(pattern, statement)
      print pattern
      if match:
        f(match.groups())
    steps.append(wrapper)
    return wrapper
  return outer

I don't have a test runner yet, but here's a complete example for invoking the steps:

import re, urllib
from functools import wraps

steps = []
context = {}

def step(pattern):
  def outer(f):
    @wraps(f)
    def wrapper(statement):
      match = re.match(pattern, statement)
      print pattern
      if match:
        f(match.groups())
    steps.append(wrapper)
    return wrapper
  return outer

@step(r'Given I set variable "(.*)" to "(.*)"')
def set_var(args):
  context[args[0]] = args[1]

@step(r'When I request "(.*)"')
def urlopen(args):
  context['response'] = urllib.urlopen(args[0])
  
def go(statement):
  for func in steps:
    func(statement)
  
if __name__ == '__main__':
  print context
  go('Given I set variable "x" to "17"')
  go('When I request "http://www.google.com"')
  print context

Storing data on the filesystem

Relational databases are passé. NoSQL is where it's at.

Right??

Sometimes both are overkill, and we're not doing anyone a favor by over-architecting a solution. Providing a bigger solution than is needed wastes time and money, increases complexity and maintenance costs, and more importantly, doesn't provide any extra value to our users.

The de facto approach for application data storage is to use a dedicated database product, for (mostly) good reasons. However, since we're all well aware of the benefits of using a database, let's take some time to explore the filesystem as a candidate for storing your data.

File writes are atomic

To be more precise, file rename operations are atomic on POSIX systems, according to the Python documentation. Sorry Windows users, you're out of luck.

os.rename(src, dst)

Rename the file or directory src to dst... If successful, the renaming will be an atomic operation (this is a POSIX requirement). On Windows, if dst already exists,OSError will be raised even if it is a file; there may be no way to implement an atomic rename when dst names an existing file.

To perform atomic file writes, you must first write your changes to a temporary file, then rename the temporary file to it's final destination. Sounds harder than it really is. The code would look something like this:

import os

f = open('temp.txt', 'w')

f.write('do the monkey')

f.close()

os.rename('temp.txt', 'final.txt')

The filesystem is reliable

You'd better hope so anyway, everything ultimately lives on the filesystem, including, yes, that fancy and expensive relational database.

Backups

Storing data in files allow you to use standard filesystem-based backup solutions. In addition, many filesystems have snapshot features built in.

Instant API using a web server, or even WebDAV

I suggest storing your data in a document-oriented fashion. That is, store your data using a single file per entity. If you're storing data about 6 different users, then that should be 6 different files. This will greatly simplify things, and allow you to expose this data via an HTTP API.

If you follow this advice, you can simply point your favorite web server at your filesystem and you immediately have an API. Requesting data from this API couldn't be simpler, and may look like this:

http://myserver.com/api/user1.xml

Enable more features on your webserver, such as PUT, or even WebDAV, and you now have a read+write API.

The filesystem scales (probably)

I currently have 454,823 files on my computer consuming ~140GB. I don't know if there is a practical limit to filesystem storage, but I'm willing to bet that you and I aren't going to reach it.

Files work with the network

See above note about APIs. Or, see: NFS

Everybody's doing it

Subversion does it. Oracle does it.

--

The relational databases and NoSQL data stores will still be there, waiting for you if you need them in the future. My advice? Ignore your DBA. Drop acid and think about data.

Be careful what you measure, you just might get it

When attempting to measure the performance, or productivity, of a team, you need to use great care when choosing your metrics. By choosing a metric, you are creating an implicit incentive for the team to optimize themselves against that metric.

Incentives can be a useful tool for a leader, the trick is to understand which incentives are effective, and which incentives your team is optimizing for.

A classic problem, especially in software development teams, is that leaders and managers unknowingly create incentives that are counterproductive. A few examples to watch out for:

Test coverage

If you expect 100% test coverage, then there is a good chance your team will provide you with just that. The problem is there is a world of difference between a test, and a good test. Would you rather have 50% coverage using high quality tests? Or 100% coverage with mediocre or tests?

Lines of Code

Lines of code is often used as a measure of productivity. There are numerous problems with this approach, and by no means am I the first to make these observations.

First, a claim: in any given software system, there is a correlation between lines of code and number of defects. That is, the greater the line count, the more bugs you have. Why would you want to incentivize your team to create more defects? If anything, you should incentivize your team to decrease the rate at which code is added, or even to decrease the total line count in your application.

However, correlation does not imply causation, and not all lines of code are created equal, so any metric based on lines of code is fraught with risk. Incentivizing the net negative production of code can lead to widespread use of clever tricks, obscure hacks, and generally unreadable and unmaintainable code.

Using work estimates to measure delivered value

You are estimating your work, right? Many teams estimate future work in hours, or days. Agile teams often have a concept of story points (a story point is simply a unit of effort used to estimate future work). Estimates allow for planning (aka guessing), hiring, organizing, and any number of useful activities. However, an estimate is just that, an estimate, and does not capture the total effort invested to actually deliver a given feature, and does not capture the value that was actually created.

I'll argue that there exists a relationship between effort and value, but that it is impossible to derive one from the other. Effort results from things like expertise and complexity, not from the delivered value. Anyone who has toiled away, spending long hours on a useless deliverable can attest to this. Likewise, value is determined by things such as how much revenue can be generated, or how happy you make your users. The relationship exists between value and effort because not everything is worth the cost required to attain it.

So perhaps it is just plain wrong to use effort metrics to measure delivered value. Beyond that, you may be implicitly incentivizing the inflation of estimates, decreasing their usefulness. Even worse, you may be disincentivizing the delivery of actual value!

--

Software development is hard, and I have a tremendous amount of respect for teams that are successful and make the world better place. Anything you've read here which suggests that I'm trivializing the hard work, dedication, and focus it takes to create software products was not intentional. Just the opposite actually; I'm saying software development is probably harder than you think it is.

Can you think of any other implicit incentives? Please share in the comments section.

The success of a software product...

I don't want to believe this, but I've seen too much evidence over the years that suggest it is the truth:

The success of a software product is inversely proportional to the quality of its codebase.

Therefore, if you want to develop a successful software product, just do it. Don't worry about doing it right.

Creating persistent SSH tunnels in Windows using autossh

1. Download Cygwin (http://www.cygwin.com/)
2. Install Cygwin, selecting the autossh package.
3. Start the Cygwin shell (Start -> Programs -> Cygwin).
4. Generate a public/private key pair.
1. At the command line, run: ssh-keygen
2. Accept the default file locations
3. Use an empty passphrase
5. Copy your newly-created public key to the SSH server.
1. scp .ssh/id_rsa.pub user@ssh.host.name:id_rsa.pub
6. Add your public key to your list of authorized keys on the server.
1. Login to your SSH server.
2. mkdir .ssh
3. cat id_rsa.pub >> .ssh/authorized_keys
7. Test your key.
1. Logout of your SSH sever.
2. Login to your SSH server again. This time, your key will be used for authentication and you won't be challenged for your login credentials. If you are not logged in automatically, review the previous steps. Or contact your server administrator.
   
3. Logout of your SSH server.
4. Exit of the Cygwin shell.
   
8. Install autossh as a Windows service.
1. Now back in Windows, open a new command Window (Start -> Run -> cmd).
   
2. cd C:\cygwin\bin
   
3. cygrunsrv -I AutoSSH -p /usr/bin/autossh -a "-M 20000 -L localaddress:port:serveraddress:port user@ssh.host.name" -e AUTOSSH_NTSERVICE=yes
9. Tweak Windows service settings.
1. Open the Services management console (Administrative Tools -> Services).
2. Edit the properties of the AutoSSH service.
3. In the "Log On" tab, select the "This account" radio button and set the service to run as your current user.
4. Start the service.
10. Test your tunnels.
    

Subversion post-commit email script via Powershell

I thought it'd be neat to keep tabs on all commits made to a certain Subversion repository I participate in, so I whipped up the following script in Powershell. Yeah, there are lots of post-commit scripts floating around, but they're primarily for non-Windows environments. Besides, it was easy and gave me an excuse to play with Powershell.

To use:

1. Make sure you've installed Powershell (duh!).
2. Copy the contents below to a file, say, c:\emailer.ps1
3. Be sure to customize the $server, $to, and $from variables.
   
4. Create a post-commit.bat file in your repository's "hooks" folder.

Put this in your post-commit.bat file:

%SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe c:\emailer.ps1 "%1" "%2"

And here is the Powershell script itself:

# Sends an email message to the configured addresses, notifying
# of changes to the current repository
#

# customize the following
#

$server = "mail.server.ip.address.yourdomain.com"
$to = "youremailaddress@yourdomain.com"
$from = "fromemailaddress@yourdomain.com"

#
#

$REPOS = $args[0]
$REV = $args[1]
$name = $args[2]

if ($name -eq $null) { $name = $REPOS }

$author = svnlook author $REPOS --revision $REV
$subject = "SVN Commit: ($author) $name ($REV)"

$body = ""

foreach($log in svnlook log $REPOS --revision $REV) { if($log -ne "") { $body += $log + "`n" } }

if ($body -ne "") { $body += "`n`n" }

foreach($changed in svnlook changed $REPOS --revision $REV) { $body += $changed + "`n" }

$mailer = new-object Net.Mail.SMTPclient($server)
$msg = new-object Net.Mail.MailMessage($from,$to,$subject,$body)
$mailer.send($msg)

Indy Ducati and the GT1000

Over the summer a new Ducati dealer opened in Indianapolis, Indy Ducati. I was elated, because another bike at the top of my list was the Ducati GT1000. I like the style and that characteristic Ducati engine. I'd been absolutely drooling over the photos I'd seen, particularly this one:



Pulling into the parking lot we smell racing fuel and hear a bike on the dyno. Fantastic! This is my kind of place, I'm thinking.

And there it is, a new GT1000, in red. Also, a demo in that gray-ish, green-ish color, with uncorked Termignoni pipes. Winter is approaching and they're looking to offload the well-used but fully-warranted and smartly-tuned demo, great time to make a deal. So I take it for a test ride...

The sound from the Termignoni pipes is simply glorious. This isn't that lazy, obnoxious Harley sound... it's tight, energetic, racy, but still loud and deep. If you haven't already, add this to your to-do list: ride an uncorked Ducati at full tilt, you won't be disappointed.

I was almost sold just on the sound alone, and the rest of the bike didn't disappoint.

In the end I decided to pass on the Ducati and went with the Bonneville. Honestly I'd be happy with either. Tough life to have to make these choices, eh?