At the Buzzer

December 31st, 2015

I find myself again at the end of the year, realizing that I’m a few hours away from getting to the end of 2015 without a blog post. Keeping the streak alive has become a sad tradition.

2015 was, in my defense, a jam-packed year. By far the biggest change was in parenting, as we moved from man-to-man to zone defense. Having three kids is awesome, and exhausting, and has somehow sent us over a laundry-management threshold.

2016 will, by all indications, be a big year as well, but for very different reasons. I look forward to sharing why sometime in the coming weeks.

Happy New Year!

Writing in the New Year

December 31st, 2014

I’ve been in the process of overhauling my blog for about a year now. As you can tell by the lack of progress, it’s been slow going. So it goes when you have a job, two kids, and one on the way. Those “extra hours” don’t exist anymore, and any spare moments prior to sleep aren’t exactly full of brain- or will-power.

Squeezing in before the New Year
This post is coming at the last day of December because I couldn’t let my 15-year streak of posting come to an end. (I’ve had at least one post a year since 1999!) As I’ve been working on the overhaul, I saw the last post and felt confident that I’d posted in May, but just realized a few weeks ago that it was May *2013*. Sigh.

Conversion to
For a while, I’ve been eyeing blog platforms that generate from static files. This seems like a bit of a regression (since the first version of my website was essentially static files), but these platforms are a reaction to the hack-friendly bloatware that modern blogging platforms have become. (This all comes from trying to be the ultimate platform for creating any website under the sun. I’m looking at you, WordPress.)

My short-term rationale is that I want something stripped down to its barest elements, where even comments are add-on (with Disqus, most likely). Long-term, I want to continue to own my web content. Thanks to Aaron VanDerlip for the pointer to the Indie Web movement. It’s still taking shape, but I’m on board with their principles.

Being a Python guy, I looked into Pelican, but wanted to start someplace even simpler, something that would allow me to focus on writing. Since I use (read: can’t live without) Evernote, and also use Dropbox, seemed like a logical fit. They’ve built a really nice platform. It has had some downtime issues, and the re-indexing isn’t quite as rapid as I’d like, but it’s hard to argue with cheap. I’m viewing this as a trial-run of static blogging.

I’m doing my writing these days in Byword, which is a fantastic lightweight editor with built-in Markdown support. My original planned workflow was Byword=>Evernote=>, but:

  1. Byword seems to like to publish to Evernote instead of sync to it;
  2. Evernote doesn’t preview Markdown (web client has a plugin, but I like the Mac client);
  3. has a funky interpretation of Markdown via Evernote (too many linebreaks);
  4. I don’t feel like using Evernote’s WYSIWYG markup (not enough precise control over web rendering); and
  5. Though I love Evernote, I can more easily create non-cloud backups of Dropbox folders.

So instead, I’m doing Byword=>Dropbox=> The big hurdle is going to be converting all by existing blog posts to individual static files. Python to the rescue!

Goals for 2015
My main goal for 2015 is a simple one: write more frequently. I don’t expect much of an audience, which is good since I’ll be spending the first few months scrubbing the rust off. But if I can get into a routine, my posts should increase in quality, and then I’ll feel more comfortable sharing on Twitter, etc.

About 5 years ago, as we were building GHDonline, we made a few design decisions that have not stood up well to the test of time. These mistakes were due in part to our relative inexperience with Python and in part to the newness of Django (we were using a fork of the trunk because 0.96 didn’t have some of the features we needed and 1.0 was under development).

A few of these decisions were driven solely by premature optimization, which is rarely a good idea. I’ll illustrate with a small example.

The “Problem”

    in which we thought we knew better than the Django core developers

Django’s ORM contains an EmailField type, which for the purposes of this story is used in (django.contrib.auth). The EmailField is essentially a varchar, but it does some validation (e.g. is it of the form name@domain.ext ?) on save. The default length for this field is 75 characters, which we decided was far too short, since the standard put forth in RFC 2821 specifies: Size limits and minimums

There are several objects that have required minimum/maximum sizes.
Every implementation MUST be able to receive objects of at least
these sizes. Objects larger than these sizes SHOULD be avoided when
possible. However, some Internet mail constructs such as encoded
X.400 addresses [16] will often require larger objects: clients MAY
attempt to transmit these, but MUST be prepared for a server to
reject them if they cannot be handled by it. To the maximum extent
possible, implementation techniques which impose no limits on the
length of these objects should be used.

The maximum total length of a user name or other local-part is 64

The maximum total length of a domain name or number is 255

Django provides a means to increase the max-length of EmailField:

address = models.EmailField(max_length=320)

We wanted to extend the length of EmailField, and we also wanted to allow for multiple email addresses per user, so we ended up creating our own Email model and stopped populating the field. Though it occasionally required some code gymnastics, this approach worked pretty well for us.

The Attempted Upgrade

    which should have been much simpler than we made it

Fast-forward 5 years to this week, and we’re looking to upgrade our old fork of django-registration, a very commonly-used package that handles user signups on the web and account activations via email. A new contender, django-allauth, has emerged; it handles the same functionality as django-registration, provides more out of the box (like basic templates), and integrates social authentication.

Email addresses in django-allauth are stored both in (as before, a single email address per user) and their own Email model, which allows for multiple addresses per user and looks a heck of a lot like our own.

Unlike us, they’re actually using the field, so we had to find a way to extend that field to 320 characters. Turns out, there’s a hack that does this via monkey-patching.

Alternatively, we could have built our own User model, but that would have required an upgrade to Django 1.5, which introduces too much risk for this project (we have a pretty complex system; we’re 3-4 months out from upgrading).

This solution seemed to work, so I kept developing… until I realized that django-allauth’s Email model also uses the default length of 75 for its EmailField, as well. The earlier hack felt slightly wrong, but it seemed to be a pretty common approach, so we went with it. However, when we hit this second case, I realized that we might be doing something wrong.

The Realization

    that the Django core devs knew what they were doing

Then it hit me: perhaps our assumption that the standard should be followed was wrong (or was at least a premature optimization). Was 75 characters actually enough?

We have five years of user data, so I created a histogram of lengths of email addresses from our current userbase. The wonderful collections.Counter helped immensely:

>>> from collections import Counter
>>> lengths = [len(e.address) for e in Email.objects.all()]
>>> c = Counter(lengths)

In case you’re curious, here are the results:

>>> print(json.dumps(c, indent=4))
"10": 8,
"11": 9,
"12": 90,
"13": 109,
"14": 196,
"15": 292,
"16": 577,
"17": 695,
"18": 845,
"19": 905,
"20": 956,
"21": 882,
"22": 888,
"23": 810,
"24": 679,
"25": 579,
"26": 517,
"27": 422,
"28": 357,
"29": 324,
"30": 305,
"31": 263,
"32": 213,
"33": 158,
"34": 99,
"35": 62,
"36": 47,
"37": 25,
"38": 21,
"39": 10,
"40": 6,
"41": 2,
"42": 5,
"43": 3,
"44": 3

So while ideally, the length should be set based on RFC 2821, it turns out that after five years of gathering members and their email addresses, we haven’t once come close to even 50 characters, let alone 75 or 320.

The Lessons

    in which I pedantically state something you already know

1. Always know what assumptions you’re making, and be proactive about questioning them.
2. Before you build something to solve a potential problem, figure out how likely that problem is to occur.

Jack is Back

May 13th, 2013

One by one, the shows I watch are ending or have ended: 30 Rock, The Office, House, Simpsons (wait, that’s still on?!), Futurama, Star Wars: The Clone Wars.

I’m not complaining (except in a few cases), since it leaves me with more healthy non-TV time… but now one of my all-time favorites, Community, was given extra life, Arrested Development is back on May 24th, and now this??

Just when I thought I was out…

About a year and half ago, a colleague recommended the book “The Shallows” to me, but my reading queue is so backlogged that I haven’t gotten to it. Honestly, this backlog is due in part to the percentage of time I spend each day consuming information via the Internet, favoring that over books and long-form papers…

…which is ironic.

So here’s a version of the message of the book, complete with quotes from the author:

I recognize that there is a lot of bite-sized-information-hating going around these days, and the knee-jerk reaction here could be the Luddite “Down with Twitter!” / “Throw away your smartphone!” / “Move to the Northwest Territories!”, but I believe there has to be a middle ground. These new streams of information bring to us new perspectives, new ideas, and with the right tools, the ability to curate the information.

So how can we give ourselves the space and time to do higher-order thinking, allowing our minds to process the corpus of new ideas and inspiration we’ve accumulated from Internet-based information streams?

The Boston Spirit

April 17th, 2013

In the moments following the attack, as I moved my family as far away from the marathon route as I could, this exact phrase kept going through my head: “they messed with the wrong city”… “they messed with the wrong city”. It’s not about revenge. It’s about our spirit.

Dennis Lehane describes it using better prose than I can muster. If you read something, read this:

I’m writing a blog post series (one; two; more to come over the next few weeks) for work about our love / hate relationship with email. There’s an experiment I have been running that wasn’t appropriate to go into on the GHD blog, but I wanted to write about it: unsyncing email from my smartphone.

I’ve been giving this a try, and I’ve made it almost five months now. “But why?!” you ask. “What’s the point of having a smartphone at all if you’re not going to use it to check your email?” Well, there are plenty of other things I can do on the phone to consume mobile bandwidth and keep myself occupied. The issue that I was having was with my own behavior. Prior to this experiment, in having my email linked to my phone–which I’ve had enabled since I first got the Droid 1 in 2009–I noticed the following effects on my behavior:

  • Compulsively checking my email account in every spare moment, often without thinking about it.
  • Opening an email and being unable to properly respond to it, either due to a lack of time to fully think through a response or the inconvenience of typing out a long reply on a phone.
  • Being stressed out by incoming email at a time where I was unable or ill-equipped to deal with the issue within the email. Filing the email away, both physically and mentally, didn’t remove the stress but rather pushed it out to my subconscious, where it festered. Imagine I’m in the middle of doing something enjoyable — now the experience has been ruined.
  • Spending more time in a “reactive” mode, waiting for emails to come in versus doing real work. I don’t know the source, but a while ago I read and have co-opted a phrase: “No one ever got a promotion or raise based on the speed of their response to emails.” Of course, if you’re in sales or you’re a CEO, this might not apply to you, but the principle holds: be proactive and do the work that generates the most value. Emails add minimal value at best, and I’d argue they often decrease value because they distract you from what’s important.
  • Being *less* responsive to email because the phone encouraged me to push off until later (when I was at a computer) longer responses. This habit carried over to email on the computer as well, and emails went a long time without a response (which increases stress).

On the flip-side, the supposed benefits–being more connected and responsive–are minimal, and there’s really only a few hours a day that I’m not on or near a computer. Very rarely does someone need an answer *right that minute*, and if they do, they’re likely someone who knows me well enough to text or call.

Honestly, though, I didn’t notice many of these until I started to reflect on my stress level, my phone use, and my lack of responsiveness to emails. At first, I thought the solution was to schedule email syncing for certain hours (not straightforward to do on the Android Gmail client, BTW). However, I was spurred on by this fantastic, reflective post by Harj Taggar of YC, and decided to give complete disconnection a try. With a few lapses, it’s been nearly five months now, and here are my observations:

  • It took me a solid two weeks to undo the muscle memory of checking the email accounts.
  • Complete removal of the accounts from the phone doesn’t work for me, because I still want my contacts and calendars. Unfortunately, though I can un-sync Gmail on Android, I can’t remove the accounts from the list in the Gmail app, so I still have a stale view of my account at the time I un-synced. Not a huge issue, but I’d rather not even see the account.
  • I’m finally starting to reclaim my inbox and get back to being a better emailer, partially because I’m more focused when I sit down to hash through my inbox.

Un-syncing email from the devices we have next to us 24-7 is just one component of dealing with the issues of email, but it’s a big step.

(I also removed Facebook from my phone, but that’s worthy of a separate blog post.)

Rob Pike – ‘Concurrency Is Not Parallelism’ from Waza on Vimeo.

Rob (@rob_pike) is a software pioneer. His influence is everywhere: Unix, Plan 9 OS, The Unix Programming Environment book, UTF-8, and most recently the Go programming language.

At GHD, we’ve been hosting our newer code on github (most of which is private, but I’m working this year on making it public). It’s mostly worked well, but I noticed a while ago that I stopped receiving email notifications for commits (ok, technically “pushes”, but I’m still working on my conversion from Subversion).

Now I’m not a huge fan of email (in fact, I agree with Paul Graham that email is ripe for an overhaul), but I do like my commit notifications to show up in my inbox.

Turns out it was related to the new organization we created and my role as an owner of that organization, as Alex King explains here. Unforunately, github doesn’t make this clear on either their Notifications page or Teams page.

[Why wouldn’t owners want email notifications? We don’t spend all day staring at our github dashboard, but we do have email accounts…]