unique hazards may exist
a blog about startups and other unexpected things.

Well, here we go. My startup (BlockChalk) is setting up shop in San Francisco. That means I will soon be spending a lot of time on Caltrain. And that means I need a 3G wireless data service, stat.

But which provider to use? Surely here in Silicon Valley I’m not the first Caltrain commuter to ponder this question. Or am I? I searched around for relevant comparison data and personal stories, and to my surprise didn’t find much. So I decided to run my own comparison test and publish the results for make great benefit my commuter rail brothers and sisters.

The contestants

I went out and purchased a Sprint U301 and a Verizon MiFi 2200, each with an accompanying 3G data plan. Given my horrific experiences with iPhone data service in the Bay Area, I frankly didn’t even bother testing AT&T’s 3G data service.

The test

Over the span of two days I tested each device (and thus, each network) on a northbound local train (stopping at every single station) from San Jose (Diridon) to San Francisco (4th and King). On Day 1 I tested Sprint. On Day 2 I tested Verizon.

At each station I ran the following command from my MacBook Air’s terminal window:
ping -c 20 servername.com
…where “servername.com” is a public UNIX host that I control. This test thus measured minimum, average, and maximum latency across a series of consecutive pings. I am aware that there are a multitude of other (and probably better) ways to measure the performance and availability of a data connection. But this is the approach I took, mostly out of laziness.

Due to operator error there were a few stations where I failed to collect data for one of the two networks (I was probably distracted by lolcats). In such cases, I have left that station out of the final results entirely.

The numbers

The following graph shows average latency for each station. It tells the story pretty clearly. Verizon’s latency was relatively low for the entire trip from San Jose to San Francisco. In comparison, Sprint’s latency was dramatically higher and on numerous occasions jumped into territory that was well beyond usable levels.

Notice that the Sprint results shoot off the top of the graph near the end, never to return. This is because Sprint’s service completely gave out as we entered the final series of tunnels that make up the approach to San Francisco. It went completely dead. Verizon kept on chugging.

This next graph is interesting as well. It shows the standard deviation for each test. From this you can see that Sprint’s service was inconsistent even within the span of 20 consecutive pings (i.e. just a few seconds).

The experience

From a user perspective, Verizon was also the winner. The service was fast, consistent, and highly usable. Aside from running my ping test at each station, I spent the rest of the trip basically forgetting that I was using a wireless data service. It was seamless.

Sprint was, unfortunately, a different story. The service kept coming in and out, speeding up and slowing down. One moment I was surfing smoothly; the next moment I was realizing that Gmail was frozen. I never felt confident that I could predict what was going to happen next. And trying to use SSH was nearly impossible — at times I experienced 3-4 seconds typing delays.

The conclusion

My test was far from scientific, and I would love to see someone more knowledgeable come along and make it obsolete. But for me, it was more than sufficient to proclaim a clear winner.

If you ride Caltrain and want wireless data coverage, get Verizon. It pwns.

Postscript: Sprint is planing to roll-out their WiMAX-based “4G” service in the Bay Area this summer. It will be interesting to see how that system performs in comparison to the current 3G service.

blog comments powered by Disqus