Harper Reed: Technology and death metal

BUSY! busy busy.

2011-10-02T00:00:00-05:00

It has been about nine months since I last posted on this blog. I promise I haven't completely forgotten about it. I am so busy that when I have a quick second to think about posting - I am suddenly thinking about the next thing and the thoughts about my blog have disappeared.

I am busy because of the awesome work I am doing with all the amazing people at Obama for America. Although, I miss writing articles about my coding and traveling, I love (and more importantly believe in) the work we are doing.

I will attempt to write a meaningful post soon (hah. who am I kidding. It will be many moons before I post again!). In the meantime, please feel free to hit me up on twitter (I am @harper), check out the books I am reading and give to my personal fundraising page.

OMG FREE HOSTING: How to use GAE to host static sites for free

2011-01-26T00:00:00-06:00

I am constantly telling my friends about the new technology tricks that I learn in my internet travels. I learn a lot, which has caused my friends to ignore around 90% of what I say about technology. I don't mind, because I know that I am a genius(heh) and they will come around some day.

This is exactly what happened with my most recent hacked upon hosting setup. I told my buddy kanno, and he didn't pay attention. Then he started diving in a bit on his own. Once he experienced a taste of the magic, I helped him to the rest of the koolaid and he was hooked. The result was him begging me to do this post to help people understand how awesome this setup is.

The problem

I hate hosting. I hate (really love - but love to hate) servers. I reallllllllllllllllllly like not having to deal with scaling, servers, server software and all that magic.

Besides servers and dealing with the tech side of hosting, I really hate paying for hosting. I especially hate paying to host sites that will get no traffic and will not change.

Here are some examples of the sites I am talking about:

fastmilo.com - this site gets 21 visits a year.
chicagocanoeclub.com - this site gets 13 visits a year.
weownthesun.com - this site gets 8 visits a year
harpercloud.com - this site gets 5 visits a year
nutpunt.com - this site gets 3 visits a year.
stevereed.org - this site gets 2 visits a year.
noahleaks.com - this site gets 1 visits a year.
biofuelmenace.com - this site gets 1 visits a year.

Basically simple one or multiple page joke/history/ego sites. Easy to make, no maintenance and no dynamic content.

There were all hosted at Rackspace Cloud Sites for the longest time. I got tired of paying $100 bucks just to host stupid stuff. I thought about moving them all to one tiny linode/slice/ec2/rackspace cloud box - but that would have been more than $100/month of maintenance. It also wouldn't have solved the problem for me. I would still be paying to host these sites.

Just as I was about to give up and delete myself from the internet completely - in walks App Engine with its fancy pants friend, DryDrop.

DryDrop is an easy way to host static content on App Engine

DryDrop is at its most basic form a bucket that holds the various pieces of a static site, and serves the pieces when they are requested. It is git backed for easy content management. It is mostly for nerds.

From the DryDrop site:

DryDrop is a tool that lets you host your static site on Google App Engine and update it by pushing to GitHub. Thanks to GitHub post-receive hooks your App Engine site can be updated automatically when you push new content."

From @dansinker:

it's a tool that you install on AppEngine that lets you publish web pages on Google's servers using nothing but github. So like, I update the Scorecard simply by typing "git push" and suddenly, WHOOOOSH, the scorecard is updated. I seriously love it more than is probably healthy.

As their site points out, this means that you can host your static content with zero cost, zero maintenance and very little real tech knowledge (just git), a bit of rake) and some gigantic internet nuts).

The DryDrop workflow is as follows:

Checkout the DryDrop repository from github
Deploy your drydrop app to app engine (rake upload project=drydropsample)
Create a git repository on github
Place static content in the github repository
Link the two (add the github repo raw url in the drydrop admin settings)
Visit app engine app url (this pulls the static content from github and caches it)
BAM you have a site hosted on google's infrastructure.

Super easy and convenient. The best part is that this is free if your static site doesn't use more resources than the free tier of app engine allows.

I used this and my mind was blown. However, there were a couple of things missing.

Hosting multiple domains on one instance

After I played with drydrop extensively and launched a few sites, I noticed a hole in the drydrop armor. For every site you launched, you had to deploy a version of the drydrop container to app engine. Although they were all free, most people only have 10 application slots, and it was a pain to manage. I needed a way to host multiple sites on one instance.

I thought about this for about 2 seconds before I remembered solving this problem with some other python apps hosted on App Engine. I needed to key the datastore off of the domain (now-a-days you could use the multitenacy stuff and just set the app namespace to change for each served domain (this might be a good idea to migrate drydrop too - i will do that later)).

I made the change on my fork and deployed it to GAE. Then did a bunch of tests to make sure it worked properly. It did and the code was pulled into the main drydrop repo. Excitement.

Now you can deploy one drydrop app and host many many domains off a single app instance. I currently host about 15 domains off of one drydrop instance.

Free Hosting, FTW

Let's review what we have:

Free hosting for multiple domains consisting of static content on a google scale infrastructure. For free.

This is awesome. and once you have it set up, there are only two steps to getting another site hosted (1. register domain, 2. point to drydrop instance). I launch stupid sites constantly now. It is an addiction. I need help.

Pro-Tips

I have used this setup for about 6 months. It works super well. I noticed that things loaded super fast all over the world, which is pretty awesome.

I have done 3 things to make things faster: use a CDN, use the high performance replication scheme, and use always on instances to decrease startup time.

I will attempt to cover each of these quickly.

Use a CDN

I use Amazon Cloudfront with a custom origin. This is simply a way to take advantage of using a cdn without having to uplaod my data and assets to s3 first. I have two scripts that I use to manage this process. The first sets up the origin server with cloudfront, the second clears my cached items when I need to deploy a change.

How this works is simple. I cname media.nata2.org to cloudfront. Then every request to that domain goes to cloudfront. Cloudfront checks if the asset exists on itself, if it doesn't it goes and fetches it from the pre set origin server (in this case www.nata2.org) and then serves it. All subsequent requests are served directly off of the CDN. No more scripts to upload shit.

I think this is the only civilized way to use a CDN. I hope that Akamai/Rackspace rolls this out in the future. I prefer Akamai's network to Amazons.

Use the High Performance Replication Datastore on App Engine

Very recently Google released a neat addition to the App Engine world: High Replication Datastore. From the google app engine post:

The High Replication Datastore provides the highest level of availability for your reads and writes, at the cost of increased latency for writes and changes in consistency guarantees in the API. The High Replication Datastore increases the number of data centers that maintain replicas of your data by using the Paxos algorithm to synchronize that data across datacenters in real time.

BOO YEA!

This makes your simple static no write app engine app bullet proof (it seems). If there is a data store outage - this should keep your app up and running.

The downside is that it is about twice as expensive to host. There is a chance that it will still be free for a static site.

I have played with this less, but have deployed it to my blog and we shall see soon if it works well or costs me an arm and a leg.

Set the app to be "Always On"

Another recent feature added to App Engine is the ability to keep an instance of your app "on". This should decrease startup times for stale apps.

Sometimes, when I visit one of these sites that don't get a lot of traffic there is a small pause while the app starts up. With an Always On instance, this should stop happening. App Engine keeps an instance of your application warm and ready to server for a measly $0.30 per day ($9.00 per month). A pretty sweet deal if your site suddenly not performing worries you.

I have only played around with this - and haven't deployed it to my blog or in production sites. I also don't notice a huge problem with sites not serving quickly enough.

Awesome

The last two tips are app wide - which means that they will work for all sites that are deployed on that instance of DryDrop. You will be able to split the costs associated with the high performance replication and the ALways On instances across the number of domains you are hosting.

This makes me LOVE hosting again.

2010 Retrospective

2011-01-01T00:00:00-06:00

2010 was a great year. I rarely write these retrospectives (only 2008 I think), but i think about writing them every year. It is so cheesey, but also a fun exercise in memory and retrospection.

So.. here goes.

2010 was a great year. I spent a bunch of time helping startups, hanging out with friends, spending time with Hiromi, traveling and enjoying life.

Work

I spent most of 2010 doing my best to avoid deliverables and enjoy not having a day to day job.

I started out 2010 in San Antonio, TX hanging out with my friends at Rackspace. Rackspace is an awesome company and is powered by a great group of people. They are doing some neat stuff and are fighting a good fight. I am excited to see some of the stuff we talked about come to fruition. The coolest of those things is Open Stack taking off. This changed everything in the cloud world. I finished my Rackspace gig in April. It was a great opportunity and hilarious to hang out in a mall. I was also excited to have Rackspace acquire my friends at Cloudkick. That was a great move. Congratulations to everyone involved.

After the Rackspace gig was over, I had nothing else planned. It allowed me to jump into my next plan. Learning about Startups, Venture Capital and the business of business. ;)

Startups

2010 also allowed me to further my interest in venture capital and startup funding.

I spent a lot of time with Sandbox Industries. They have a bunch of neat things going on and are succeeding in helping change the Chicago startup landscape. I am excited to see what 2011 brings fro Bookyap, Lab 42 and Marbles: the Brain Store and the other Sandbox startups.

I also had a chance to hang out with my friends at DFJ Mercury. Blair and Aziz are awesome. I also am super excited about my friends and DFJ portfolio companies Infochimps and Graphicly. 2011 is going to be awesome for them. I can't wait to see this crew at SXSW 2011. It should be insane and fun.

Startup funding has changed quite a bit in 2010. I was lucky to be at the forefront of this change by being a Angel List scout. You can learn more about it on my scout blog post. Interacting with Naval and Nivi has been awesome. They are great dudes who are changing the face of startup funding whether the industry wants it or not. It will be neat to see how 2011 turns out with early stage funding.

I cannot wait for Chicago to see the funding impact of Groupon's success. It is really exciting to see my friends achieve so much and I am very excited for how this impacts Chicago in the next 5 years. My buddy Leon Chism wrote a post on this topic awhile ago, I suggest you read it.

Speaking of startups in Chicago, 2010 was a record year for startup culture and execution for us Chicagoans. We saw the rise of Groupon, the Chicago hacker news meetups and a few good conferences. I am super exctied to see what this Chicago startup community achieves in 2011. Stay tuned.

Technology

In 2010 I launched a few neat applications. I soft launched CTA Alerts with Dan X. O'Neil - a aggregation of various alerts in the Chicago Transit system (with the coolest URL shortener - trn.st).

I launched Supertracker - a really neat "track everything" via xmpp site powered by my friend Julien's startup Superfeedr.com. SuperTrackr got a bunch of writeups and was doing crazy incoming PuSH traffic. All scaled magically by App Engine.

I also launched the hilarious ProximityCheckin.com - a site that merges your Google Latitude location with foursquare checkins. Originally I wanted it to be automagic - but that is complicated. I am still experimenting with it and should have some progress shortly.

All of the above applications were weekend projects built on App Engine. I enjoy App Engine and can't wait to see what is next for that platform.

In 2010 I also got addicted to personal informatics. I got a withings scale and a fitbit and started tracking all sorts of things manically. I hooked it into my personal site and made some graphs.

I also started tracking my location as well. You can see up-to-date location infomation on my personal site as well. Including a relatively accurate real time display of my location. I love Google latitude and am excited for all the things that can be powered by constantly updating location information.

I also scored a really neat award based on the work my buddy Dan X O'neil and I did on the CTA api. You can check out my write-up to learn more. I love the civic hacking.

2011 is going to be an amazing year for technology. My simple "2 word" predictions are: Cheap Android, More Javascript, Kindle Epub, more Nodejs, auto scaling, app engine, and minor AI. I am excited.

Personal

In 2010 we moved from lincoln square to wicker park.

We hung out in Japan for a bunch of time and we spent a bunch of time in Greeley, CO. It was nice to spend some quality time with Hiromi's family and my family.

My brother and I played Minecraft a bunch. It was a bit like when we played legos as kids. However, I was better sharing the pieces with Minecraft. Here is a map of our world as of 01/01/2011.

One of the best parts of 2010 was hanging out with my amazing wife Hiromi. She is awesome!

It was a great year and was awesome. Things were low key and everything was wonderful.

Some 2010 statistics

Here are some stats on various things in 2010:

Social

Twitter
- I tweeted an average of 12 times a day.
- The most tweets I tweeted was 52 on January 10th. This was one of my first days hanging out at Rackspace. Apparently I was enjoying hanging out with all my friends.
Foursquare
- I checked into an average of 2 places a day.
- The most checkins in a single day was 9 checkins on March 14th during the checkin frenzy that is SXSW.
Flickr Photos
- I uploaded an average of 1 photo per day.
- The most photos I uploaded in a single day was on July 20th when I scanned and uploaded 100 of my ticket stubs. Check out the set.
Google Reader Shares
- I shared an average of 10 shares a day.
- The most shares in a single day was 50 shares on [January 7th]. It was my first day in San Antonio and I apparently was waiting to go into the office.

Health

I tracked my weight 263 days and my steps 142 days. I wasn't very good at tracking my sleep, I only tracked 65 nights.
Sleep
- Of the nights tracked, I slept an average of 7.1 hours.
- My average bedtime was 2:18 am.
Steps
- On average I do an average of 7197 steps a day. That is about 3.4 miles per day.
- I was a gigantic slacker on July 28th and only took 203 steps.
- On August 9th I did about 21423 steps. I was at a lollapalooza afterparty and dancing my ass off. It was awesome.
Weight
- My weight in 2010 was hilarious. It was all over the place. Check out this graph.
- I started out at about 178 lbs in the beginning of 2010.
- On June 6th I was the heaviest I was in 2010 at 180 lbs.
- My goal was 170 lbs and I achieved that around October 2010. On October 29th I weighed in at 166.9 lbs, the least I weighed during 2010.
- I currently weight 172.5 lbs and I blame the holidays and candy.

Travel

I traveled more in 2010 than I have any year previous. I was out of Chicago around 88 days. That is about 23.84% of the year away from home.
Of the 87 days, I spent 68 days (18.63% of the year) traveling in the US and 20 days (5.48% of the year) traveling internationally.
Within the 68 days spent traveling in the United States I spent time in the following states: California (4.66% of the year), Texas (7.40%), Colorado (5.48%), Massachusetts (0.82%) and New Mexico (0.27%).
I spent 20 days (5.48% of the year) in Japan. It was amazing.

2011 is going to be awesome

I am very excited for everything that 2011 is going to bring. I have a lot of plans. Stay tuned.

Lincoln Square Retrospective

2010-12-07T00:00:00-06:00

I meant to post this months ago. hah. For some reason, it didn't quite make it live. Then a couple days ago, I was talking to a friend about his recent move to Lincoln Square. We spent some time talking about great places to go eat and hang out, and I remembered this post. I thought I would get it up and out there.

A couple months ago, Hiromi and I moved from Lincoln Square to Wicker Park. We had wanted to live in Wicker Park for many years and since we have nothing keeping us in Lincoln Square - we decided to go ahead and move.

Lincoln Square was awesome.

I really enjoyed living there for the 3 years we were there. The neighborhood was awesome. The people were great. The nightlife was... well, there was a bunch of great restaurants. ;)

I wanted to take a minute to chronicle the places we went to a bunch and the places we liked. Hopefully this list will helpful to someone. These are all the places I will miss.

Jury’s - This place is insane. It has great hamburgers, great food and great service. It also has one of the strangest crowds in Lincoln Square. Always makes me chuckle.
Drew’s Eatery - I like drew. He is nice. His eatery is a great place to get an afternoon snack or quick dinner. Very vegan friendly. This is key. They have Italian beef sandwiches. It isn’t often you can go to a vegan friendly place and order an Italian beef sandwich. I enjoy their water in a box.
The Globe - I think i went here 500 times for lunch with the SkinnyTech crew. It is a good place to watch people, football, rugby or have cheese fries.
Cafe28 - I love cafe 28. I suggest you go there immediately.
Delicious Cafe - another vegan friendly place in the area. They have great vegan sandwiches and sweets. They often host dinners and other special vegan events. *Rosded - This is my favorite Thai place. It is cheap, good and small - as all thai places should be.
Costellos - makes good sandwiches. We often walked there on an afternoon. It is always packed with families.
Half Acre Brewery - a recent addition to the neighborhood. It is a great way to grab a growler for a party or BYOB place. *Mythos - Mythos is insane. I love every bit of it. If you are a fan of Greek foods and like BYOB then this place is for you. Mix it with some wine or the aforementioned Half Acre growler.
Glenn’s - I think this is one of my favorite places in Lincoln Square. It is hilariously packed. Make a reservation.
Mertz - this is for hiromi and hippies. They have nice soap.
Hazel - A nice gift store/nick nack store. A great place for picking up gifts or party favors. I recommend it. We got many good items here. The people are always nice and they do gift wrapping.
First Slice cafe - another great lunch spot. Out of the way. tasty and nice.
Cho Sun Ok - The insane people who went here took ALL of our parking spots, always. But for good reason. this is a great korean place.
Davis Theatre - It is like a grindhouse theatre mixed with yuppies. A great place to see a cheap movie.

I imagine I missed a bunch of great places. If you have any favs, leave them in the comments.

I really like Lincoln Square. I think we would live there again.

The Tech Cocktail Startup Mixology Conference

2010-11-01T00:00:00-05:00

On October 28th, I had the pleasure of speaking at the Tech Cocktail Startup Mixology Conference. It was a fun quick conference with a set of amazing speakers.

Below is my talk. I spoke about fostering a community of problem solvers. Here are my slides.

Would love to know your thoughts.

I am excited to see Tech Cocktail grow. I have been going to TC for years now. Threadless sponsored the first TC in boulder, we had tables at various TC's in Chicago. Scott and I did a really amazing presentation on printers at the first TC conference. It has been awesome. I can't wait to see what Frank has in store for TC next.

CTA Hacking, Unofficial APIs and Winning a Data Innovation Award

2010-09-25T00:00:00-05:00

Back in 2008 I hacked an unofficial api together from the exposed endpoints of the CTA’s bustracker application. This spawned a bunch of CTA applications in the iOS app store, the android app store and created a really neat ecosystem of CTA hackers.

For awhile, nobody but us transit nerds paid attention to this api/hack. However, as more and more apps were built and more and more people used those apps - more people started noticing the benefits from the opening of this “api.”

The API got a number of really good write-ups, particularly about how the API was unofficial and how I went around hacking it. The first (and best) was by my good friend Dan X. O’neil: "The power is not the mashup. It's the data.". From here a bunch of really great posts popped up talking about the unofficial API and innovation. Joe Hughes blogged about whether the unofficial API is innovation on borrowed time. Programmable Web summarized Dan’s piece and added some commentary.

In the aforementioned post by Dan O'Neil, He talks about how we visited the CTA and chatted with them about the API. In the world of litigious organizations and scary governments - working with the CTA was awesome. They listened to what I did, asked really solid questions and then told us some of their plans. A lot of what we talked about ended up being included in their official API release. Dan’s brother Kevin O’Neil wrote this relationship and the results up on the CTA Tattler blog: Great example of government-private cooperation: Harper Reed wins award for CTA's Bus Tracker API.

Interacting with the CTA directly and working with them on their app was great. It really demonstrates how a private citizen can affect the mega government type organization to make things better for the future. I don’t think this is an easy pattern to replicate - but it is something to aspire to when hacking/proselytizing open data and open gov type APIs.

The most recent chapter in this story is that the MCIC awarded me a Data Innovation Award. I was sadly unable to participate in the award ceremony(I was drinking Chu-Hi in tokyo), but I hear it was awesome. Dan O’Neil was able to pick my award and say some words about what we did.

A lot of people helped make this happen. Dan O’Neil was an inspiration and really helped me figure out a strategy in working with the CTA. The CTA itself was awesome - especially Tony Coppoletta and Graham Garfield. Both of those dudes made it easy to work with the CTA and are also sweet guys outside of the CTA.

This has been a really good hack. ;) I am now a data innovator.

Yes! I am a venture hacks scout! Chicago RPZNT

2010-09-03T00:00:00-05:00

I am excited to be a part of Venture Hacks helping them find amazing startups for the Angels on the Angellist. The scout program is going to be a lot of fun and hopefully will allow some great startups to be funded by the amazing angels from the angel list. Check out this great write up in GigaOm.

If you have a startup that you want to submit to the angellist, you can do so yourself by submitting directly to the angel list. If you want more information about the angel list, or would like me to take a look at your startup (and get all the perks of interacting with a person as amazing as myself - hah) - then send an email to me directly. I am pretty easy to contact and can help out in many ways.

I am looking to help out and interact with midwest startups. I am very serious about making sure that Chicago continues to be a good place to be an entrepreneur. Having venturehack/angel list representation in Chicago will continue to help the community. Be sure an check out the listing for Chicago Angels.

Making the Midwest/Chicago a great place for starting a company isn’t just for people like myself, the groupon guys, sandbox and the angels. It is about the entrepreneur as well. Please let me know if you have ideas, concerns or just general insights into making the Midwest/Chicago the best place to start a company.

If you are outside of the Chicago/Midwest area - feel free to ping me anyway. I am always looking for new startups to talk to and people to connect.

I migrated this blog to Jekyll on App Engine. So long Wordpress.

2010-08-17T00:00:00-05:00

I don't like to move my blog around. It is super annoying. The RSS feed gets all messed up, I spend months tweaking everything. I find silly bugs and then fascinate about fixing them. All in all it takes months to complete the process. I avoid it like the plague.

However, I really wanted something simpler than wordpress. I didn’t need a CMS. I barely need a blogging engine. I update so infrequently. I want something that creates well formed html (hah), static content and is easy to use.

I started looking into hosting this blog on Google’s app engine awhile ago. I looked into Bloog - an awesome restful bloggin engine for app engine. It was very hopeful. i spent a bunch of time hacking on it and eventually ported my theme over. I didn’t feel it though. it worked well. it had all the features i wanted. It even used a bunch of neat app engine tricks. I gave up on using it for my blog. It just wasn’t the correct choice.

During this time, I watched most of my hackerish friends start to use Jekyll to power their blogs (or write and then use igor as anders did (protip: do not search for “anders jekyll” in google)). I admired the simplicity of the jekyll engine. It was so easy and fun to use. I wanted to use it for my blog - but it didn’t play as nicely with app engine as i had hoped.

I then found Drydrop.

DryDrop is a neat application for app engine that let’s you host static content. In my current phase of life, I really hate managing servers. It is fun and all - but if i don’t have to - i am not going to. ;) Instead, I try and use platform as a service services(heh) to minimize the number of servers that I touch on a daily basis (my current server number is 4 (I secretly want it to grow back to 100+)). DryDrop was a nice solution for my server hatred. It allowed me to serve simple static content on app engine. I immediately ported a bunch of my static sites over to app engine: harpercloud.com, biofuelmenace.com, weownthesun.com and of course ryankanno.com.

After a few weeks with these sites on app engine - I decided that it was probably the right place to put my precious blog.

Jekyll is an interesting beast. It is exactly as the wiki proclaims: a simple, blog aware, static site generator. It has no features (well. a couple), it has no built in community aspects (no commenting, trackbacks, reactions, etc). It doesn’t have any admin. it is just a static site generator that is geared for blog like sites. It works amazingly.

The first thing I did was port my theme over to Jekyll. You can check this out at my blogs git repository. It was a pain in the ass, but i was able to clean up a bunch of annoying HTML issues that i had from changing shit all the time. I still need to refactor it a bunch.

Once the theme was done, I started working on the content. This is the biggest issue. I have a bunch of content. Like 1000s of posts. Jekyll is not necessarily the quickest of generators when you have thousands of posts. Luckily I was able to do some simple tricks to ensure that the old posts work and the new posts work as well. Its honestly pretty annoying and i need to solve this better.

A couple hints for hosting Jekyll on App Engine with DryDrop: * check out this post by Carl Sverre on tweaking the site.yaml file for drydrop to handle the pretty permalinks * the wordpress migration script that jekyll uses is pretty nice. i hacked mine to export tags and datetime as well. * there are hundreds of blog posts just like this one. they all have great hints * if you get stuck, just check out the list of sites that are using jekyll. Often they have source on github. you can glean all sorts of goodness from them.

It is a bummer to leave wordpress.

I have been on wordpress since early 2005. I think that is the longest I have stayed with 1 piece of software. I really enjoy the wordpress community. I really like matt and all he has done for open source and the internet. I like the fights they pick and i enjoy the innovations they are bringing to blogging. I don’t like having to fight hackers off every single day. If there was one reason i am abandoning wordpress it is because of the hackers.

I hosted about 25 wordpress blogs on my mosso account for various friends. I kept most of them up to date, but a lot of them were for friends and were not under my control. 100% of them got owned. hah. It was just something they did. no matter how fast or often i updated the wordpress software - it would be owned at least one time. My personal blog was safe for some reason. Maybe it was because I always ran the bleeding edge version from SVN. I will not miss the constant updates and the attacks. The wordpress community does a good job of handling this issue. I, however, was tired of it.

I migrated at least 22 of the 25 hosted blogs to wordpress.com or posterous. I killed all the unused wordpress installs. No more hackers getting in through plugins, themes or trickery.

If i were to use wordpress again, I would either use wordpress.com or vaultpress. Both of those products seem to solve my problems. Right now, however, jekyll has pretty thoroughly solved them.

I can’t wait to start updating this blog a bit more often. I don’t know if i will write like i did in 2003/2004 - but i plan to update it a bit more than i did in 2009. We shall see ;)

If you see this, this blog is hosted on app engine

2010-08-12T00:00:00-05:00

Decided to ditch wordpress for jekyl and drydrop. Let's see how it works.

I will update this post later with more info later.

Smitten by the iPad: thoughts and complaints.

2010-04-05T00:00:00-05:00

I was tricked by the gravitational forces that an apple product announcement causes into visiting the apple store. Five hundred bucks later I walked out of the apple store with a fancy new iPad. Excited and Doh!

The iPad is a new and very exciting paradigm for computing. I played with it afternoon and it felt very minority report. It felt like the future. no physical keyboard. I was controlling things with my finger/mind. it was awesome. I am smitten.

Besides the irrational distortion field emitted by this shiny device, I have some general thoughts/complaints on the iPad.

The lack of camera is irrational. I have no idea why apple didn't include two cameras on the iPad. It would be an amazing device for parents, students, people, etc. Chatting is annoying on non-multitasking devices, however it might be nice to Skype with a slate like device. It will obviously be added to the next iteration of this device.
The iWork cloud platform is annoying. Importing and exporting documents is very hard. It is obviously not meant to be used by people who have documents already. The only way to get shit on the device is to download it. Why wouldn't Apple integrate iWork with MobileMe? silly. Google will have to change their Docs platform to allow easy downloading from the mobile google docs page.
The lack of any clock or alarm app very strange. I use the clock app on the iphone for a lot of stuff. not having that is a bummer. but in the grand scheme of bummers its about a 0.00001/5.
There is no usb host mode (that i know of). I would love to charge my phone, etc via USB with this device. it would minimize the number of cables and what not i would bring. It would also be awesome to plugin external disks or keyboards to it.
Google Reader blows on the iPad. This is a real bummer since that is pretty much the one thing i want to have all the time. You can't scroll with the desktop version and the mobile version seems silly on the iPad. I can't wait for Google to "fix" all of their webapps for the iPad.
Using an iPhone after spending a lot of time on the iPad makes the iPhone seem like a midget. It makes the nexus one seems like it is made out of paper.
The iPad is probably one of the coolest ways to show presentations to someone while sitting at a bar. I have been pitching some ideas to people and having the iPad with my deck is pretty awesome. A great way to go over your presentation with one or two people. Way better than a laptop.
The Kindle app is awesome. I like it a lot. I want to try reading a book on the iPad, but i really like the kindle's screen.
Most of the apps are boring. Seriously. The apps that are impressive are: Kayak, Maps, Photos, Kindle, Twitterrific, instapaper, Netflix and Square
Square is one of the coolest apps I have seen. It should blow peoples minds. I imagine that it will be a game changer for friends who do retail. I can't wait for my square card reader to show up. I will be charging my friends for everything.
This is an amazing device for casual internet use. I imagine that Hiromi, my parents, my brother and anyone else who isn't attached to a computer all day long would love these devices.

Overall the iPad is amazing. I can't wait to see how developers iterate on the platform and create really nifty applications. It will also be fun to see what features Apple magically unlocks on the iPad.

I am going to try and build some applications now. If you have ideas - let me know.

Musings about femtocells: the futures!

2010-02-11T00:00:00-06:00

I wrote most of this entry the day before the earthquake in Haiti. It was motivated by the magicJack announcement and my interest in mobile devices within developing nations. However, the thoughts in this post suddenly seemed more real after I started hearing reports of communication breakdown after the earthquake.

The femtocell

At the recent CES, the magicJack company announced that they wanted to release a consumer grade FemtoCell device as an upgrade to their USB FXO/Softphone device. Note: magicJack is an insane company. You probably have seen them on television commercials while watching USA 'Up All Night'. While these commercials almost seem scam-like, they have apparently sold 5 million USB magicJacks. That's a lot of phone calls!

A femtocell device is basically a mini GSM/CDMA tower. Think of it as a wi-fi AP but instead of wi-fi - it is GSM. You could drop it into your house, plug it into the Internet, and suddenly get 5 bars of connectivity on your mobile handset - instead of the standard AT&T 1 bar. magicJack is offering the same service the USB dongle provides (unlimited calling and what not), but for mobile phones. As a consumer, you would need to configure your handset to use the magicJack femtocell to receive/send calls. When you are in range of the device (w/ signal strength enough to cover ~3000 sq. ft. house), you would bypass your carrier and use the magicJack for all incoming and outgoing calls. AWESOME. Especially with the likes of Google Voice.

There are questions on the legality of such a device - but it is apparently legal to use inside your residence.

Side note: Magicjack is SIP. And hackable. There is a cat and mouse game being played out trying to unlock it. You can even use it with Asterisk.

The thoughts

So I began to think. Femtocells could be used to open up mobile VOIP. One scenario is that you could jump on a 3rd party carrier for a limited time to make a p2p VOIP call using your current GSM/WCDMA mobile device. That is really awesome. More importantly, what's REALLY awesome is the pirate (3rd party) carriers.

Imagine purchasing 100 Motorola F3's, a GNU Radio USRP2 (~$1400) running the OpenBTS stack and a super huge antenna - and suddenly becoming a rural/developing country mobile provider. Or - if you're more interested in being a pirate carrier - you could run this setup in a city and be a carrier for the dark side (drugs, gangs, nerds...). This inexpensive setup is available (and achievable) today! In case of the apocalypse/disaster/camping/nerdcamp - I think it would be nice to have a set up to deploy to organize/help people.

My mind started to work into overtime. I began thinking about the ramifications of hacking the magicJack. Wouldn't it be neat to use femtocell technology to distribute ad hoc mobile carriers around a neighborhood/disaster area/rural/developing country? You could hook the magicJack femtocell up to a Fon2 device (usb/linux/broadband) and distribute them around in places with connectivity. All you would need is a better antenna and some software to manage the provisioning and device identification. BAM! Instant pirate P2P GSM carrier.

I am hoping that this type of convergence will lead us to the future.

I love phones designed for developing countries.

2010-01-17T00:00:00-06:00

I love mobile phones. I don't know why - but mobile devices represent so much of what I love about technology. The ubiquity, the fascination, all the new technology that pops up and all the different way people use their devices. I also feel like mobile devices will be the future of whatever the internet turns out to be. They will power the internet. With all this in mind - I have started to become interested in the devices that fill the opposite niche. The devices that are SO simple that their are almost NO features.

These are not Razrs, or other shitty mobile devices which act like they have all sorts of amazing features. These specific phones are single feature, cheap devices and are often deployed in developing countries.

There are two that I really enjoy and I suggest checking out.

The first is the Nokia 1100. Also known as my favorite phone in the world. I first saw this phone while working for a minute in Bangalore. It was advertised as a phone "made for india" and apparently was hardy enough to be used by the working class there. I was drawn to the device. I neeeeeded it. It's weird, because at the time I was running a symbian device that supported about every possible feature available in mobile - so thinking about jumping to a mobile device that was pretty much featureless was crazy. But - I was compelled.

The phone isn't that fancy. It has very few features. It is super tiny and is super cheap. It is just a basic, simple phone that does nothing more than look "Nokia" and probably makes calls. The features it has are incredibly handy: flashlight, alarm clock and sms messaging.

I didn't realize it then, but the Nokia 1100 is possibly the most popular electronic device in the world. According to the infallible wikipedia: Nokia has sold ~200 million of the Nokia 1100s since it's launch in 2003. To put it in perspective - there have been ~30MM iPhones sold and ~50MM Razrs sold.

I eventually got a couple of these and used them as lenders, experiments and generaly fucking around phones. I even sent one to my brother for him to use - which he eventually washed in a washing machine - it still boots. They are HARDY. and 20 bucks.

The second phone that I am interested in is a bit newer and is a bit more tech. The Motorola F3. My buddy Hemmant showed it to me, told me a bit about the history and I couldn't stop thinking about it. Anecdotally, this phone was made by a Motorola team in india - again for use in the developing countries market. It is simple, has no features (even less than the 1100) and is JUST a phone. There is one major differentiator. The Motorola f3 is one of the only phones to have an e-ink display. This means that it uses very little power in standby (standby time of ~17 days), can be used in low light/no light situations and is rather resilient. The phone didn't sell as many as the Nokia 1100 - in fact it was a veritable failure.

The F3 has no features. It barely send txt messages. It has voice prompts for all menus. It is just a phone. A user serviceable, easy to charge, thin, e-ink phone. It works well making calls and receiving txt messages. The F3 would be an amazing phone for a child, elderly person or someone who doesn't want to deal with learning pesky features. I got one for my "go bag" as an emergency phone (I am not a survivalist, I just crush a lot).

You can find an unlocked Motorola f3 for about 30 bucks on amazon. I suggest you get one.

2009 Hosting Summary (or where does Harper host in 2009/2010)

2009-12-09T19:29:06-06:00

2009 was an awesome year. A lot of amazing things happened in the world of hosting (and a lot happened in the world of Harper too). Being a crazy hosting aficionado, I will summarize my thoughts on hosting in 2009 (I will leave the predictions for 2010).

Disclaimer: I am paid money by Rackspace to think about these things. However, I am an equal opportunity hosting consumer. I will try and be as open as possible (but probably a bit biased as well). This is on my personal blog and does not represent Rackspace in the least. These opinions are mine and do not represent Rackspace in anyway.

I guess I will go ahead and jump in. This is based on what I use and would most likely suggest if we were drinking beers and talking about hosting. Your mileage may vary. I am sorry if I end up causing a nuclear war based on these recommendations.

"Platforms"

Two platforms are really getting me going these days. Google's App Engine and Heroku.

App Engine is known by my close friends as my true love (haha. second only to my lovely wife). I spend almost 90% of my development time hacking on App Engine apps (Excla.im, CityPayments, ChicagoAlerts, Transitapi, AwesomeUpdater, and Harper's Friends). App Engine does a few things very well and allows the developer to do the rest. Quite a few of the lessons I learned while scaling out Threadless are baked into the app by default. You don't have to scale all the pieces independently and you can really take care of things all at once. I recommend it and use it for prototyping and building new apps.

Heroku! I go to Heroku.com I spend about 10 minutes playing with sliders, clicking on features and acting like I am going to deploy an app to their infrastructure. It's awesome. Heroku makes me want to learn ruby frameworks well and build applications fast. It seems that it might be expensive though. I would love to know if people have actually used it in paid production environments (i have only done hackings).

If you are interested in Django, Ruby/Rails or JVM languages (I am looking at you Scala!) - I highly recommend App Engine or Heroku.

"Cloud" Server Providers

Cloud Servers are amazing things. Usually they are XEN hosted and wrapped by an API or a nice pretty interface. I <3 them. If I would have had these server options in 2001 I would have been able to make the server kingdom I dreamt about. It would be awesome. Luckily, now in 2009/2010 I can build my server kingdom in the CLOUDS. Anyway....

Slicehost is my favorite. I really enjoy the user interface. I prefer websites with less buttons. Slicehost delivers. However, it doesn't explicitly have usage based billing (pay per hour) - which is why i am also excited about Rackspace Cloud Servers. Rackspace Cloud is everything you need to have fun with servers and host enterprise apps. My most recent Cloud server was used to pop up a VPN in Ubuntu so I could use public wifi without being frightened. It took about 10 minutes from deciding to do it, to connecting with my macbook pro. awesome.

EC2 is pretty sweet as well. It is the heavy duty infrastructure that powers so many awesome sites. I really like watching what Amazon is doing. Amazing stuff. I am super excited about booting from the EBS. However, I don't like the complexity that comes with EC2 by default. Which is why I only use EC2 if I need AWS specific utilities.

Sidenote: Linode is getting a lot of buzz based on their performance/cost ratio. I recommend you take a look at them and see if they will work for you.

If you do find yourself in the dark world of cloud servers, you should check out my favorite utility for managing all things servers: CloudKick. CloudKick is a pretty solid app that allows you manage your all your Rackspace and Amazon servers. You can do it very easily and with very few buttons (I love a small amount of buttons).

If you are looking for awesome easy servers - use Rackspace Cloud Servers. If you are looking for a full stack (load balancer, block storage and what not) - use Amazon. I use Rackspace Cloud Servers.

File Hosting and Serving.

File hosting is simple stuff. I follow this simple guideline:

If you want to host a file for everyone (i.e. a popular image, a iso, download etc) where performance is important - use Rackspace Cloud Files with limelight integration. This is the closest you will come to paying for enterprise CDN capabilities for a tiny amount of money. You can't beat the level of performance that limelight offers (unless you want to pay for Akamai).

If you need to put a BUNDLE of files somewhere that needs to be public or private - use Cloud Files or S3. Depending entirely on what vendor you like or what API you are most familiar with. I use both for backing up various data from my servers and home (I recommend s3sync, s3fs for S3 or MossoFS, CloudFiles/Duplicity for Cloud Files).

Multi Tenant/Shared Hosting.

Shared hosting is such a pain in the ass. ;) I really don't like it. However, I use it because of the price and because of how easy it is to host stuff for friends and others.

I use two providers regularly for shared hosting:

Webfaction is by far my favorite multi tenant hosting company. It is SUPER NERD. They have this unique approach to hosting based on proxying a bunch of servers through apache. This allows you to easily host a django app that has a rails component and a wordpress blog all at the same domain root. You can even compile your own build of whatever flavor of server you like best. Its a great place for prototyping servers and generally having a great time inlinux. Webfaction also has a developer community that offers great support. I host most of my personal stuff onWebfaction (this blog, harperreed.org, etc).

Rackspace Cloud Sites is a great multi tenant platform. I use it hosting various apps for your friends and family. It is a no brainer to light up a domain for my brother or friend and have them host it. The only problem I have with Cloud Sites is that sometimes you have to compromise your code to fit in their super scalable infrastructure. You will NEVER go down - but you might have to tweak your drupal or wordpress code to make it easy.

If file storage is just what you need - then check out Dreamhost. For the love of god they have cheap prices. I looked today and they were offering unlimited everything (that doesn't make sense). The only problem is that they don't allow you to use it as a backup service (you have to use the space in a very specific way).

Enterprise/Physical Servers.

Enterprise servers are easy. You use Rackspace. ;) Seriously though - while at Threadless, Rackspace saved our asses more than once. Using Rackspace allowed us to concentrate on our application rather than concentrate on our servers. Although they can be expensive - they offer so much that it is often worth it.

If you are looking for something a bit cheaper or a bit more hands on, I suggest my favorite chicago hosting company: FDCServers.net. They offer great support and cheap servers. You are not going to get near the support that Rackspace offers, but you are going to pay a LOT less. I host a couple big apps at FDC and have had no problems at all.

Conclusion.

This wraps up my hosting review. I recommend App Engine, Cloud Servers, Cloudfiles/S3 and Rackspace Managed hosting. heh. See - super easy. All your hosting questions are answered.

2010 is going to be an exciting year. I imagine that Rackspace and Amazon will continue to fight head to head. Which will only make us consumers have better options. I also am very excited to see what Google does with App Engine (I imagine that they will NOT be adding PHP support). I hope that 2010 is as productive as 2009 was for hosting, if so - then it will be an exciting, innovative time.

Leave your questions in the comments - I will attempt to answer them the best i can.

Leaving Threadless

2009-10-07T21:52:50-05:00

I am leaving Threadless after 4 awesome years.

It was super hard to leave a job that taught me so much, helped me grow and was staffed by all of my friends, but I needed to make that leap towards the future. (Here is a nice interview covering all sorts of aspects of my leaving.)

I did it, and the future is looking bright.

In the next few months, I am going to be helping Rackspace in their cloud computing division. My official title is Nepholologist; a person who studies clouds. I will also be working hard to launch a startup (secrets secrets). This is extremely exciting. I'll be spending half my day working with amazing technologists at Rackspace and the other half pursuing my own thing. Rackspace has been amazing. They've accommodated my want for freedom and the ability to concentrate on my own stuff while helping them out. It will definitely be the best part-time job I've ever had.

I plan on detoxing. I'll be cleansing my brain of all the assumptions that I formed while at Threadless and ready myself for the startup life. I want to be more scrappy and lean in my thinking. Part of the plan is to complete all the various projects that I've worked on throughout the years like citypayments.org, excla.im, exclaim tracking, and harpersfriends. I've done a lot of fun hacks in the last few years, and I want to work on making them much more robust. This will allow me closure on one chapter of my life - while some doors close, new ones are opening.

Thanks to everyone who was supportive of me in this time. I could never have made this decision without the support of my family and friends. To my lovely wife Hiromi, who was constantly telling me to pull the trigger. My friends and coworkers who knew something was up - but dealt with me until the end - even when I was flighty. My parents who helped me think rationally. Most importantly, the entire Threadless family. Without Threadless and the knowledge I've gained from my experiences there, I would not have the opportunities I have today. The last four years have been utterly amazing (and awesome).

I have a couple of posts lined up for later this week. One about my history at Threadless and another about the hijinks that have existed over the last few years.

Stay tuned. It's only just beginning.

My thoughts on the kindle: 1 vs 2 and the iphone app

2009-03-08T15:54:09-05:00

First things first: The Kindle is awesome

I read a lot. I am not like those crazy people who read a book a day or anything, but i have been reading up to about 2 books a week for the past few years. They are not crazy books, mostly science fiction and fantasy, but they are not small books. I read to chill out. My day pattern is to go to work, work, get home, nap, hang out with hiromi, play computer and then read until i fall alseep. The reading part is VERY important to get my mind and body to a place where I can actually sleep. Needless to say, I take my reading seriously.

I ordered an original Kindle way back when they first came out. I was back-ordered for quite awhile. It was enough time for me to question whether or not I would like the device. Luckily, my friend edith was also in the same place - she and i chatted incessantly about what the future would be like with the Kindle. It was a bright future.

I ended up getting it around mid january 08. The first book I read was Terror by Dan Simmons. That is a serious business book. I had started the hardcover of the book earlier in the month and finished it on the kindle. That was my test. It was successful. Although i hated the fact that kindle pages and paper pages don't really line up - the experience was the same. I was able to read fast and was able to forget about the buttons and terrible design of the kindle (like Zach Klein said - it looked like the consumer electronics version of the Pontiac Aztec).

After Terror, I read the appropriate Diamond Age by Neal Stephenon. I felt that I was closer to the "Young Lady's Illustrated Primer" than I had been before. I was reading books on an electronic device that was hardy, easy to use and quick. I was able to give it a seemingly unlimited amount of content and could use it for quite some time without a battery change. Like I said above - it is awesome.

Around this time I discovered the dark and scary underworld that is ebooks on the internet. There is an awesome community of people surrounding ebooks. They are involved with liberating DRM'd ebooks, converting from one format to another, uploading mass libraries of free ebooks, cataloging ebooks, etc etc. It is awesome. If you are interested in free content - there is an almost unlimited amount of free content that is compatible with the Kindle available online. There is also an unlimited amount of pirated books too. They are harder to find and I feel terrible when I find them (I apparently would rather pay authors than musicians. hah). One thing i noticed while I was experimenting with ebooks - technical manuals and what not don't render as well as non-technical books. I think it has to do with the number of diagrams and tables. They don't render terribly - but often are not nearly accurate to what i imagine the author intended.

How you get the ebooks to the kindle is part of the magic. The kindle has what Amazon like to call the whispernet. It is, as far as i can tell, a sprint EVDO modem built into the device. Amazon uses this to wirelessly deliver content to your kindle that you buy from amazon.com. Once again - it is awesome. You click on a book in amazon and BAM - its on your kindle. You are not limited to using this JUST for amazon's content. You can do it with your own ebooks as well. All you need to do is register your email address with amazon and then send books to the yourname@kindle.com email address. The books will appear on your kindle for a small cost ($0.10 i think). If you don't want ot pay for it, you can send the content to yourname@free.kindle.com and it will return the book in the kindle format. You can do this .txt, .pdf and .doc files easily.

The kindle file format is just the .MOBI ebook format. Which in turn similar to the .PRC ebook format. The Mobi file is a remnant from the Palm days. it is basically the Palm Database format that was used for various applications on the original palm platform. What this means is that all the ebooks that end in .mobi and .prc are natively compatible with the kindle. If you had purchased books for use with the mobipocket reader (an awesome ebook reader for windows), there is a good chance that they were .mobi files. You can copy those to the documents folder of your kindle (when you plug it in to your computer via USB) and they will show up in the book list without any work. This really opens up your options for cheap or free ebooks.

The trick then is to figure out how to convert pdf files and what not to well formed, well meta data'd .mobi files. This makes your kindle happy and allows you to continue to sort by author without having your email address be listed as the author.

I went a year reading just the Kindle. I read two paper books that were not available in a digital format during this time. It was interesting. I felt as though I needed to work out to lift the heavy hardcover books. The kindle makes you forget how heavy books can be. My arms are now wusses.

When the Kindle 2 was announced, i ordered two of them. One for me and one for Hiromi.

The kindle 2 is a huge improvement on the Kindle 1. It is thinner, better performing and a bit easier to use. It is not as easy to turn a page - but it is also not as easy to accidentally turn a page either. I like the new design. The optional leather case is amazing. It works VERY well. i was having to remove the leather case from the original when I would read it - i don't have that problem with the new one.

There are a lot of small software tweaks that have trickled their way to the Kindle 1 as well. They are separating out your uploaded content from the content that came from directly Amazon. I am also having a hard time finding things that are lost on the kindle. I suppose i could search - but i never think of that when I am looking for content. The interface on the Kindle 2 is much faster. The pages turn faster and the books load faster. The screen is amazing. I really like it. I don't like the fact that I can't easily turn off the wireless with a hardware switch. I liked being able to not interrupt my reading to turn wireless on or off.

All in all - the Kindle 2 is a huge improvment over the Kindle 1. If you are able to upgrade, do it. And then give your original kindle to someone close to you who wants one. That is the best plan.

Hiromi loves hers. I was worried that she wouldn't like it - but it turns out that it works very well for her. She likes the size and the ease of use. She took to it very quickly.

When I got my kindle 2 I was showing a couple of my coworkers some of the data about it. Specifically the whispersync stuff. I told them that I wouldn't be surprised if they released an iPhone app that allowed ebook reading on the Kindle. I based this on the success of the Stanza app. I stated to my buddies that I was only confused as to the strategy to release such an app, especially on how it could accidently stifle the Kindle 2 sales.

Then a couple days later the Kindle app was released. Like the Kindle - it is awesome. hah. With the Kindle app for the iphone and the Kindle, you are able to sync your placemarks on the books you are reading. You could read to a certain point in a novel while on your commute - then go to a meeting and pick up your iphone and continue reading from where you left off on your kindle. Then you coudl pick up the kindle and it would sync to where you left off on the iphone. That is cool. I tried it and it works. It is a bit wonky - but for the most part it works great. The only issue i have with the app is that it doesn't have ANY of the content that you upload via email. It is only amazon purchased content. Lame. I read a lot of "found" books and a lot of purchased books. An example is a series of books where 2-4 are in the amazon kindle store, but book 1 isn't. I find book 1 elsewhere and then buy 2-4. However, I can't read book 1 on my iphone. Only the kindle. WHAT IS THE WORLD COMING TO. ;) I understand amazon's reasoning for this - and I don't expect it to change. I do wish i could push my emailed books into their cloud and handle them a bit better (with metadata, etc). I doubt that will happen.

If you read a lot and want a great reading experience, you should buy a kindle.