rywalker.com

Websolr on Linode

2012-01-10T00:00:00+00:00

Just got off a great call with Nick Zadrozny. The purpose of the call was us trying to convince him to bring WebSolr to Linode.

Websolr has great monitoring and replication features, and allows developers with little Java deployment + management experience to avoid having “mysterious black boxes” in their server cluster(s).

At the end of the call, he agreed that if more Linode users were known that have serious interest in the availability of Websolr on Linode, it’d push up the project in his queue.

If you’re interested in Websolr on Linode, drop Nick a note, help us make it happen :)

Recipes and Ingredients

2011-10-13T00:00:00+00:00

I finally got my “CincyReHackathon” project done, and I’d like to share a little about how the app works.

Google Prediction fail

Despite spending hours trying to figure it out, I couldn’t get 2-legged Oauth 2 working with Google API client. On top of that, the Google Prediction training queue just would not execute a training request of non-trivial size without erroring. So I dumped it.

Directed Edge

Searching around for a replacement, I found Directed Edge which is a graph database-based recommendations engine, and using it was a breeze. Unlike Google, they had decent documentation for Ruby developers and a simple API authentication method.

RecipePuppy

For source data, I used RecipePuppy’s very simple and very open API. Thanks for that!

Typekit

I took the project as an opportunity to experiment with Typekit. Very straightforward, I think the site looks much better for the effort.

VCR for testing

Since the bulk of the app is about interacting with two external APIs, I made good use of Myron Marston’s VCR library which was a great experience as well. It just works.

Guard for testing

I also used Guard for the first time on this project, and really liked it. Learn more about it from Ryan Bate’s screencast on the tool.

Source code available

The proejct uses Rails 3.1.1 and is deployed to Heroku, full source @ Github.

Lean Startup Book Notes

2011-10-10T00:00:00+00:00

Lean Startup is a great book by Eric Ries, highly recommended.

Five Principles

Entrepreneurs are everywhere—not just in startups, big companies need them too.
Entrepreneurship is management—you’re building an organization in an environment of extreme uncertainty.
Validated learning—the primary reason for existence for a startup is to learn. The best way to do that is through scientific experiments that test each element of the vision.
Build-measure-learn—ideas are built into products which are measured generating data that we learn from and form new ideas, including whether to pivot or persevere. Through this loop learning informs the vision, and vision drives the learning.
Innovation accounting—establish the baseline, tune the engine, pivot or perservere. Cohort analysis is important concept here, because you don’t get fooled by the snowball effect of traditional metrics. Good metrics need to be actionable, accessible, and auditable. Use A/B split testing once the baseline is established to improve the ratios.

Key insights

Any work that generating value (providing benefit to the customer) is waste. In the context of a startup, any work that isn’t directly linked to a learning experiment is waste.
The two most important assumptions entrepreneurs make are the value hypothesis and the growth hypothesis.
- Value: will the product or service really deliver value?
- Growth: how will customers find the product or service?
Concierge MVP—operate the product with people doing the work that computers will eventually do, to test the viability of the whole concept before building a lot of software.
An experiment is a product. And first products should not be built to be perfect. Their role is to generate data that we can learn from.
Learn the difference between dangerous vanity metrics vs. nuts-and-bolts actionable metrics.
Keep batches small—strive to keep delivery as close to continuous as possible.

Four questions before starting something

Do customers recognize they have the problem we’re trying to solve?
If there was a solution, would they buy it?
Would they buy it from us?
Can we build a solution to that problem? (The common tendency is to jump right to this one.)

Types of pivots

Zoom-in Pivot—a single feature becomes the product.
Zoom-out Pivot—what was considered the whole product becomes a single feature of a larger product.
Customer Segment Pivot—the product solves problems for a different customer than originally conceived.
Customer Need Pivot—we learn about the customer, and that they have bigger problems that we can solve, which may require tweaks to the product or perhaps a different product altogether.
Platform Pivot—change from an application to a platform or vice versa.
Business Architecture Pivot—switch from high volume, low margin to low volume, high margin, or the reverse.
Value Capture Pivot—changes to revenue or monitization model.
Engine of Growth Pivot—change in growth strategy: viral, sticky, paid.
Channel Pivot—change in sales or distribution, where or how it’s sold or bought.
Technology Pivot—change to capture superior cost and/or performance.

Four primary ways past customers drive our growth

Word of mouth
As a side effect of product usage—others see the customer using the product.
Funded advertising
Repeat purchase or use

Three engines of growth

Sticky—product is addicting to use.
Viral—people tell people about it.
Paid—buy ads that generate customer acquisition.

How to nurture disruptive innovation

Provide scarce but secure resources
Give independent development authority
Embed a personal stake in the outcome
Protect the parent organization if it’s not used to an experimental culture

Rails web application development process

2011-10-09T00:00:00+00:00

Here is the process I am presently using, from high-level planning down through actual BDD-style coding.

Brain Dump

Using notecards, high-level brain dump of stories:

roles
action/tasks

When complete, organize content and transfer to digital doc, toss cards.

Modeling & User Stories

Develop “ubiquitous language”–Words that work for both the technical implementation and with conversations with users or customers. This language is also important for the screens, which should have include clear direction on how the system works. This would be in contrast to typical programmer-generated screens, which will often have usability issues.

To each story, add a few sentences to describe it; just enough information to serve as a token for a conversation that will take place closer to implementation.

Minimum viable product

In true “Lean Startup” style, development of the app should be treated as an ongoing series of experiments to test hypothesis on what success may look like.

So an important step is to ask what is the first validatable theory to begin with, preferably an important one. Then choose only those stories that directly help create an experiment to learn from.

Continuous delivery

Continuous integration should be setup to continously deploy the app. There should be no such thing as “releases” and “iterations”.

UI-First Development

Screens

Which screens - REST conventions can do a lot of work - what is the logical name of the noun-verb combination?
What’s on them - design from the inside out. Just the content without the outer frame of navigation. What matters to the user of the screen. What are they trying to do? Put in words what things they are trying to do on the screen, including the various paths they may take. (Don’t just assume the page should be a standard REST-template form.)
Be completely rational, “don’t make me think” - when I see the screen, I instantly know what to do with it, it’s easy to complete the task.
Least effective difference - give things visual priority, doing just enough to create the hierarchy.

Flows

Think with actions, every action has 3 screens…

How you get to it
How do you do it
What happens afterwards - how do i know it worked. What do i want to do next?

HTML & Coding

1. HTML mockup

Explore UI options on paper, in Photoshop, or in HTML to get to a happy design for the feature. It’s best to explore this in as lightweight a form as possible to avoid wasted effort.

2. Write new feature/scenario

Guard runs the features and outputs pending steps, copy and paste them, and write simple code to define them (using code semantics you wish you already had.)

3. Pick a red step & write failing spec @ the rspec level

Using Rspec, specify the missing functionality. Guard will now see red at the rspec level.

4. Write as little code as possible to make the spec pass

Get the spec green, as quickly as possible. Guard pops the stack and reruns the scenarios you’re working on. Once green, refactor anything that’s needs it.

5. Repeat until your steps are all green.

CincyReHackathon

2011-09-22T00:00:00+00:00

CincyHackathon was a great experience.

I met some great new people, I grew as a developer, I got to stay up and program until 3am and didn’t feel like a bad person for doing it :)

I won the contest, but don’t feel “done.”

Not too many people participated, but I did pull down first place and $500 in gift cards, so my wife wasn’t too pissed when I came home bedraggled early Saturday evening.

But the app isn’t live on the internet, due to my lack of Oauth2 understanding and less-than-optimal guidance from Google on how to make it work.

So starts CincyReHackathon

I’ll give myself 12 hours, through Saturday afternoon, to do it better.

DirectedEdge over Google Prediction

Google Prediction let me down quite a bit during the event. Their API has become so complicated, and isn’t well supported or documented for Ruby. I think part of it is because they’ve gone for a discovery-based API, and Google Prediction isn’t very popular, so I was really struggling for working examples.

I did finally come across a sample sinatra app buried in their API library, and it formed the basis for my final presentation, but the Oauth2 concepts (especially the way I was doing it, using the 2-legged variation) was so poorly documented that I literally spent 1/2 my time trying to get that all working rather than on my chosen problem domain.

I did a little googling and have discovered DirectedEdge who I will try to use in my re-do, and were nice enough to give me a free developer account after explaining my intentions.

Test-driven vs. hackity hack

While I started out test-driven during the event, I quickly abandoned the practice when shit started hitting the fan, and things felt very “spiky.” Now that I know what I’m building, I’ll approach this re-do in a more disciplined way.

Live, working app on the internet vs. 1/2 working

My goal at the end of this is a working, snappy app on the web.

Should be fun… here we go!

Repo: https://github.com/ryw/cincyrehackathon
Site: http://cincyrehackathon.heroku.com

CincyHackathon

2011-09-09T00:00:00+00:00

My project: peanut_butter_jelly_time

I’ve decide to help Meruni on their challenge to develop a way to discover complementary products. For example, “jelly” goes with “peanut butter”.

Challenge #1: Data

I was hoping that Meruni’s 2011 program-mate Receept would have some grocery shopping receipt data to leverage, but they don’t have any – woulda been cool to mash them up. So then I had to look for some other source of data, Meruni recommended looking at recipe data.

They like Punchfork, but to get recipe data requires paid API, and they had no way for us to upgrade free account via web. Oops, lost sale there, PunchFork.

I’ll use Recipe Puppy and the Google Prediction API.

Here’s the process:

Pull a list of popular recipes.
Search RecipePuppy to get a list of ingredients for each popular recipe.
Create training data in CSV to upload to Google Prediction API.
Build simple web app where one can type an ingredient in, and get top 10 ingredients that complement it.

Wish me luck!

Update

See code @ the git repo
See CincyReHackathon where I am giving myself a do over.

Applying Google Prediction API to GymboreeDeals

2011-04-26T00:00:00+00:00

I won’t go into all the details behind GymboreeDeals, but it is operated by my wife, and involves manually categorizing content. Maybe Google Prediction API can help speed up her work, or (if the gods of Olympus are smiling upon her) eliminate aspects of it.

With my newly acquired knowledge of Google Prediction API it was trivial to develop a prediction model.

Fortunately (for my data needs), she has manually categorized more than 10,000 eBay auctions since September (~50/day) which provides a fairly good example set for Google’s machine learning algorithms.

To review, the process to create the trained model was:

Generate a CSV file with the training data (export from SQL, normalizing the text by removing punctuation and converting all text to ALL CAPS)
Upload file to Google Storage
Invoke training process

With that, I have a trained model that I can invoke:

./oauth-predict.sh gymb/prediction_models/genders \
  "\"NWT GYMBOREE BURST OF SPRING LOVE SWEET STRAWBERRY SZ 2\""

Which returns:

{
  "kind": "prediction#output",
   "id": "gymb/prediction_models/genders",
   "selfLink":
     "https://www.googleapis.com/prediction/v1.2/training/gymb/prediction_models/genders/predict",
   "outputLabel": "Girl",
   "outputMulti": [
     {
       "label": "Girl",
       "score": 0.999855
     },
     {
       "label": "Boy",
       "score": 1.45E-4
     }
   ]
 }

“Sweet” “Love” “Strawberry” were probably dead giveaways. How about a tougher one…

./oauth-predict.sh gymb/prediction_models/genders \
  "\"NWT GYMBOREE WILD SAFARI BASIC ORANGE KNIT SHORTS 4 4T\""

Which returns:

"outputLabel": "Boy",
"outputMulti": [
  {
    "label": "Girl",
    "score": 0.315243
  },
  {
    "label": "Boy",
    "score": 0.684757
  }
]

Much closer score, but identified “Boy” correctly. Very cool. Must have been the word “Wild”.

Integrating into Rails project

Let’s try out the Ruby wrapper written by peeps @ Google. I add to the project’s Gemfile:

gem 'google_prediction', '0.0.1'

I run bundle, then try to startup the Rails app. (Rails 3, and Ruby 1.9.2 for the new hash syntax.)

Did I mention that the Google Prediction API released version 1.2 last month?

The app won’t start, no such file to load -- jcode which means the gem isn’t updated for 1.9 compatibility.

And looking at the README, it’s using old ClientLogin methodology, not OAuth. So the Ruby wrapper is for API version 1.1, not 1.2.

Am I dead in the water? No.

I just need to write my first gem :)

Google Prediction API First Looks

2011-04-24T00:00:00+00:00

Sitting around on a Sunday night, decided to finally take a little dive on something that’s had my eye for a while, the Google Prediction API.

Visited the Google APIs console and turned on the Google Prediction API.
Needed to sign up for Google Storage account - this seemed discouraging, since it was a Google Docs waiting list, and I figured I’d be dead in the water tonight, maybe waiting forever. I wrote as persuasive an application as I could :)
I noticed that they metioned they were going to turn on billing in early 2011, pricing was only $10/month. From the console, there was a “billing” link that I could enable. So I decided to enable billing, hoping that would help with Google Storage account issue.
Checking my email, I had already been emailed a Google Storage invite. Not sure if it was automatic or my application caught their eye (I suspect the former) but the hurdle was jumped. Woot!

Hello Prediction

I followed the hello world tutorial that Google provides. I won’t cut-and-paste all the steps, since the tutorial is well-written, but here are some notes from the process.

I needed to install gsutil - pretty easy; instructions here.
I needed to create a Google Storage access key.
Can’t use bucket ‘examplebucket’. I used ‘play-prediction’ and it allowed me to create bucket and then I copied the file up using gsutil cp ./language_id.txt gs://play-prediction/prediction_models/languages.
Save your Google API Access Key (found on the API Console through the “API Access” link on the left, at the bottom of the page “Simple API Access”, to file googlekey.
I copied oauth-train.sh, oauth-check-training.sh, and oauth-predict.sh to my local project folder, and chmod 755 on the to make them executable.
I was able to train using ./oauth-train.sh play-prediction/prediction_models/languages command.
./oauth-check-training.sh play-prediction/prediction_models/languages yielded the expected result, training accuracy was 0.95.
./oauth-predict.sh play-prediction/prediction_models/languages "\"Sitting around on a Sunday night, decided to finally take a little dive on something that’s had my eye for a while, the Google Prediction API.\"" was interpreted as English correctly.
./oauth-predict.sh play-prediction/prediction_models/languages "\"El sol se levantará mañana y quién sabe lo que traerá la marea?\"" was interpreted as Spanish correctly.
./oauth-predict.sh play-prediction/prediction_models/languages "\"La gloire est éphémère, mais l'obscurité est éternelle.\"" was interpreted as French correctly.

I noticed the engine didn’t perform as well on short phrases, to be expected.

Closing

Very cool system, took me about an hour to get through the tutorial.

Mission accomplished, Google Prediction API is much less abstract in my mind now.

The invisible forces that shape us

2011-04-08T00:00:00+00:00

At all times, there are two forces tugging on us:

The current moment, including our current emotional and physical state.
Our long term view of the world, which shapes what things mean to us.

If we bring this tug-of-war into our active conciousness, we can focus ourselves to towards the achievement of our goals.

We are making three decisions every moment of our life:

What am I going to focus on? On the past? The present? The future? Myself? Others?
What does it mean? Is it the end? Is it the beginning? Is it punishment? Is it a reward?
What am I going to do? Give up? Move forward?

Token input for multi-select

2011-04-03T00:00:00+00:00

Great Railscast by Ryan Bates showing how to do a Facebook-like multiselect in Rails 3 with very little server side work.

View RailsCast