tag:blogger.com,1999:blog-5907747129489064192024-03-17T20:01:57.579-07:00Pete MironNeither a conductor nor an engineer. <br>Follow me on twitter: <a href="http://twitter.com/petemiron">@petemiron</a>pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.comBlogger19125tag:blogger.com,1999:blog-590774712948906419.post-75641119913431439892013-04-23T08:34:00.001-07:002013-04-23T08:34:44.773-07:00<h2>
Building Excellent Things - Redirect after Login</h2>
<div>
There are many details that are easy to overlook when building web products. These details are opportunities to either delight or annoy.</div>
<div>
<br /></div>
<div>
When you build a web app that requires a login, users will often share links inside the app with other users. When those links are shared, and a different user clicks on them, the expectation is that they land on what was referenced by the link. This becomes slightly more complicated if the user is not logged in. If you must present a login screen, the correct detail is to log the user in and then redirect to where the user originally intended to go (delightful) -- assuming appropriate authorization (eg. don't let me share my bank account page with others).</div>
<div>
<br /></div>
<div>
Otherwise, the user is required to go back and click the link twice (annoying).</div>
<div>
<br /></div>
<div>
Don't make your users click the link twice.</div>
pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com0tag:blogger.com,1999:blog-590774712948906419.post-62317253206609900262012-11-24T09:55:00.002-08:002012-11-24T09:55:57.897-08:00Book Notes: Moonwalking with EinsteinSince my time working on education software and thinking about the best ways to educate my daughters, I've been curious about the role of memory and memorization drills. A couple months ago, I read Bill Gates's recommendation of <a href="http://www.thegatesnotes.com/Books/Personal/Moonwalking-with-Einstein" target="_blank">Moonwalking with Einstein</a>, the topic again piqued my interest.<br />
<br />
The story tracks journalist Joshua Foer on his quest to move from covering memory championships to becoming a memory champion himself. The book provides a great source of information regarding history and methods of memory training from the <a href="http://www.utexas.edu/research/memoria/Ad_Herennium_Passages.html" target="_blank">Rhetorica Ad Herrenium's</a> few passages on memory to modern day commercial memory training master, <a href="http://en.wikipedia.org/wiki/Tony_Buzan" target="_blank">Tony Buzan</a>.<br />
<br />
While <i>Moonwalking</i> reminds me that I need to work on furnishing my Memory Palace and increasing my working memory, I find it's utility as a reminder that becoming an expert relies on "<a href="http://www.uvm.edu/~pdodds/files/papers/others/everything/ericsson2007a.pdf" target="_blank">deliberate practice</a>." Experts "develop strategies for consciously keeping out of the autonomous stage while they practice by doing three things: focusing on their technique, staying goal-oriented, and getting constant and immediate feedback on their performance." These three things are incredibly important for me as a manager of a team, a programmer, an amateur runner and an extremely amateur classical guitarist. What am I doing every time I spend time on an activity to actually get better at that activity? Running (run faster or longer every time I run) and guitar (progressively harder studies moving from 1/2 speed to full speed) are actually pretty straightforward.<br />
<br />
As for managing a team, what am I doing every day to make sure I am becoming a more effective manager and leader? How am I getting feedback every day? Much to think about here... what are your thoughts?pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com0tag:blogger.com,1999:blog-590774712948906419.post-32234923136469959752012-09-05T20:53:00.000-07:002012-09-05T20:57:12.073-07:00I'm probably not running the marathon in 2012.But, if you'd like to, you should sign up <a href="https://campinteractive.wufoo.com/forms/teaminteractive-2012-nyc-marathon-application/" target="_blank">here</a>. It's for the the team I raised money with last year, Camp Interactive, who works to teach kids from the Bronx leadership skills and technology through a unique blend of outdoor activities and technical training.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVZtRt5Q5yMI25YfzzqYGVIhPWhNefvwuPlQDXl1kfjnBqtHidMLO7c__Cv_V74pFsqxDlgzHETizFBfCZkwCGdbOFAfeIUYVuxeu6ZAQde94aE5JoOp6XiJuhaUvxo6kUE8a35A6byqY/s1600/IMG_2222.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="298" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVZtRt5Q5yMI25YfzzqYGVIhPWhNefvwuPlQDXl1kfjnBqtHidMLO7c__Cv_V74pFsqxDlgzHETizFBfCZkwCGdbOFAfeIUYVuxeu6ZAQde94aE5JoOp6XiJuhaUvxo6kUE8a35A6byqY/s400/IMG_2222.JPG" width="400" /></a></div>
<br />
I'm pretty sure you can better my time of 5:40. Seriously, how many opportunities do you have to come in 42,759th place in a foot race and still beat 4,421 people?<br />
<br />
I'm not going to lie, I'm very, very tempted to run again to try to beat that time, but this year is probably not the year for me. Maybe 2013 is the year for me to have a Paul Ryan-esque marathon pace.pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com0tag:blogger.com,1999:blog-590774712948906419.post-4428508112695726472012-09-03T17:27:00.002-07:002012-09-05T20:56:53.682-07:00Where to find all the great recruits?<div>
<h3>
Engineers I'd love to work with again</h3>
I've recently begun the recruiting process at bitly (<a href="http://bit.ly/PXq6eQ" target="_blank">where I'm looking for just about every type of engineer</a>). To help focus my recruiting efforts, I decided to go through the top 25 engineers I've worked with in the past that I'd be most likely ask to work with me again (or that I have actually asked to work with me again).<br />
<br />
The data for my top 25 seem to split in half along 2 major categories for inexperienced (< 3 years professionally) hires and experienced hires (> 3 years).<br />
<br />
<h3>
Inexperienced Hires</h3>
<div>
From the data on my top 25, the majority of inexperienced hires come from college recruiting with the remainder being internally developed. Internally developed hires are folks that were brought into the company to do some other job and later became engineers. Internally developing engineers requires time and effort from your more experienced folks, but I've seen it frequently pay off. For the cost of training time, you get engineers who you trained with your team's habits, and who have deep understanding of the business. We added 50% to the Knewton technology team by internally developing a couple of folks, at a time when the team only consisted of 4 people and we hadn't yet launched. If we could do it then, you can probably do it now. If you consider the time you spend recruiting candidates through most other means, the time spent training people you already know are great for the company is probably a wash.</div>
<div>
<br /></div>
<div>
College recruits mostly came from on-campus recruiting visits, and an email directly from a dean. I've had the most luck with CMU, but have had a surprising number of good engineers come out of the University of Chicago, Harvard and Cornell. The challenges of college recruiting are the schedule. You need to be ready to interview and get on-campus when the school has openings, you also need to be able to wait 3-5 months for your newly hired recruit to start work.<br />
<br /></div>
<h3>
Experienced Hires</h3>
<div>
The experienced hires showed a similar nearly 50/50 split between referrals and internal recruiters. A referral is a recruit that someone already on your team recommended. The distinction between internal recruiters vs. recruiters or outside recruiters is that they are hired on-site and typically salaried. My top 25 is skewed towards internal recruiters, but that's most likely due to the fact that I primarily used internal recruiters throughout my career.<br />
<br />
Experienced hires have the benefit of a track record of delivery and can usually start within 4 weeks of offer. The downside is that you'll often spend a lot more time going through interviews of people that almost meet your criteria.<br />
<br /></div>
<h3>
A pie chart!</h3>
</div>
<div>
as embarrassing as this is ... I've included a pie chart (even better, made in excel).</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg95FPyaO8UEVRJ9dV-4MaRjPk1zOrSjJg4C0298X4TeCIWxjdXam_9o13QI2fyF1HemBoHRY0epu52iFWod14_du7qVExG8vfecNYr4dterd5bFmOYvjtFnwWEc5PnxchpSXFGXVT9yw8/s1600/recruiting_sources.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><br /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg95FPyaO8UEVRJ9dV-4MaRjPk1zOrSjJg4C0298X4TeCIWxjdXam_9o13QI2fyF1HemBoHRY0epu52iFWod14_du7qVExG8vfecNYr4dterd5bFmOYvjtFnwWEc5PnxchpSXFGXVT9yw8/s1600/recruiting_sources.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="332" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg95FPyaO8UEVRJ9dV-4MaRjPk1zOrSjJg4C0298X4TeCIWxjdXam_9o13QI2fyF1HemBoHRY0epu52iFWod14_du7qVExG8vfecNYr4dterd5bFmOYvjtFnwWEc5PnxchpSXFGXVT9yw8/s400/recruiting_sources.png" width="400" /></a></div>
<div>
<br />
<br />
<h3>
What's this all mean? </h3>
</div>
<div>
Here's what I'm recommending for myself:</div>
<div>
<ol>
<li>Ask every member of the team, all the time, if they know any great engineers we should speak with.</li>
<li>Focus on college recruiting and <a href="http://petemiron.blogspot.com/2012/01/hire-interns.html" target="_blank">intern hiring</a> to build the pipeline for the spring.</li>
<li>Improve the filtering process with recruiters to minimize time invested.</li>
<li>Look for opportunities to develop team members outside the engineering team. </li>
</ol>
<h3>
How were the people you'd like to work with again recruited on to your team?</h3>
</div>
<div>
I'd love to hear other tips and tricks for finding great folks.</div>
pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com4tag:blogger.com,1999:blog-590774712948906419.post-48242362418300193742012-01-10T16:57:00.000-08:002012-09-05T20:57:32.620-07:00Hire Interns<h1>
Get real work done and build a recruiting pipeline</h1>
<br />
Hiring great engineers is really hard. In his book, <a href="http://www.amazon.com/gp/product/1590598385/ref=as_li_ss_tl?ie=UTF8&tag=pete06-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=1590598385">Smart and Gets Things Done</a>, Joel Spolsky estimates great engineers are actively looking for work 4 times throughout their careers. The only time you're guaranteed that great engineers are actively looking is when they're in college.<br />
<br />
By hiring Interns, you're hoping to find the next great engineer before someone else does.<br />
<br />
Before we get into the mechanics of hiring interns, let's establish that this is a give and take relationship. There is some work you'll need to do with interns, but you'll also get plenty in return.<br />
<br />
<h2>
You will need to mentor your interns to:</h2>
<ul>
<li>work on a team</li>
<li>create production quality code</li>
<li>be self sufficient. Show them that all the answers are in the machine sitting in front of them</li>
</ul>
<h2>
In return, you get:</h2>
<ul>
<li>a pipeline for filling future engineering positions</li>
<li>practice for yours and your team's mentoring skills</li>
<li>some work done by a fresh, eager engineer</li>
<li>to learn a few things yourself</li>
</ul>
<h1>
How to recruit</h1>
There are 3 common ways to recruit interns: setting up or using an internship program, going through campus career centers or going through faculty to find their best students.
<br />
<ol>
<li><h2>
Setup or use an internship program</h2>
<div>
Interns that came through internship programs helped drop all of our base page load times under 1 second through instrumentation and MySQL configuration improvements. We've also had interns help drastically reduce administrative times for our content developers by improving page load through SQL query optimization and adding bulk update features.</div>
<div>
Internship programs typically use some prestige to attract top students to apply.<br />
<br />
Some NYC specific examples of internship programs are the <a href="http://nycturingfellows.org/" target="_blank">NYC Turing Fellows</a> and <a href="http://hackny.org/" target="_blank">Hack NY</a>.
</div>
<div>
You can also setup your own, like <a href="http://www.fogcreek.com/Jobs/SummerIntern.html" target="_blank">FogCreek has done</a> and pointing listings on Craigslist to your internship program.
</div>
</li>
<li><h2>
Go to the career center</h2>
<div>
We found one intern through an on-campus recruiting trip up to Harvard. He setup a state-of-the-art code, build and release system using GIT, Gerrit and Jenkins, which we’ve subsequently ported our code to.
<br />
<br /></div>
<div>
Pick the top schools that you’re interested in recruiting from. If you’re not sure, look at the <a href="http://www.usnewsuniversitydirectory.com/graduate-schools/sciences/computer-science.aspx">US News & World Reports rankings</a>, or where most of your team is from.<br />
<br /></div>
<div>
Post your open engineering internships to the career center, schedule on-campus interviews and infosessions, or attend a career fair.</div>
<div>
<br />
Over time, you’ll begin to develop a taste for how hard it is to recruit interns from certain schools and which schools seem to attract the interns that end up being the most productive for your needs.</div>
<div>
<br />
I’ve had mixed results with career fairs. Generally, my return has been better at on-campus interviews and through job postings than at career fairs.</div>
</li>
<li><h2>
Contact Faculty</h2>
I hired an intern who designed and implemented our highest converting trial to buy flow for nearly 18 months by simply letting an assistant dean know I was looking for interns. He let some of his top students know we were looking.<br /><br />To contact faculty, use the list of target schools and your alumni network to locate faculty you're interested in speaking to. Send them an e-mail, tell them what you're working on and ask if they'd be willing to connect you to their top CS students.</li>
</ol>
<h1>
Hiring Process</h1>
<ol>
<li><h2>
Review and sort resumes as normal</h2>
Your resume sorting process should be the same as the one you use to hire full-time engineers -- unless your normal screening process screens out anyone with fewer than 5 years of experience.</li>
<li><h2>
Interview remotely</h2>
Make sure your team is capable of interviewing remotely. We typically use <a href="http://www.skype.com/" target="_blank">Skype</a> and <a href="http://willyou.typewith.me/" target="_blank">Etherpad</a> for engineering interviews. Your interviews should be similar to what you use for full-time engineers -- you may want to skip questions about development process (test first, source code management, etc.)<br /> </li>
<li><h2>
Pay them</h2>
Your interns are going to do real work, you should pay them for it. Most internships seem to pay between $3,500 and $5,000/year.</li>
<li><h2>
Close on-site</h2>
Once you've narrowed down your finalists, bring the ones you'd like to make an offer to on-site. Sell them on the company and your location. Make sure they would enjoy spending the summer at your location. Take them out for lunch or dinner with the team. These are valuable resources and the best ones will have multiple offers; convince them your company is right for them. </li>
</ol>
<br />
<h1>
What to do once they accept</h1>
<ol>
<li><h2>
Help them find a place to live</h2>
<br />If your interns don't live in your city, find them a comfortable, affordable place to live. In NYC, <a href="http://www.nyu.edu/summer/housing/" target="_blank">NYU housing</a> is a good option.</li>
<li><h2>
Give them REAL work to do</h2>
<div>
Don't give your interns a side project that you don't care about. Give them something that needs to get done this summer. Give them something that the rest of the team would need to deliver if the intern wasn't there. Assigning an intern unimportant side-projects that the rest of the team doesn't care about will result in an unproductive internship for both you and the intern.</div>
As a reminder, here are all the things interns at Knewton have completed:
<br />- upgraded our build and release process
<br />- produced our top converting landing page
<br />- improved site performance
<br />- became my first hire at Knewton (8 years after he interned for me)
</li>
</ol>
<h1>
Go hire interns ... get tomorrow's great engineers today</h1>
<br />
Great interns will become great engineers one day. If you find the right interns, take the time to mentor them, and give them an opportunity finish a project that matters. Maybe one day they'll be a valuable full-time engineer.<br />
<br />
If you're an intern looking for a job, Knewton is hiring interns! <a href="http://jobs.knewton.com/apply/6WpWPK/Tech-Internship.html">Click here to apply.</a>pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com1tag:blogger.com,1999:blog-590774712948906419.post-41896141705977507752011-11-01T18:45:00.000-07:002012-09-05T20:56:35.015-07:00From 262 to 26.2I'm terribly nervous about running the NYC Marathon this Sunday, Nov. 6, 2011. I started training the first week of June. Over the past 5 months, I've ran over 350 miles. 70 miles/month. I've destroyed 1 pair of sneakers. Ran 15 miles in 100 degree heat. Ran across the Golden Gate Bridge. Ran on the Miami Beach Boardwalk at Sunrise. Just this weekend ran in 3" of slush.<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhV_4Z6DKgfUz05s5ErvhokSuKfewOFrdEVVzDRutrzEnnWX9A4e-nmvaFwSzH8z2WFimLTMH1m71_M5SsZKzkUY-1VgKDNzMuXGARSv_V7Kp5JnTihuzLH4w0jn_HZOFamHwiB6gWse80/s1600/IMG_2084.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhV_4Z6DKgfUz05s5ErvhokSuKfewOFrdEVVzDRutrzEnnWX9A4e-nmvaFwSzH8z2WFimLTMH1m71_M5SsZKzkUY-1VgKDNzMuXGARSv_V7Kp5JnTihuzLH4w0jn_HZOFamHwiB6gWse80/s320/IMG_2084.jpg" width="239" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">18 Miles with Josh Anish in SF.</td></tr>
</tbody></table>
And for what?<br />
<br />
Nearly a year ago, rushing to the PATH station, I felt my stomach slump over my ice cold belt buckle. "Holy Shit! How did I get so fat?" Early on in our relationship, my wife and I passed a man in the mall whose belly stuck out from under the bottom of his shirt over his pants. We agreed we would never let ourselves to go like that. I was not keeping up my end of the bargain.<br />
<br />
This is how I decided I needed to do something? Neither fear of diabetes nor fear of heart disease had been enough to scare me into fixing my diet and exercising. This simple, all too fleshy sensation was the trigger. I knew I needed a change.<br />
<br />
Shortly after my epiphany, <a href="https://twitter.com/#!/TFERRISS" target="_blank">Tim Ferriss</a> released <a href="http://www.fourhourbody.com/" target="_blank">4 Hour Body</a>. The most important aspect of the book was that it illustrated how other over-worked techies could fix their diets by making simple, repeatable changes. He also got me to think of my diet as a science project instead of something to dread. Cheat day was a god send and got me through a couple tough weeks of travel. At my heaviest, I was 268, at my lightest, I'm down to 222 (really want to be ~210-215). I used to rush off to the train in the morning, and grab a 600 calorie breakfast sandwich, I now make myself a couple eggs with veggies and turkey bacon every morning. It doesn't take me anymore time. And I lost 40 pounds.<br />
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjURvFUETB4O8CIuEYOr6ewFSkOrZ_Hag9qNt9ZdAQOf2cnSHflAiohLpuket_za1SBG0sYbESbSOvlDcCpirMy_RC99CsPw2QsNh9IAON08lY2tfzS3v3Bfy3LJejvRCaxyX3xfADeCYo/s1600/IMG_1905.JPG" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjURvFUETB4O8CIuEYOr6ewFSkOrZ_Hag9qNt9ZdAQOf2cnSHflAiohLpuket_za1SBG0sYbESbSOvlDcCpirMy_RC99CsPw2QsNh9IAON08lY2tfzS3v3Bfy3LJejvRCaxyX3xfADeCYo/s320/IMG_1905.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sunrise run in Miami.</td></tr>
</tbody></table>
Then, in March, <a href="http://twitter.com/naveen" target="_blank">@naveen</a> from Foursquare retweeted a message from <a href="http://twitter.com/dens" target="_blank">@dens</a> about <a href="http://campinteractive.org/" target="_blank">Camp Interactive</a> looking for marathon runners for their team in the 2011 NYC Marathon. I committed to run a marathon the year before as my New Year's Resolution, because I'm an idiot. Nevertheless, I filled out the form, I was honest that I didn't think I could run 26.2 miles or raise $5,000.<br />
<br />
In June, Camp Interactive accepted me anyways.<br />
<br />
I hadn't been exercising or running regularly for over a year. I had no idea how I was going to do run a marathon. Fortunately, as usual, our VP, Product Development had a plan for me. Brian setup some simple, achievable goals for my monthly distances. I also found a first-time marathon plan with the minimum amount of running during the week.<br />
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgY1XTGpKAzFdPJ51zYXYglWMYD7E-g0AjMDTY0lDnp7BgztxHmNMpo3xB1m4SJtYZvj8hKTKCKN_ufnutNjCgaM882sXU2OrGxP09F2_5smx3wk6cf_7N8r2ersC8B1FrEJ2KkYy1L3_M/s1600/IMG_2019.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgY1XTGpKAzFdPJ51zYXYglWMYD7E-g0AjMDTY0lDnp7BgztxHmNMpo3xB1m4SJtYZvj8hKTKCKN_ufnutNjCgaM882sXU2OrGxP09F2_5smx3wk6cf_7N8r2ersC8B1FrEJ2KkYy1L3_M/s320/IMG_2019.jpg" width="240" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Final run in the snow?!?!</td></tr>
</tbody></table>
Every run was a new, faster time, or a new longer distance. I learned how to hydrate, when to eat Goo, how to master a negative split (fingers crossed for getting any/all of those right this Sunday), but most of all, I learned that I my body is an amazing, flexible (figuratively) piece of machinery willing to be pushed well beyond my mind's breaking point.<br />
<br />
In less than a year, I've dropped 40 pounds and have run at least 20 miles. Fingers crossed for making it the full 26.2 this Sunday. Thank you all for your support and donations (not too late for <a href="http://bit.ly/runpeterun" target="_blank">more</a>), especially my amazing wife who's put up with me spending 1 day every weekend for the past 5 months disappearing in the woods for 2-4 hours and coming home to ice my knees and pass out on the couch.<br />
<br />
For all of my friends in or near the city, I'd love to hear your cheers on Sunday. My bib number is: <span class="Apple-style-span" style="font-family: inherit;"><b>44712</b>.</span><br />
<span class="Apple-style-span" style="font-family: inherit;"><br /></span>
Best of luck to all my fellow runners -- esp. Team Interactive and fellow Knerds: Ken and Jen!pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com2tag:blogger.com,1999:blog-590774712948906419.post-16991274350692996312011-02-14T04:57:00.000-08:002011-02-14T04:57:04.932-08:003 things you forgot about when adding a new language to your runtime environment.Many engineers seek to use the right tool for the job (and they should). Most tech managers get a bit freaked out at the idea of endless language proliferation (who cares?). To help your tech managers be a little less freaked out and ensure you never ever again get stuck in a J2EE stack, there are 3 things that you need to keep in mind when adding Python, Ruby, Haskell or Lisp to your runtime environment.<br />
<br />
<ol><li>The MySQL bindings will suck.</li>
<li>The initial appserver config will be wrong.</li>
<li>The exception handling and reporting will be useless.</li>
</ol><div><table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzseF7Tq4vaGg12uTALfa4nJ7Z29xv_LCT2lGi7tlBO3pWBmxlgTQtIbNs00hZp6PywXjgILS5OF7y4fshzhP8V8fP442e3oHB8SAlmymEtsRd7MZunU6fUgVvnCUhK0xoEY1Vb4NTM4M/s1600/Java_Duke.gif" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzseF7Tq4vaGg12uTALfa4nJ7Z29xv_LCT2lGi7tlBO3pWBmxlgTQtIbNs00hZp6PywXjgILS5OF7y4fshzhP8V8fP442e3oHB8SAlmymEtsRd7MZunU6fUgVvnCUhK0xoEY1Vb4NTM4M/s1600/Java_Duke.gif" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">We'll miss you too Duke.</td></tr>
</tbody></table><h2>The MySQL bindings will suck.</h2></div><div>I'm 0 for 3 (Java, Ruby, Python) in implementing a new language and having anything sort of like smart idle connection timeout handling. I've needed to implement wrappers in Java (hopefully this isn't still the case), patch the Ruby MySQL gems and have tried to figure out the one right Python wrapper for ensuring my app doesn't need a restart once connections are timed out by the server. These are all solvable. Just make sure you test for these issues and resolve them prior to production deployment.</div><br />
<div><h2>The initial appserver config will be wrong.</h2></div><div>Most languages run in some sort of appserver, typically that's wsgi or some mod_ apache plug-in. But, some appservers have strong language preferences - esp. in the Java world. Make sure you understand how your new language is going to run in a new appserver - multithreaded or multiple processes? how many threads or processes do you need? are database connections pooled?</div><br />
<div><h2>The exception handling and reporting will be useless.</h2></div><div>On our team, at <a href="http://www.knewton.com/">Knewton</a>, we email all engineers exceptions and stack traces. We've needed to implement this for both Ruby and Python. For both, we needed to add in important information for debugging - request params, headers, along with a usable stack trace. One other common issue we've encountered is that all libraries for connecting to external resources (curl) have useless default errors. Most of these libraries do not tell you explicitly what they were trying to connect to when the exception has occurred - "Destination Unreachable" is not useful; which destination (hostname, ip, port)? We've patched curb in Ruby and are currently patching Python to let us know what we were unable to connect to.</div><br />
These are the most common issues I've seen when adding a new language to an environment. Are there any I'm missing (or conveniently forgetting about)? If you can think of any, drop me a comment below.pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com0tag:blogger.com,1999:blog-590774712948906419.post-48802937174004212332011-02-10T19:54:00.000-08:002011-02-10T19:54:30.155-08:00How-To: Turn a .pdf to plaintext using Google Docs (even if it's an image)Every once in awhile, I'll receive a large set of documents that I need to quickly read and categorize. Some day I hope to use NLP for those categorizations, but I still have much to learn. One document format that I always struggle with converting on Mac OS X with Python is .pdf. But, not anymore...<br />
<br />
Last year, google docs introduced the <a href="http://googledocs.blogspot.com/2010/06/optical-character-recognition-ocr-in.html">ability to do optical character recognition (OCR</a>). Using a tiny bit of Python, I was able to upload a document and pull it back down as a plain text file. Here's how.<br />
<br />
<h2>Step 1:</h2>install <a href="http://code.google.com/p/gdata-python-client/downloads/list">gdata python libraries</a><br />
<br />
<h2>Step 2:</h2>create pdf2txt.py<br />
<pre>import os.path
import gdata.data
import gdata.docs.client
import sys
if __name__ == "__main__":
# read in the pdf file
f = open(sys.argv[1])
# setup your google docs client
client = gdata.docs.client.DocsClient(source='pdf2txt')
client.ssl = True # Force all API requests through HTTPS
user = 'YOURUSERNAME@gmail.xxx'
password = 'TE$T'
# login to Google Docs
client.ClientLogin(user, password, client.source)
# create the media source object for upload
ms = gdata.data.MediaSource(file_handle=f, content_type="application/pdf", content_length=os.path.getsize(f.name))
# upload your pdf
entry = client.Upload(ms, f.name, folder_or_uri="https://docs.google.com
/feeds/default/private/full?ocr=true")
# get the file as text (the ext sets the format, can also be .doc)
client.Export(entry, f.name + ".txt")</pre><h2>Step 3:</h2>Run your new script:<br />
<pre>> python pdf2txt yourpdf_file.pdf</pre>this will add a file to the directory you ran python from and create a file named: <br />
<h2>Step 4:</h2>check out your file: <pre>yourpdf_file.pdf.txt</pre><br />
Use my code at your own risk, feel free to submit even better code that uses getopts() for command line args.pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com16tag:blogger.com,1999:blog-590774712948906419.post-53308513654266963492011-01-24T11:40:00.000-08:002011-01-24T13:50:51.287-08:00How-To: SMS Q+A with Twilio, App Engine, and CherryPyWhen learning a new API or programming language, I often get a bit nostalgic for the first programs I wrote in BASIC. Those programs looked a lot like this:<br />
<div class="p1"><code><br />
10 INPUT "What is your name? "; U$<br />
20 PRINT"Hello, ", U$<br />
</code></div><div class="p1"><div style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img height="149" src="http://www.vintage-computer.com/images/vic20.jpg" width="320" /></div><br />
<br />
To illustrate the simplicity of <a href="http://twilio.com/">Twilio</a>, Google's <a href="http://appengine.google.com/">App Engine</a> and <a href="http://www.cherrypy.org/">CherryPy</a>, I'll take this simple program and show you how to build it for an SMS interface.<br />
<br />
Twilio allows you to automate placing and receiving phone calls, and placing and receiving SMS-messages (or TXTs). For this tutorial, we'll focus on placing and receiving SMS-messages. App Engine is a way to inexpensively (free to start) run web applications in Google's cloud. CherryPy is an extremely simple HTTP server for Python. CherryPy has nominal configuration and reminds me a *lot* of the simplicity of the original Java Servlet specification.<br />
<br />
Before we get started, you need to signup for <a href="https://www.twilio.com/try-twilio">Twilio</a> and <a href="http://appengine.google.com/">Google App Engine</a> accounts. Both are free and easy to setup. Once you've done that, swing on back, and we'll walk through the rest.<br />
</div><div class="p1"><h1>The Spec</h1><ul><li>A user sends an SMS to your sample Twilio account phone number with the text "?".</li>
<li>Twilio sends that request to your new Google App Engine App.</li>
<li>Google App Engine dispatches the request to your CherryPy app.</li>
<li>CherryPy reads the body of the request.</li>
<li>If the request field <span style="font-family: 'Courier New', Courier, monospace;">Body</span> is "?", CherryPy responds with <span style="font-family: 'Courier New', Courier, monospace;">"What is your name?"</span> </li>
<li>If the request field <span style="font-family: 'Courier New', Courier, monospace;">Body</span> is not "?", CherryPy responds with <span style="font-family: 'Courier New', Courier, monospace;">"Hello, [Body]."</span></li>
<li><span style="font-family: 'Courier New', Courier, monospace;"></span>The user that sent the original SMS message receives the response on their phone.</li>
</ul><h1>The Development Process</h1>First, we're going to get a simple app running in CherryPy on your local machine. Then, we'll get it running on App Engine locally, and in the cloud. Finally, we'll connect Twilio to App Engine and you can watch it all in action.<br />
<br />
<br />
<h1>Get your app running in CherryPy</h1>First, create a directory for your app:<br />
<code>$ mkdir twilio_qa</code><br />
<br />
Download <a href="http://download.cherrypy.org/cherrypy/3.1.2/">cherrypy</a><br />
<br />
Unzip cherrypy in the <span style="font-family: 'Courier New', Courier, monospace;">twilio_qa</span>directory.<br />
<br />
Then, use your favorite editor to create <span style="font-family: 'Courier New', Courier, monospace;">twilio_qa.py</span>.<br />
<br />
<pre>#!/usr/bin/env python
import cherrypy
class TwilioQA:
# the main entry point, this is the method that will get called
# when someone hits your server.
def index(self, Body="?", **kwargs):
if (Body == "?"):
return self.instructions()
else:
return self.wrap("Hello, %s.\r\nShall we play a game?" %Body)
# print out the instructions
def instructions(self):
s = "What is your name?"
return self.wrap(s)
# helper function to wrap response with twilio SMS Response XML
def wrap(self, response):
s = """<?xml version="1.0" encoding="UTF-8" ?>
<Response>
<Sms>%s</Sms>
</Response>
""" % (self.trim(response))
return s
MAX_SMS_LENGTH = 160
# remove all chars after 160, otherwise Twiilo will reject the
# response.
def trim(self, s):
return s[0:self.MAX_SMS_LENGTH]
# tell cherrypy which methods it can call.
index.exposed = True
# setup cherrypy
app = cherrypy.tree.mount(TwilioQA(), "/")
cherrypy.quickstart(app)</pre>Make <span style="font-family: 'Courier New', Courier, monospace;">twilio_qa.py</span> executable and run it.<br />
<br />
<pre>$ chmod +x twilio_qa.py
$ ./twilio_qa.py</pre><h1><a href="" name="test_your_app">Test your app</a></h1>With your app running, open a browser and go to <span style="font-family: 'Courier New', Courier, monospace;"><a href="http://localhost:8080/">http://localhost:8080/</a></span>.<br />
<br />
The page should return the instructions:<br />
<br />
<pre>What is your name?</pre>Add the parameter for the body to see how it works with a response <span style="font-family: 'Courier New', Courier, monospace;"><a href="http://localhost:8080/?Body=Pete">http://localhost:8080/?Body=Pete</a></span>. # Note case for form fields matters, cherrypy automatically maps query paramaters to variables in your <span style="font-family: 'Courier New', Courier, monospace;">index</span> method.<br />
<br />
You should see the response:<br />
<br />
<pre>Hello, Pete. Shall we play a game?</pre>Viewing the source will show you the XML that you will be sending to Twilio in just a little bit.<br />
<br />
The cherrypy setup is really simple. You import cherrypy, mount the default URL "/" to your TwilioQA() class, and run the server with the quickstart() method.<br />
<br />
<h1>Run your app in App Engine</h1>Download and install the <a href="http://code.google.com/appengine/downloads.html#Google_App_Engine_SDK_for_Python">App Engine SDK for Python</a>.<br />
Create an <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">app.yml</span> in your <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">twilio_qa</span> directory.</div><pre>application: twilio-qa-CHANGE-ME
version: 1
runtime: python
api_version: 1
handlers:
- url: /.*
script: twilio_qa.py
</pre><br />
<p>In <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">twilio_qa.py</span>, import wsgiref.handlers to work with appengine:<br />
<pre>...
import cherrypy
import wsgiref.handlers
...</pre></p>In <span style="font-family: 'Courier New', Courier, monospace; white-space: normal;">twilio_qa.py</span>, comment out <span style="font-family: 'Courier New', Courier, monospace;">quickstart</span> and add the App Engine handler:<br />
<div><pre>...
app = cherrypy.tree.mount(root, "/")
# cherrypy.quickstart(app)
wsgiref.handlers.CGIHandler().run(app)
</pre></div><p>Run your app in app engine's localserver:<br />
</p><pre>$ dev_appserver.py --port=8080 ../twilio_qa/
</pre><div><p>Follow the steps in <i><a href="#test_your_app">Test your app</a> </i>and scroll back here when you're done.<br />
</p><h1>Deploy on App Engine</h1><span class="Apple-style-span" style="font-family: inherit; white-space: normal;">Now that you've tested locally, you can deploy to the cloud in a few easy steps.</span><br />
<span class="Apple-style-span" style="font-family: inherit; white-space: normal;"><br />
</span><br />
Create a new application at <a href="http://appengine.google.com/">appengine.google.com</a>:<br />
<ol><li>Click [Create Application].</li>
<li>Choose an [application identifier] (you'll use this for the application field in app.yml later). This needs to be unique across the entire appspot.com domain. Good luck :-)</li>
<li>Create a title.</li>
<li>Update your app.yml, set your application identifier as the value in application.</li>
<li>Deploy your app by typing:<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">$ appcfy.py update ../twilio_qa/</span></li>
<li><span class="Apple-style-span" style="font-family: inherit;">Go<a href="#test_your_app"> test your app</a>, instead of localhost, substitute </span><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">http://[your-application-id].appspot.com/</span><span class="Apple-style-span" style="font-family: inherit;"> for </span><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">http://localhost:8080</span></li>
</ol><p>For more on appengine, definitely check out their great knowledge base for <a href="http://code.google.com/appengine/docs/python/gettingstarted/">getting started in Python</a>.</p><h1>Connect to Twilio</h1><p>Now that you have your twilio trial account and your app ready to be connected, you just need to connect the 2.</p><p>In the Developer Tools section of your account, set you SMS URL to:<br />
<pre>http://[your-application-id].appspot.com/</pre><p>Click [Save].</p><p>Send a TXT to the Sandbox Phone Number you configured with the Body of "[PIN] ?".</p><p>You'll need to put the [PIN] in for the next question as well.</p><br />
<h1>The end</h1><p>Go forth and create inexpensive Question and Answer apps using SMS, Twilio and App Engine.<br />
</p><p>If you have any questions, issues, or create something awesome with this tutorial, please drop me an a comment.</p></div>pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com6tag:blogger.com,1999:blog-590774712948906419.post-36929231473023634122010-12-11T11:36:00.000-08:002010-12-11T15:54:04.068-08:00There's no rollback?Preventing bugs before production is really, really hard. Basically, your code moves from the warm, safe, controlled environment of your dev and testing environment to the cold, harsh reality of production. Since you know some of your deployments are going to fail, and you can't predict which ones are going to be a failure, you always need to be able to rollback.<br />
<br />
Last night, a vendor completed a hardware migration that hosed a critical component of one of my applications. I woke up this am to a flurry of crisis emails. My response was simple, so... "rollback." To my surprise, rolling back was not an option. How in the world can any vendor, with customers, release a change where rolling back is not an option? The outage lasted over 3 hours - for those keeping score at home, one of those outages moves your app from a 99.99% reliability to slightly better than 99.95%. For a portion of our students, it meant a rescheduled class, which really sucks.<br />
<br />
Rollbacks require a little bit of additional planning and sometimes additional work to support, but they are always, always, always worth the additional expense. When a bug occurs, you can't possibly know how long it will take to fix, just ask <a href="http://staff.tumblr.com/post/2127872280/downtime">tumblr</a>.<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGLbDroHWS4Abom5UvDkMooYGmRBriQ3cLOLvq82TkJCfa2D2IUEelgd8uuwYLoEMw9Ef-2i2Uck5YSRUDH4rjW9c-Lk_OIDgmS8l_PzG_sfUqCtP2ayZ_Bmvx76fU53GGwUf2kflt78U/s1600/explosion.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="141" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGLbDroHWS4Abom5UvDkMooYGmRBriQ3cLOLvq82TkJCfa2D2IUEelgd8uuwYLoEMw9Ef-2i2Uck5YSRUDH4rjW9c-Lk_OIDgmS8l_PzG_sfUqCtP2ayZ_Bmvx76fU53GGwUf2kflt78U/s200/explosion.jpg" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">your software without rollbacks.</td></tr>
</tbody></table><br />
Here are some basic tips for ensuring rollback:<br />
<br />
<ol><li>Ask the question to yourself or your engineers, "How do we rollback?"</li>
<li>Make sure your app and database changes are backwards compatible.</li>
<li>Deploy and validate database (and any other infrastructure) changes prior to any user-facing app changes.</li>
<li>Deploy and validate an app at a time.</li>
<li>Script all of your deployments and rollbacks. We have our own in-house built deployment system, but <a href="http://wiki.opscode.com/display/chef/Home">chef</a> and <a href="http://www.puppetlabs.com/">puppet</a> seem to be popular tools for creating these scripts.</li>
</ol><div>You can check out a reminder I wrote to myself about how to identify when to rollback <a href="http://petemiron.blogspot.com/2010/06/all-failed-deployments-are-anachronisms.html">here</a>.</div><div><br />
</div><div><a href="http://petemiron.blogspot.com/2010/06/all-failed-deployments-are-anachronisms.html"></a>There is exactly one type of change (that I've encountered) that is actually hard to rollback from, public DNS changes. So try really hard to not screw those up or set your TTLs really low.</div><div><br />
</div><div>On behalf of all of your future customers, please make sure you have a way to rollback your software.</div>pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com1tag:blogger.com,1999:blog-590774712948906419.post-43588531060698831862010-11-09T10:52:00.000-08:002011-01-17T05:19:02.064-08:00Start hiring interns, today!If you're an engineering manager at <s>a startup</s> any company anywhere, you need to start hiring interns immediately. Here are the most common reasons I hear for not hiring interns:<br />
<ol><li><span class="Apple-style-span">I don't have the time to manage/mentor them</span><br />
<br />
You have some set of work you need to get done, probably more work than you have people to do the work. The question you need to answer is whether or not the work that interns will help get done will be worth the time investment in the interns. I'd say about only 1/4 of the interns I've hired need more management or mentoring than any other new hire we bring on. Usually, this is a problem with confidence and can quickly be remedied by pointing the intern toward the right places to find solutions to their problems before asking for help. This does not mean that you do not need to structure projects for the interns, but certainly not more so than for any of your other engineers. The overall complexity of the projects you give them may be lower than the problems you give your most senior engineers, but the projects themselves should not require more structure.<br />
<br />
</li>
<li><span class="Apple-style-span">I don't have time to sift through all those resumes</span><br />
<br />
This is a problem common in all hiring and an extension of the 1st excuse. Set up a simple set of criteria to strongly filter your set of resumes. We look for interns with a website, high GPAs, and some set of achievements and extracurriculars. Read Joel Spolsky's <a href="http://www.joelonsoftware.com/articles/SortingResumes.html">sorting resumes</a> article for more on how to do this well for any job. When using the right process, reviewing resumes takes way less time than you think it will.<br />
<br />
</li>
<li><span class="Apple-style-span">I need people with more experience</span><br />
<br />
You probably need some people with more experience, but more likely than not, you have a lot of things you do every day that could be effectively completed by an intern. Furthermore, I am consistently <b>floored</b> by the experience level that most interns come in with. They have grown up with technology starting at younger ages than all of the more senior engineers have. They can't remember a time before computers. They know more than you might think. More importantly, if you're getting interns from great schools with great programs in their junior or senior years, they are likely learning skills and techniques that aren't even in common practice yet.</li>
</ol>Once you've accepted these 3 reasons for not hiring interns as fallacies. Here's what you need to do:<br />
<div><ol><li>Choose which schools you want to focus on. The more selective the school, the earlier you'll need to get started.</li>
<li>Post your internship position with those schools' career centers. The ideal time to do this is in September, but I've hired great interns as late as March/April. If you're in NYC, you need to apply with <a href="http://hackny.org/a/startups/">hackny</a> and <a href="http://nycturingfellows.org/">NYC Turing Fellows</a>. At Knewton, we managed to land a hackny fellow last year. Check out Knewton's very own Stuart Partin at http://hackny.org/a/2010/08/video-of-the-hackny-summer-2010-demofest/. This summer will be our first sponsoring a NYC Turing Fellow.</li>
<li>If you're interviewing remotely, use a remote screen sharing tool to do the rough equivalent of a face-to-face interview. <a href="http://www.dimdim.com/">Dim dim</a> or <a href="http://www.webex.com/">WebEx</a> are great tools for this. These tools allow you to speak with the internship candidate and white board in real-time - <a href="http://www.skype.com/">Skype</a> with <a href="http://ietherpad.com/">Etherpad</a> work great, too.</li>
<li>Pay your interns! You will get real value from them, you should give them some back - outside of experience.</li>
<li>Get them releasing production-level code early. This is probably the most important process to demystify for a new engineer. Most have never even heard of a release or deployment procedure. Show them it's not that hard.</li>
<li>If they're really good, hire them full-time.</li>
</ol></div><div>Once you've hired your interns, here is a sampling of the type of things you can expect based on what I've seen our interns accomplish:</div><div><ul><li>Significant free trial conversion improvements.</li>
<li>My first hire at Knewton (from a previous company internship).</li>
<li>Faster loading web pages, thanks to some smart SQL tuning and rewriting and a pretty kickass performance report.</li>
<li>More user-friendly administrative interfaces. </li>
<li>Expanded functional testing coverage through Selenium.</li>
<li>Chat integration with Trac.</li>
<li>Movement of large sections of code to a single build process.</li>
<li>User research.</li>
<li>iPhone app specs.</li>
</ul></div><div><br />
<div>At the end of the day, the excuses most people come up with for not hiring interns are easily outweighed by the production you can expect. Even when paying interns, the value outweighs the cost. If you get really good at hiring interns, you will not only get greater output from your team, you will also create an incredibly valuable recruiting channel.</div><div></div><br />
<br />
Please feel free to drop me any questions or comments below. If you're interested in interning at Knewton, email your resume to <a href="mailto:techjobs@knewton.com">techjobs@knewton.com</a>.</div>pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com2tag:blogger.com,1999:blog-590774712948906419.post-86988873271100137972010-09-16T13:08:00.000-07:002010-09-17T08:41:32.994-07:00How to Scale Backend InfrastructureEarlier this week, I participated in a talk at <a href="http://hunch.com/">Hunch</a> on Scaling Backend Infrastructure with <a href="http://www.tompinckney.com/">Tom Pinckney</a> (Hunch), Kiril Sheynkman (Thansys), and Jeff Hammerbacher (Cloudera).<br />
<br />
<div><span class="Apple-style-span" style="font-size: x-large;">How to scale inside the request loop</span></div><div>The majority of web applications have a user sitting behind a web browser who lands on a site, clicks on a button and expects something to happen - quickly. As more users visit your site, they compete for scarce resources - CPU cycles, RAM, hard disk access, and bandwidth. Your goal is to add more resources to support more users - here's how.</div><div><ol><li>Build a system that can be scaled.</li>
<ul><li>Make sure your hardware is clone-able.</li>
</ul><ol><ul><li><span dir="ltr" id=":2e3"><span class="Apple-style-span" style="font-family: inherit;">If you lose one machine, you need to be able to build a new machine that exactly matches that old machine from OS to configuration.</span></span></li>
<li>Common tools for configuration: <a href="http://www.puppetlabs.com/">puppet</a>, <a href="http://wiki.opscode.com/display/chef/Home">chef</a>, or some bash scripting.</li>
</ul><ol></ol></ol><ul><li>Be daring in your language/framework choice.</li>
</ul><ol><ul><li>Choose a language that optimizes for your ability to quickly develop software and your ability to acquire more developers who can program in that language.</li>
<li>Language choice impacts per-request performance.</li>
<li>Common Tools: <a href="http://www.python.org/">Python</a>, <a href="http://www.ruby-lang.org/">Ruby</a>, <a href="http://www.scala-lang.org/">Scala</a>, <a href="http://groovy.codehaus.org/">Groovy</a>, <a href="http://clojure.org/">Clojure</a>.</li>
</ul><ol></ol></ol><ul><li>Be conservative in infrastructure choice.</li>
</ul><ol><ul><li>Prefer meat-and-potatoes over next-new-thing, assuming a similar price and available required features on the meat-and-potatoes option.</li>
<li>Make sure you have internal knowledge/capability, active community support, or commercial support options if you go with the next-new-thing.</li>
<li>Use infrastructure others have proven can scale.</li>
<li>NoSQL can be made to work. But, make sure you have good support.</li>
<li>Common Tools: <a href="http://www.mysql.com/">MySql</a>, <a href="http://httpd.apache.org/">Apache Web Server</a>.</li>
</ul><ol></ol></ol><ul><li>Separate user data from reference data.</li>
</ul><ol><ul><li>Enable eventual scaling by keeping users on separate databases.</li>
</ul></ol>
<li>Make sure it performs. Know when it doesn't.</li>
<ul><li>Measure your application performance.</li>
</ul><ol><ul><li>Measuring machine performance is not enough.</li>
<li>Know how long it takes each component in your system to respond. From MySql explain plans to queue depth, to each webservice call, to end-user experience, know how long each component of a single request takes.</li>
<li>Most application monitoring and measurement requires some bit of custom coding.</li>
<li>Common tools: <a href="http://dev.mysql.com/doc/refman/5.0/en/explain.html">MySql explain</a>, <a href="http://www.nagios.org/">Nagios</a>, <a href="http://www.cacti.net/">Cacti</a>, <a href="http://appfirst.com/">AppFirst</a> (looks promising, but haven't tested, yet).</li>
</ul><ol></ol></ol><ul><li>Performance test to find obvious bottlenecks and config flaws.</li>
</ul><ol><ul><li>Analyze real-world scenarios to design performance testing. Look at what users actually do on the site to develop your performance test plans. Identify the most common paths or the most frequently accessed pages.</li>
<li>Design performance tests with product/user experience folks to make sure how they expect users to use your site is captured.</li>
<li>Design your tests to validate horizontal scalability. Run your tests with one of everything. Add machines. What happens to your performance?</li>
<li>Common tools: <a href="http://selenium-grid.seleniumhq.org/">Selenium Grid</a>, <a href="http://grinder.sourceforge.net/">Grinder</a>, <a href="http://browsermob.com/performance-testing">BrowserMob</a>, <a href="http://www.saucelabs.com/">Sauce Labs</a>.</li>
</ul><ol></ol></ol><ul><li>Address performance problems.</li>
</ul><ol><ul><li>Check your configurations against recommended configurations of your infrastructure.</li>
<li>Typically, out of the box, you are not giving your database enough memory for the hardware it's running on.</li>
<li>Startup times for app servers worker threads are often pretty slow. Try to start several at initial server startup and make sure they stay running as long as possible.</li>
</ul><ol><ul></ul><ol></ol></ol><ul><li>The database is usually the focal point of most performance problems. Here are a few suggestions to help db performance:</li>
</ul><ol><ul><li>Add indexes.</li>
<li>Tom recommended limiting joins. I find you can get better overall performance by limiting queries to the database.</li>
<li>ORM is fine for initially building your product, but once you're looking to scale, you need to replace ORM (like Rails) with hand-written, optimized SQL.</li>
</ul><ol></ol></ol></ol>
<li>Limit access to shared resources.</li>
<ul><li>Decouple reads and writes with queues.</li>
<ul><li>Reading and writing from the same logical or physical hard disk is a slow, expensive process. But, you need to store your data somewhere..</li>
</ul></ul><ol><ul><li>MySql slaving is essentially a cheap, easy to use queuing mechanism.</li>
<li>You can scale MySql slaves to insanely large sizes. I've seen 90-way slaving.</li>
<li>There are queues available for most languages.</li>
<li>Common tools: <a href="http://kr.github.com/beanstalkd/">beanstalkd</a>, <a href="http://activemq.apache.org/">ActiveMQ</a>, <a href="http://www.mulesoft.org/">Mule</a>.</li>
</ul><ol></ol></ol><ul><li>Limit contention for hard disk access.</li>
</ul><ol><ul><li>Keep as much of your database and indexes in RAM as possible.</li>
</ul></ol><ul><li>Pin users to separate databases.</li>
<ul><li>To limit the amount of reading and writing that needs to be done to any one database server, you can design your system to limit the amount of data stored on that server in a couple of common ways.</li>
</ul></ul><ol><ul><ul><li>Hash by user id for user-specific database.</li>
<li>Provision users to a user-specific database, with a shared master index of databases for users.</li>
<li>Once you've done that, you can add more databases as you add more users.</li>
</ul></ul><ol></ol></ol></ol></div><div><div><span class="Apple-style-span" style="font-size: x-large;">Notes on scaling outside the request loop - analytics</span></div></div><div>These are notes from Jeff's talk. However, you might be better off checking out his book: <a href="http://www.amazon.com/Beautiful-Data-Stories-Elegant-Solutions/dp/0596157118?ie=UTF8&tag=pete06-20&link_code=btl&camp=213689&creative=392969" target="_blank">Beautiful Data: The Stories Behind Elegant Data Solutions</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=pete06-20&l=btl&camp=213689&creative=392969&o=1&a=0596157118" style="border: none !important; margin: 0px !important; padding: 0px !important;" width="1" />. The general idea is that you cannot analyze using the same systems and infrastructure you're using to run the system.</div><div><ol><li>You can probably get by with MySql for a bit.</li>
<li>Extract, Transform, and Load (ETL) will always take more time (both developer time and run time) than you think it should.</li>
<li>The easier to analyze the data, the more requests you'll get to analyze data.</li>
<li>SQL is not a standard. (I'd generalize to no software standard is standard.)</li>
<li><a href="http://hadoop.apache.org/">Hadoop</a> (MapReduce) can parallelize data analysis across multiple machines to speed up analysis. </li>
<li>Powerful analytics can produce powerful features. (Think "people you might know" in Facebook.)</li>
<li>Use analytics infrastructure to precompute high-read information. (Think Google's entire web index.)</li>
</ol><div>Please get me your comments on this post. This is really just a start to catalog some of these points for others to use.</div></div>pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com4tag:blogger.com,1999:blog-590774712948906419.post-66195943776627697802010-06-07T18:42:00.000-07:002010-06-07T18:42:39.736-07:00All failed deployments are anachronisms.Your code doesn't care what day it released on. If there is an extended outage or degradation as a result of a deployment, the code is always in the wrong place at the wrong time. Here are all the wrong days to deploy a broken release:<br />
<br />
<ol><li>Monday</li>
<li>Tuesday</li>
<li>Wednesday</li>
<li>Thursday</li>
<li>Friday</li>
<li>Saturday</li>
<li>Sunday</li>
</ol><div>and the times:</div><div><ol><li>One O'Clock</li>
<li>Two O'Clock</li>
<li>...</li>
</ol><div>and obviously a bunch of holidays and unrelated business synchronizing events (black friday, sales deadlines, etc) that you also shouldn't deploy on.</div></div><div><br />
</div><div>However, if you really believe this, you should stop writing, managing or using software NOW! Unfortunately, you will deploy broken code, because:</div><blockquote><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px;">"I’m sorry to say so but, sadly, it’s true that Bang-ups and Hang-ups can happen to you." <br />
-</span> <span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px;">Dr. Seuss (Oh, the places you'll go!)</span></blockquote>And when you do deploy that broken code, it probably had nothing to do with when it was deployed. In fact no one probably would've noticed if you did just one thing...<br />
<br />
<span class="Apple-style-span" style="font-size: x-large;">ROLLBACK!</span><br />
<div>Three rules:</div><div><ol><li>Code must always be able to be rolled back.</li>
<li>Rollback must be a single command.</li>
<li>The rules for rollback must be simple, easy-to-follow and aggressive. (ie. Customer call related to issue with release, exception related to a release, etc.)... then, just...</li>
</ol></div><div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-size: x-large;">ROLLBACK!</span></div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Then, figure out what went wrong, how to prevent it from happening again... rinse and repeat... any day... any time.</div></div>pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com2tag:blogger.com,1999:blog-590774712948906419.post-58837978656882861652010-06-03T07:19:00.000-07:002010-06-03T07:22:45.779-07:00Follow your debugging process, stupid. In 10 easy steps.This post is a reminder to myself. I wasted a lot of my personal time and time from members of my team by not following, a simple, repeatable debugging process. The below process won't work 100% of the time, but when it does it will save you hours, stomach lining and and leave you extra gas in the tank for handling problems that are actually hard to replicate and debug.<br />
<br />
<ol><li>Verify the input and output creating the defect.</li>
<li>Replicate the defect in your development environment.</li>
<li>Write a unit test that replicates the defect using the same input as the defect and desired output. (You may want to extend this to other permutations of the defect as well.)</li>
<li>Verify the unit test breaks.</li>
<li>Fix the code.</li>
<li>Verify the unit test passes.</li>
<li>Verify that the defect is resolved in your local development environment.</li>
<li>Release to next environment (Prod/QA)</li>
<li>Verify that the defect is resolved in the next environment.</li>
<li>Repeat until defect is fixed for all users.</li>
</ol><div>Do <b><i>not</i></b> try to skip steps in the process. You will miss something important. You will waste yours or other people's time. It's not worth it.</div><div><br />
</div><div><br />
</div>pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com2tag:blogger.com,1999:blog-590774712948906419.post-81921479412880797742010-03-31T20:41:00.000-07:002010-04-02T21:49:16.995-07:00Don't be the slowest gazelle<blockquote>Every morning in Africa, a gazelle wakes up. It knows it must outrun the fastest lion or it will be killed. Every morning in Africa, a lion wakes up. It knows it must run faster than the slowest gazelle, or it will starve. It doesn't matter whether you're a lion or a gazelle -- when the sun comes up, you'd better be running.<br />
-- proverb attributed to Roger Bannister (first man to break the 4 minute mile)</blockquote>At a startup, there are inevitably moments when you question whether or not you're moving too fast. From defects, regressions, and poor performance to missed requirements, and reimplemented requirements. Now you have more customers, you're investing more in marketing, you can't afford the mistakes you used to be able to tolerate. Your startup team wants to begin establishing slower, more measured process for greater predictability and fewer mistakes. You might think about hiring a more "experienced" program manager or project manager, maybe you think about moving to a waterfall process. Although it seems counter-intuitive, you must not slow down... you need to speed up. In his post, <a href="http://www.startuplessonslearned.com/2010/03/speed-up-or-slow-down-for-harvard.html">Speed up or slow down?</a> lean startup evangelist Eric Ries refers to this as the speed-up-or-slow-down-moment. Eric advocates for speeding up -- "The <a href="http://blogs.hbr.org/cs/2010/02/how_much_process_is_too_much.html">day-to-day process</a> that startups build should also attempt to maximize speed of learning." Not only do I strongly agree with the need to speed up, I believe you need to speed up regardless of your company's current size, age or market cap. You need to focus on staying lean and agile -- or suffer the fate of the slowest gazelle...<br />
<br />
I'd like to use Eric's broadened definition of a startup, with minor modification:<br />
<blockquote>"A startup is a human institution designed to create a <s>new</s> product or service under conditions of extreme uncertainty."<br />
-- Eric Ries</blockquote>I've struck out new. While this may be a useful modifier for the traditional view of a startup, I strongly believe it is not a precondition for applying many of the lean startup principles. All businesses, regardless of product or service (old or new), are operating under conditions of extreme uncertainty. Over half the companies in the Dow Jones Industrial Average, still a key benchmark for performance of the US Equities Market, were added in just the past 20 years. Even if you think you've moved beyond startup classification, there are lions stalking you -- competitors, regulators, patent trolls, your customers' whims. You need to be faster than all of them.<br />
<br />
In 2007, at my previous company Vonage, we were found to be infringing on Verizon's patents. Verizon sought and won an injunction from our use of those patents -- which included technology at the core of Vonage's call processing infrastructure. I will abstain for now from editorializing on the sanity of these rulings but, I wrote about my general sentiments on a similar patent issue <a href="http://blog.knewton.com/2010/01/12/call-for-an-independent-invention-defense/">here</a>. Vonage managed to appeal and receive a stay for the injunction. This stay set the timeline for our designing patent workarounds. A patent workaround is basically a new way of getting something to work that can not be read on the patent. A judge then needs to approve the workaround. If you think involving business owners in designing software is hard, try working with lawyers... Then, drop a judge in as your QA team. Only if the judge doesn't approve your workarounds, it is basically lights out. Goodbye 2mil customers; farewell $50mil monthly revenues; goodnight future potential for pivots.<br />
<br />
Prior to beginning the patent workarounds, it took months to deploy substantial changes to our call processing system. The planned deployment cycle was 6-8 weeks; 3-4 weeks for QA plus another 3-4 weeks to deploy. Inevitably, between week 6 and week 8, a critical defect would be found (memory leak, call loop, etc), the code would be rolled back and the cycle would begin again. <br />
<br />
Gaining agreement to move faster with code deploys with QA and Operations was initially met with antagonism - "well, if your coders didn't put any bugs in the code, it would get deployed faster." Once our survival as a company was inextricably tied to our ability to modify and deploy our core processing system, that agreement became substantially less complicated. We had already begun to develop code quality testing and automated integration tests. The manager of the team ran to Best Buy for a switch to build a network that the development team could run automated deployments on. We worked with the QA team to define a < 1 work day set of tests to be ran for the most frequently used features. Then, we (substantially) automated the deployment and production integration testing of the call processing system.<br />
<br />
In around 2 months, we went from taking months to deploy the system, to at the height of the workaround madness, deploying the entire system of ~100 machines in less than 24 hours -- for a system supporting 2 million customers making calls next door, across continents, to the other side of the world and most importantly to 911. All systems have bugs and defects, but, when you can fix those bugs and redeploy on a daily basis the impact of any one bug is substantially reduced. Most importantly, we learned what we needed to do to release quickly and often, with lower stress and higher quality thanks to fewer regressions, better automated testing and faster identification of bugs caused by rare or edge cases. By hitting those edge cases early on in the process, we knew we had a stable base to build from if there was a ruling that required any new change.<br />
<br />
Being able to "run faster" helped ensure Vonage's future in a very uncertain time. No matter how big your company is, there is no way of knowing where the "lion" is going to come from. Rest assured they're out there; will you be ready to outrun them when they pounce? For a technology company, choosing to speed up improves the likelihood you'll be able to outrun your competitors, react to regulators, work around predatory patent litigation and most importantly react to your customers' needs. Don't be the slowest gazelle.<br />
<br />
* As a footnote, Vonage did end up settling with Verizon before the workarounds came to a court decision. Even though we were confident we could quickly modify our code to whatever specs were needed to qualify for a workaround, in an appeals situation you're not guaranteed the judge will accept your workaround. A single day without using our call processing technology would have essentially put Vonage out of business.pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com1tag:blogger.com,1999:blog-590774712948906419.post-9539830071475501292010-03-01T03:59:00.000-08:002010-03-01T08:52:19.968-08:00On becoming a practicing software engineerIf you're a recent or soon-to-be college grad (or maybe you realized your undergrad degree in Art History ain't gonna pay the bills) and you are passionate about computers and computer programming, here are my tips for becoming a successful practicing software engineer. Many of these things probably aren't the things they taught you in your college programming classes, but all of these are important.<br />
<ol><li><b><span class="Apple-style-span" style="font-weight: normal;"><b>Practice! Practice! Practice!</b> You learn to code by, um, reading and writing code! If you don't have much experience and want to get started, find an open source project you care about and contribute a patch or two. The "View Source" feature of Web Browsers and the open source movement are 2 of your greatest assets when learning to code. Use them! As someone who started his career copying BASIC programs from Compute! and Byte magazines, I can't tell you how great it was to discover the magical "View Source" menu item in Netscape. Open Source projects will teach you about packaging, style guidelines, automated testing, bug tracking, and version control -- while also giving you much needed practice.<br />
<br />
Don't forget, your code doesn't *work* until someone else uses it. If you can't work at a startup and need to get code into the hands of users quickly, open source projects are a great way to go. I still recommend going with the startup, though. At <a href="http://www.knewton.com/">Knewton</a>, everyone from summer interns to new full-time engineers ship code (that customers actually use) in their first couple weeks of starting. I'd like to get this down to the first day.<br />
<br />
In your early practicing, make sure to develop really strong habits. I learned the most of my habits from Steve McConnell's <i>Code Complete </i>and Kent Beck's <i>eXtreme programming</i>. Write code others can support if they need to, but try to make it so they don't need to support it :)<br />
<br />
I'd also recommend checking out PragDave's Code Kata site to work on solving problem:<br />
<a href="http://www.codekata.com/">http://www.codekata.com</a><br />
<br />
</span></b></li>
<li><b>Work at a startup!</b> You will learn more in your first month at a startup than you will in your first year in any other company. The first company I worked for was a 3-person shop in Syracuse, NY. I learned everything from how to become a practicing software engineer, to how to be a customer support person, estimate and bid consulting jobs, write requirements, QA, write user manuals, configure SQL Servers, configure IIS Servers, configure linux firewalls ... If your first job entails you being handed requirements that you then write code for and hand "over the wall" to QA - run!<br />
<br />
Don't just take my word for it. Here's what one of our former interns who recently graduated and landed a sweet job in CO had to say:<br />
<blockquote>"I had NO experience as a coder. You guys gave me a LOT. In fact, more skillz than you can really understand. Things that transferred over beyond Ruby, RUnit, Rails, MySQL unix commands (I'm loving that I actually understand how to use the CLI in my Ubuntu set up btw...), etc - more the ability to take a bunch of instructions I barely understood and google my way/solve my way to a solution."</blockquote><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Chris Dixon also has a couple nice posts on this topic:</div><blockquote class="webkit-indent-blockquote" style="border-bottom-style: none; border-color: initial; border-left-style: none; border-right-style: none; border-top-style: none; border-width: initial; margin-bottom: 0px; margin-left: 40px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"><a href="http://cdixon.org/2009/05/11/joining-a-startup-is-far-less-risky-than-most-people-think/">Joining a startup is less risky than you think.</a></blockquote><blockquote class="webkit-indent-blockquote" style="border-bottom-style: none; border-color: initial; border-left-style: none; border-right-style: none; border-top-style: none; border-width: initial; margin-bottom: 0px; margin-left: 40px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><a href="http://cdixon.org/2010/02/11/every-time-an-engineer-joins-google-a-startup-dies/">Every time an engineer joins google a startup dies.</a></div></blockquote><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
More specifically, work for my startup:</div><blockquote class="webkit-indent-blockquote" style="border-bottom-style: none; border-color: initial; border-left-style: none; border-right-style: none; border-top-style: none; border-width: initial; margin-bottom: 0px; margin-left: 40px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><a href="http://www.knewton.com/jobs">Knewton Jobs</a></div></blockquote><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
If there's nothing interesting to you there, check out these couple of other sites:</div><blockquote class="webkit-indent-blockquote" style="border-bottom-style: none; border-color: initial; border-left-style: none; border-right-style: none; border-top-style: none; border-width: initial; margin-bottom: 0px; margin-left: 40px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><a href="http://startuply.com/">Startuply.com</a></div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><a href="http://jobnob.com/">Jobnob.com</a></div></blockquote><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div></li>
<li><b>Don't be afraid to make mistakes! or Perfection is for popes and Chinese emperors.<br />
<span class="Apple-style-span" style="font-weight: normal;">In 1999, I got my first big job at an online brokerage (Datek) in NYC. This was back when everyone, probably including you, traded online and doubled their money daily - basic fundamental laws of economics changed... until they didn't. The day of my first big release, I took down the ability to login to the production site at market open. This was, as my CTO at the time reminded me, "reeeeaaaaaaalllllly baaaaaddddd." However, we talked through the problems and took the appropriate steps to ensure I couldn't make the same mistake again (it involved improper DB connection handling, lack of performance testing, deployment timing and rollback procedures). If he fired me, maybe I'd feel differently about making mistakes. Fortunately, for me, he understood these types of mistakes will happen -- as long as you're willing to grow from the mistakes and not repeat the same mistake twice.<br />
<br />
Don't forget that coding is a creative endeavor; typically, there is no <b>one</b> correct solution. Be prepared to try several.<br />
</span></b><br />
</li>
<li><b><span class="Apple-style-span" style="font-weight: normal;"><b><span class="Apple-style-span" style="font-weight: normal;"><b>Reality...zen and the art of "boring" tasks.<br />
<span class="Apple-style-span" style="border-collapse: collapse; font-weight: normal;"><span class="Apple-style-span" style="font-family: inherit;">You studied sexy problems in school, you know how to solve nine-queens efficiently, find shortest path in a graph, compute Chebyshev distance in a metric space...the large part of your day as a software developer will not be spent working on such problems. More than likely you will spend a day on a far wider range of tasks that are comparatively less interesting in an engineering/problem-solving sense. These lower-level tasks are generally more simple and yet each decision that is made in their execution can be evaluated and perhaps improved upon. Design details as small as method signatures, naming conventions, loop constructs or recursion, tail-recursion or not (maybe even that's too sexy!), are both extremely important and regularly overlooked. A large program is built on many lines of code. Each line contains required prior decisions to produce. An appreciation for these small details will contribute a lot not only to the quality of the program as a whole, but to the education of the coder.<br />
<br />
If you're not already in a coding-related field, look for ways to make your current job more efficient through automation. This can be as simple as creating access databases, word mail merges and batch files to automate tasks that used to take you hours or days to complete. This is actually how I got my start coding professionally, by building MS Access database apps that made week-long tasks takes hours (mass mailings to customers at an HVAC rep and students at the SU Masters of Public Administration program).<br />
<br />
</span></span></b></span></b></span></b></li>
<li><b><span class="Apple-style-span"><b><span class="Apple-style-span"><b><span class="Apple-style-span" style="border-collapse: collapse;"><span class="Apple-style-span" style="font-family: inherit;">Finish!<br />
<span class="Apple-style-span" style="font-weight: normal;">You don't get any points for effort. You need to finish what you start. Half-written, incomplete code atrophies very, very quickly. If you find yourself starting more than you finish, you need to revisit the scope of your problems. Code against smaller problems, but finish the code!<br />
</span></span></span></b></span></b></span></b></li>
</ol><div><span class="Apple-style-span" style="border-collapse: collapse;">While not everyone is destined to be a great coder, if you're interested in learning how, I strongly recommend the list above. I'm not certain this is an exhaustive list; I'd love to hear any other suggestions. Good luck, and have fun!</span></div><ol></ol>pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com6tag:blogger.com,1999:blog-590774712948906419.post-3958867911110638832010-02-18T05:39:00.000-08:002010-02-18T05:39:30.423-08:00Continuous Deployment Prereq #1 - Maker's Mark NagiorbAt <a href="http://www.knewton.com/">Knewton</a>, we've used a build orb to continuous build success for the past 1.5 years. We're currently working on <a href="http://radar.oreilly.com/2009/03/continuous-deployment-5-eas.html">Continuous Deployment</a> for one of our subsystems. As a prerequisite, we've setup <a href="http://nagios.org/">nagios</a> to monitor system state. If you work at a startup, have customers and don't have <a href="http://nagios.org/">nagios</a> setup yet... start setting it up today. It can easily monitor system state, mysql (replication monitoring is especially useful), and scripts for new probes are straightforward - for example, we're currently monitoring our <a href="http://zendesk.com/">zendesk</a> support ticket queue, you can also synthetically test your web app. Out of the box, nagios has email, and web-based monitoring, there are also scripts for sending IMs... The most useful of all of our custom extensions from <a href="http://twitter.com/devondjones">@devondjones</a> is the Maker's Mark Nagiorb. Using an <a href="http://www.arduino.cc/">arduino</a>, an LED array and a liquid etched, <b>empty </b>Maker's Mark bottle, our entire office now knows if all systems are fully functional, of if we're going to need another bottle of whiskey.<br />
<br />
<b>All Systems Go!</b><br />
No warning, critical or unknown nagios alerts.<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8I06hZQnAs4xkaWcGunILEDiiXvwDYjEvyqN_DzGyX9WZoUya4pWXOpk-92AdiEoeb8AjoX4Uv9xgYW8J4nSC20-RqRgV3O0z20AAN3CdHjDBTaZQvTIrYXmktcRe7aH8BPTjGsw-rxA/s1600-h/makers-mark-all-systems-a-go.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8I06hZQnAs4xkaWcGunILEDiiXvwDYjEvyqN_DzGyX9WZoUya4pWXOpk-92AdiEoeb8AjoX4Uv9xgYW8J4nSC20-RqRgV3O0z20AAN3CdHjDBTaZQvTIrYXmktcRe7aH8BPTjGsw-rxA/s320/makers-mark-all-systems-a-go.jpg" width="240" /></a></div><div class="" style="clear: both; text-align: center;"></div><div style="text-align: left;"><b>Warning!</b><br />
New ticket comes in from zendesk, or mysql replication behind. Also, while a bit harder to show in static photos, the orb will blink with the number of open warning statuses. So, if you have zendesk tickets and your mysql replication is behind, it will blink twice every minute.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhflTLF-UWE7dtVnQZLL4-TeCq41mifyKuGE5cpRVHXlbQQU3TrD9K2UlzxHbtYz10oH6HjGBJUOBgs1-Yh8ZV_wdQGXkiI2NcIZhrxlmxCHwWAlg06o7d_mgDXwaAT3d0UlPFV2m1byjU/s1600-h/makersmark-warning.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhflTLF-UWE7dtVnQZLL4-TeCq41mifyKuGE5cpRVHXlbQQU3TrD9K2UlzxHbtYz10oH6HjGBJUOBgs1-Yh8ZV_wdQGXkiI2NcIZhrxlmxCHwWAlg06o7d_mgDXwaAT3d0UlPFV2m1byjU/s320/makersmark-warning.jpg" /></a><br />
<br />
</div><div class="" style="clear: both; text-align: center;"></div><div style="text-align: left;"><b>Critical</b><br />
Critical messages are similar to warnings, but will flash red instead of yellow. If there are several blinks, it's time to hit the full Maker's Mark bottle :)</div><div class="" style="clear: both; text-align: center;"></div><div style="text-align: left;"><b><br />
</b><br />
<b>DIY</b><br />
If you're interested in having your own Nagiorb, you can get most of the way there by following <a href="http://www.instructables.com/id/Arduino-Orb-Build-Warden/">Devon's build orb instructable</a>. I'll try to coax him into adding a sample nagios script in there as well.</div>pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com0tag:blogger.com,1999:blog-590774712948906419.post-42504994345114340552010-01-23T17:11:00.000-08:002010-01-23T17:39:00.844-08:00Microsoft does not want your money!This just in... Microsoft does not want your money! At least for up to 24 Hours beginning 10pm PT Monday, January 25th.<br /><blockquote>As part of our continued efforts to improve the Zune Marketplace service onXbox LIVE, Zune Marketplace will be undergoing scheduled maintenance for up to 24 hours, starting 10:00 P.M. (PT) on January 25, 2010. During that time, you will be unable to rent or purchase video content.</blockquote>How is it o.k. for any major web business to go offline for up to 24 hours? How many people would have started using the Zune Marketplace on January 25th that never will now? I know, you're thinking nobody - except maybe this guy:<div><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPp5KceuUlbWOlaT5qujZN_L-bmkaX4-mmHBjycyzkvvTp7DGbpPbNtct3q25g2ATVD5ekgdJHpPdH7Znf7xHpYsOV57bD1DNtR1khwCBAjsKHV8yFj5uVjm7PWw2Gpx9WPSgD5u8_h6c/s200/zune-tattoo.jpg" style="cursor:pointer; cursor:hand;width: 200px; height: 179px;" border="0" alt="" id="BLOGGER_PHOTO_ID_5430114596684204578" /></div><div><br /><div><div>All joking aside, designing solutions to required downtime is not hard to do. I'm responsible for an <a href="http://www.knewton.com/">online education site</a> that can't have <i>that</i> many more users than the Zune. Yet, <a href="http://twitter.com/knewton_tech">my team</a> (which I'm sure is significantly smaller, but better looking than the team working on the Zune Marketplace) managed to release many times per week with only :30 mins of scheduled downtime for the entire year (the :30 mins was avoidable, but would've required ~3 days of work).<div><br /></div><div>Typically, all you need to do is decouple client releases from application server releases from database releases. Then release in reverse order... required db changes go before application server releases go before client releases.<br /><div><br /></div><div>Now, we're starting work on a continuous deployment system so that all deployments happen on check-in. (Don't worry, there will be automated tests.) If you're a Microsoft engineer, read more at: <a href="http://radar.oreilly.com/2009/03/continuous-deployment-5-eas.html">Read Eric Ries's post on Continuous Deployment in 5 Easy Steps</a>.<br /></div></div></div></div></div>pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com2tag:blogger.com,1999:blog-590774712948906419.post-56946173357457359252010-01-23T13:48:00.000-08:002010-01-23T13:48:44.074-08:00Taylor Mali: What teachers make | Video on TED.com<div>On behalf of my wife, my sister, my aunt, Ms. Mitchell, Doc Isles, Mrs. Drennen, Ms. Gramaglia, Prof. Leuthold, <a href="http://www.popularlaboratories.com/">Prof. Swischuk</a>, and all the teachers at <a href="http://www.knewton.com">Knewton</a>.</div><div><br /></div><div>What do teachers make ... Teachers make a goddamn difference, now what about you?</div><br /><object width="425" height="344"><param name="movie" value="http://www.youtube.com/v/0xuFnP5N2uA&rel=0&color1=0xffffff&color2=0xececec&hl=en_US&feature=player_embedded&fs=1"><param name="allowFullScreen" value="true"><param name="allowScriptAccess" value="always"><embed src="http://www.youtube.com/v/0xuFnP5N2uA&rel=0&color1=0xffffff&color2=0xececec&hl=en_US&feature=player_embedded&fs=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="425" height="344"></embed></object><div><br /></div><div><a href="http://www.ted.com/talks/taylor_mali_what_teachers_make.html">Taylor Mali: What teachers make | Video on TED.com</a></div>pete mironhttp://www.blogger.com/profile/10573198508233227961noreply@blogger.com0