Monday, November 28, 2005

Into the breach

Ron sez:

I dove into the adstoo project with as much enthusiasm as I could muster, which I'm ashamed to say wasn't much. The situation was exacerbated by the fact that we had no Java development infrastructure. We were writing bareback, so to speak. We had no debugger. We were using JSP, but had no editor support for JSP syntax. (That turned into a real debugging nightmare. It could take many tens of minutes to find a single typographical error because the only indication that there was a problem was that the output just looked all wrong, but the actual problem could be very far away from the apparent source of the problem.)

Fortunately for me, I was assigned a junior engineer to work with/for me, and he actually knew what he was doing. While I struggled to learn the Java libraries and debugging techniques (I knew the basic language, but I had never done any serious development in it before) this guy just took the bull by the horns and pretty much just wrote the whole damn thing in a matter of weeks. I sometimes pull this old joke out of the dustbin, that in the ancient tradition of senior-junior relationships, he did all the work and I took all the credit.

That's not quite true. I did end up writing the credit card billing and accounting system, which is a nontrivial thing to get right. Fortunately for me, just before coming to Google I had taken some time to study computer security and cryptography, so I was actually well prepared for that particular task. Back in those days internal security was more or less nonexistent. All the engineers had root access to all of the servers. But I believe in planning ahead, and I anticipated the day when Google was not going to be a small company any more, and so I designed the billing system to be secure against even a dishonest employee with root access (which is not such an easy thing to do). I have no idea if they are still using my system, but if they are then I'd feel pretty confident that my credit card number was not going to get stolen.

But on the whole I was struggling, not just technically, but personally as well. The situation was exacerbated by my manager, who thought that the answer to my falling behind schedule was to start micromanaging me. That just made things worse. Much worse. I had been a senior scientist at a national research lab, essentially the equivalent of a tenured professor (but without the teaching responsibilities). I was used to being my own boss more or less, and I really resented being asked to make detailed lists of everything was going to do every single day. After all, Google had come asking me to join them, and so I felt I ought to be accorded more respect. But on the other hand I had to admit that I was not really performing, and so all the micromanagement was in some sense justified. I responded, I'm ashamed to admit, with some passive-aggressive head games, and so the situation deteriorated rapidly to the point where my manager and I were barely on speaking terms.

Things were made worse by the fact that I had been assigned an office mate who was also new to Google, and who was not part of the ads group. Most of the other ads group members were sharing offices (or cubicles) with other ads group members, and so I felt I wasn't really part of the club. On top of that, I was away from home and didn't really have a life up there in Northern California. The stress mounted. I started to get paranoid that I would get fired before reaching the one-year mark. I started experiencing stress-related health problems, some of which are still with me today. On more than one occasion I came that close to quitting. To this day I have no idea why I didn't.

It was about this time that I had my one and only meeting with Larry Page. It was to discuss the progress of the adstoo project and to set a launch date. My manager was there along with a couple of other people (including Doug I think). Things went smoothly until Larry suggested changing the way billing was handled. I don't remember the details, but my response was that this would be significant work. No one challenged me, but I found out later that the reaction of people in the room was something along the lines of, "Is he crazy? This ought to be a trivial change." This little incident turned out to have very far ranging repurcussions later, but that will have to wait for the next blog entry.

Somehow we actually managed to launch AdWords on schedule, in September of 2000. It still seems like a bloody miracle. Most of the credit goes to Jeremy, Ed and Schwim. It could not have been done without them.

I can still remember watching the very first ad roll in. It was for a company called Lively Lobsters. Two months ago, after five years of intending to do so, I finally bought myself a little toy stuffed lobster to commemorate the occasion. (Update on 12/9/2005: It appears that Lively Lobsters has gone out of business. There's some irony for you.)

About two weeks later all hell broke loose.

13 Comments:

Blogger Philipp said...

You got me really curious now...

2:40 PM  
Anonymous Zoli Erdos said...

Ron, this should be illagal! Like in soap operas, at the most exiting line...stay tuned .. ahhhhhhhh :-)))

3:03 PM  
Blogger http://search-engines-web.com/ said...

One bit of information that would really be helpful to clarify

Who came up with the idea of SPONSOR LINKS
The (two, now three) listings ABOVE the SERPs with the pastel background color..

Was there any concern/debate about ethics.

Whose idea was it to create the ADWORDS BOXES -then years later take them away.

The Google Sandbox (suggestion) tool - why did it NOT include numbers - like the inventory.overture tool?

How did the transition from Pay per View - to pay per click happen?

3:16 PM  
Anonymous AP said...

It appears that the boys at google didn't realize Ron's talent and lost out eh?

5:01 PM  
Anonymous John said...

Man, I am hooked.. very interesting.. keep going...:-)

9:09 PM  
Anonymous Anonymous said...

I have a question about the 20% time to pursue one's own interests. Must the project be a totally different project or may someone just do more work on their current projects?

11:40 PM  
Blogger netron said...

aah man - this is like one of those cliffhangers that you see in soap operas...

compulsive reading!

3:40 AM  
Anonymous Anonymous said...

Ron,

The lisper in me wonders why you didn't write a lisp program to generate your Java program -- at least then, you'd get to use all your lisp tools/skills, and the stuff would be syntactically correct (as long as it didn't "blow out" the girly-man Java development tools).

But then again, you'd probably have only gotten into big trouble for that sort of smartass behavior.

Also, I sympathize with your AMT/options/salary woes. A lot of people can't deal well with making monetary sacrifices for risky businesses -- it keeps them up at night.

If you own risky securities, you can always sell them and buy less-risky ones (but perhaps get whacked by the 365-day capital gains tax) -- but if the risky securities come as part of your job, you are really stuck. And you got screwd with the 5x revaluation! What a shame!

8:15 AM  
Anonymous Anonymous said...

Your blog is interesting. The Google persona I read about on tech sites (being super friendly and laid back) seems to be at odds with your experiences. There is a lot more business than engineering involved in your work there.

-Ivan

9:18 PM  
Blogger Kevin said...

I liked the lobster part

2:03 AM  
Blogger markg said...

Doug, When do we get our Xooglers Tee Shirts. Love the stories can't wait until you start telling about the Google Ski trips to Squaw Valley!!

3:00 PM  
Anonymous Ryan said...

Ron,

Here is some more irony for ya... I founded Lively Lobsters in 1998 and was the guy who placed the first AdWords ad you speak of. I am not sure if you are aware, but a nice young lady named Allegra emailed me requesting a high-res version of my company logo for use on some t-shirts being made for the engineers to celebrate. (I never received one... I have asked around and heard mixed news on whether they were ever made, and if they were, I would love to have one now!) You correctly stated that Lively Lobsters went out of business, but I am reviving it this fall. The only reason I shut it down in the first place was because I became so involved with other profitable AdWords advertising that it no longer made sense to focus on Lively Lobsters. These other AdWords ventures have kept me a faithful advertiser ever since, and I am one of Google's top spenders all time, having purchased literally many millions of dollars worth of clicks through my various companies. The power of AdWords itself is responsible for my success, and those very fruits are being used to resurrect Lively Lobsters. In a nutshell... your system (AdWords) made Lively Lobsters, then me, a success and now that success is coming back to revive Lively Lobsters! I am not sure if this is interesting to you or anyone else, but I thought I'd share my story when I came across this blog via a Google search for "Lively Lobsters" while satisfying my curiosity about references to Lively Lobsters still floating around. Thanks for everything you did for Google and AdWords... I owe ya some long-overdue lobsters. Get in touch with me and they are yours!

10:42 PM  
Blogger Ron said...

Hi Ryan,

Wow, what a great story! Glad to hear everything worked out so well. If you are spending millions on AdWords then you must be doing well for yourself indeed. Congratulations! And glad to hear the Lively Lobsters is rising from the broiler. Maybe I can return the favor by placing the first order on the new site :-)

I owe ya some long-overdue lobsters. Get in touch with me and they are yours!

Thanks! We'd gladly take you up on your offer, but we don't know how to reach you! Would you please send a clue to xooglers at gmail? Doug has some things he wants to ask you privately as well.

8:33 PM  

Post a Comment

<< Home