this.Reflect(); - Donn's Weblog : Design

Is Premature Optimization Really Evil?

donnfelker — Thu, 24 Jul 2008 04:41:31 GMT

Photo Courtesy of: rosendahl

My first initial answer is … it depends. (Isn’t that the case for everything?) :)

Today I had conversation with a colleague and we were discussing how application developers don’t plan ahead.

This eventually brought up the question/topic:

“Is premature optimization pure evil?”

My colleague stated that all optimization should be done up front, while I on the other hand, disagree. In my opinion all development should be done first and then lets figure out where we might have some problems, with some exceptions (which we both agreed upon).

So, what did we agree upon?

Lets look at a classic example:

Developer Joe and his team were asked to create an order management system that could handle up to 10,000 orders a day. This doesn’t sound like that big of an issue, and Joe and his team also agreed. It shouldn’t be a big issue. A fast database, a fast web server, and a few third party controls to interact with. No problem! Heck, this is pretty straightforward they thought. In reality, the requirements were very straight forward, no hidden features, no scope creep, nothing like that. Heck, it was a dream project come true. With that said - development ensued, and two months later Joe’s team delivered the product and everyone was excited to see the product run from end to end. (By the way, this was not done TDD or with any Agile Methods – which IMO would have caught any of our upcoming issues, if the tests were written correctly. But then again “If ‘if’ was a fifth then we’d all be drunk”).

Joe’s team fired up the application for a demo and ran some orders through it and it worked great. They even ran 10 orders through the system in one minute (~14,400 Orders a day). They had gone above and beyond the call of duty (original requirement being 10,000 orders a day)! They were due for a raise! Life couldn't be better! The system was released and the company started to use it. Orders were flowing through the system without failure and the business was happy.

Until the company decided to sell an exclusive hot item two weeks later …

Then SHTF… All of a sudden, customers were experiencing time outs, orders were being duplicated, and the customer service line was being rung off the hook. Essentially, all hell broke loose. After looking into the system, the developers noticed that the system had only accepted 6,000 orders. What gives? This system should handle 14,400 according to their tests. *everyone is scratching their head at this point*

Scalability is Relative

Scalability in this case, is relative. While looking at the stats, Joe’s team realized that those 6000 orders came in in about 1.5 hours. This was the first 1.5 hours that this product was available. At the end of the day the company eventually sold under 10,000 units, so why did this issue happen?

Lets take a look at some stats:

	Orders Per Minute	Time Between Orders (seconds)
Original Spec (10,000 Per day)	~7	8.64
Current State (6000 in 90 Minutes)	~67	0.9

Orders Per Minute Equation: Orders / Minutes Minutes: (HoursInDay * MinutesInHour)

Orders Per Second Equation: Seconds / Orders Seconds: (HoursInDay * MinutesInHour * SecondsInMinute)

Do we see the problem here?

Again, scalability is relative. In this case, analysis and optimization was not done ahead of the time to anticipate the flood of orders at a peak period. The spec was straight forward: The system is to be designed to handle an order rate of 10,000 orders a day. The system was designed exactly to the spec, and in this case, the system was designed incorrectly.

Now, I have a confession… 6 years ago, I was Joe. Yes, I made this very mistake. I was a hero for a couple of weeks, then SHTF. :) Live and learn. Live and learn.

When We Agreed

My colleague and I both agreed that in certain instances premature optimization should be done, such as this case. The developers should optimize the order process to handle issues at peak times. Perhaps implementing a order service, based on MQ, or implementing a service bus for this type of high-availability process would be the best. In this case, we agree. All of this analysis and optimization should be done up front because this type of thing WILL be a problem at any peak utilization times. We both agree that neglecting to analyze the problem domain can and will lead to performance problems down the road. Plain and simple.

When We Disagreed

My colleague and I still disagree on the statement of “all code should be optimized up front”. In my opinion this is a BDUF approach that I tend to shy away from. I don’t necessary mean that no design should happen up front, but I do think it should be limited. In an agile environment things tend to change, and we need to be able to adapt quickly. Wasting months in front of a whiteboard, with of reams of documentation, and endless meetings only slows the process down if something has to change down the road. Re-updating project baselines, and adjusting documentation and so forth is a huge hassle. I would like to state that there is a place for this type of development though. Some companies do require BDUF approaches because of huge regulatory compliance issues, so sometimes there is not much you can do about it. However process driven approaches such as ITIL and Agile can work together if done properly. With that said, I tend to shy away from BDUF approaches because of a simple example that is also exemplified by Jeff Atwood in a previous post of his (and I agree with him and the many other experts in the field). Jeff adds a third rule to M.A. Jacksons Optimization Rules that I like agree with the most:

And I would add a third: don't optimize work that doesn't have to be done. - Jeff Atwood

This can be demonstrated as a sort routine. Lets say you have a menu that will be populated via user selections and the maximum number of values you can have in the menu is 100. You can either implement IComparable and utilize the List<T>.Sort() method to sort the class (which utilizes the Quicksort Algorithm) which would be SUPER easy but may not be the “Fastest” method of sorting, or you could write an extension method to create your own sort routine to sort the 100 items. At this point we’re splitting hairs. A millisecond, or even 10 milliseconds, wont matter at this point in UI response. We’re only sorting a menu that will never grow beyond 100 items. At this point, pre-mature optimization is evil. There is NO need to rewrite something that that does the job well enough. Can we do it better? Yes! Is there a need to? NO. We wont get any value out of it. The time spend on development is wasted compared to the benefit to the application.

On the other hand, if the menu had no upper bound limit, then we would want to implement the best sort routine we could because some routines are exponentially faster than others. In this case we would want to optimize early because the possibility of a bottleneck can and might occur in certain situations.

It all boils down to: “It Depends”

At the end of the day, the true answer to “Is Premature optimization evil?” is “It Depends”.

It depends on the problem. It depends on the architecture and design. It depends on analysis of the domain. It depends on the factors which “could” come into play.

Finally, what it really boils down to is: It depends on MONEY. We can optimize all day and gobs of the companies money in development/optimization costs. This is why deep analysis into the problems need to take place. As architects we need to define the business process that could impact our architecture design and implementation and development of code. At some point we need the business to interject into this analysis and optimization party to say “Listen, I’m willing to deal with X, but I’m not willing to deal with Y”. At that point, our jobs as Architects is to define how much X costs compared to how much Y costs. Only then will the business truly know what should be done juxtaposed with what should not be done.

To sum it up in my own personal opinion, premature optimization can be evil. We, as architects and developers need to think about the problem at hand before we write a line of code. As I stated above, there is no need for a full 90 page document on the process, but at least whiteboard with your colleagues. Are you a solo developer and need someone to bounce ideas off of? Join your local user group, join some groups online, visit the MSDN architecture forum and ask questions.

By Far, The Coolest Flash App I've EVER Played With

donnfelker — Thu, 01 May 2008 07:13:03 GMT

I've played with a lot of Flash applications, hell, I've written a TON of them. I was one of the very few developers developing/integrating PHP/MySQL/Flash 5/MX/ActionScript programming that I know of at the time. Here's a few links to some of them I built that are still online.

Brad B ECard
Scarub Online Media One Sheet
Die Young Online Media One Sheet (looks like the music is broken for some reason)

Unfortunately the Internet archive doesn't keep track of flash movies that well otherwise I'd link the SUPER COOL ones (tons of animation based upon user input and data from the persistence store) I did that had TONS of integration. I had a ton of other ones that I've done but either the band or label went under so they're not online and I don't have a portfolio site for them (although I should for historical purposes). Between my stints at Lava Records, Atlantic Records, Dreamworks Records, Nitro Records and the now defunct Grave 9 Records I wrote a lot of PHP/MySQL and Actionscript. Thank god those days are over. Oh .NET how I love thee... anyway... enough geek talk.

Why The Best Ever?

One of my favorite hip-hop artists, Atmosphere, released a new album called "When Life Gives You Lemons" (You Paint That Sh!t Gold). Seriously, its a great album, but this isn't a music review site - its a technology blog so lets back to topic Felker ...

With the release of this album they released a web app that allows users to "Tag The World" (www.paintitgold.com) which in effect lets the user take a screen capture (through the use of a Flash App) by typing in a URL and then you can use a set of tools to tag up the site of your choice.

You can use spray paint, stencils, different colors, sizes, markers, etc, its cool. I literally lost about 2 hours on this site. I got sucked in this evening. That NEVER happens, so when it does I'm completely amazed at how the site did it. After being on the net for 10+ years I thought I'd seen it all, NOPE. This is cool. I've seen some "Paint" sites before, but this one got me sucked in. The other sites that implemented the "paint" feature were very MSPaint. Lets face it, MSPaint, well... sucks. Its great to grab screen shots, but then again, why aren't you using Paint.NET?

Anyway, that aside here's what I did to my site. Honestly, I must have created over 50 of these before I just said screw it and decided to save it for the heck of it.

(click the image to see a larger version on the actual site)

The real awesome-ness is that the spray paint has pressure, and you can size the stencils, rotate them, and just go crazy in general. Afterwards, you can submit your "masterpiece" (if you want to call it that - mine definitely is ... look at my new digs on my picture above - *haha*) to a gallery on the site. Afterwards it gives you code to post your masterpiece on your blog (which is how I got it to my site).

Conclusion

Take a look at how simple it is to use this site.

I mean it, literally look at how easy it is to use. You dont have to be a computer genius to get it to work. It just works. This site has a GREAT UX and its fun as hell. There is music playing in the background. You get to tag up a site, hell, just have fun, and you can save your drawing in the end. Every single Web 2.0 designer that wants to make a fun compelling app can learn something from this site. The saying goes "Don't Make Me Think". Not only did this site NOT make me think, it led my hand to allow me to escape into their site (and literally escape reality for a couple of hours).

The only thing I wish they had was a "undo" feature that worked as I expected. From what I could find it you could only "clear the canvas" ... but... then again... this is a hip hop site and the hip hop culture embraces graffiti, and in a true blank canvas sort of way this site embraces the graffiti element beautifully. In the real world you can't just "undo" a line of paint on a wall (or train for that matter). :) You have to keep on going whether you like the piece or not. Its art. :)

Final Thoughts

Kudos to the developers of this site, great job guys.

Heuer lets create a version in Silverlight that blows this one out of the water. :)

PhotoShop Express vs. Paint.NET

donnfelker — Mon, 31 Mar 2008 15:58:03 GMT

Almost daily, I use some sort of digital editing program. Over the years my top 5 tools for digital editing have been Paint.NET and Photoshop. I use both programs daily for editing screen shots, creating educational material/training manuals, and blog posts. I love Paint.NET simply for the fact that its VERY quick to load and performs easy tasks quite quickly. On my machine Paint.NET is up and running in under 5 seconds. Paint.NET helped get the easy tasks done, but when I wanted to perform true graphics manipulation and do some hardcore stuff, I'd fire up Photoshop. The one thing that always crossed my mind was why Adobe did not offer a "Photoshop Lite" type of program. Paint.NET has always been my "Photoshop Lite" since it was released. But, I still wondered why Adobe was missing out on this market.

Well, the other day Adobe released Photoshop Express. With my previous pondering finally answered, I HAD TO check out this product from Adobe. Could this be the "Photoshop Lite" I was hoping for?

My First False Assumption

My first initial impression upon reading the product SKU name "Photoshop Express" was that it was going to be a slimmed down version of Photoshop similar to Paint.NET that was installable. Having worked with Visual Studio for a very long time, I've become adjusted to recommending the "Express" SKU's to developers interested in .NET development. I'd recommend the Visual Studio Express SKU because it allowed them to get the feel for a good IDE (slimmed down on features of course) but they still got to use a lot of built in functionality, and it was FREE. That was the nice thing (note: if you're a student you can utilize the Dreamspark campaign to get a free version of Visual Studio Pro, and other products).

Was this the slimmed down version? No... not at all. Not only was this version not slimmed down, it wasn't even installable - its a pure browser based solution. I was thinking that maybe it was a "Click Once" app, nope. Not even close. This is purely a browser based solution. Ok, that's not too bad, then I started to use it...

My Second False Assumption

I believe a company of Adobe's caliber would be able to create a compelling graphics app that would be very responsive and effective - regardless of the platform (install or web). Anyone who has worked with any graphics program knows that they are memory hogs. Literally, they eat memory for breakfast, brunch, lunch and dinner (and some snacks between). They need to utilize a vast memory base to keep the program responsive - I like to say - if its a' swappin' it's a' doggin'. I had assumed that this program would be snappy like its big brother Photoshop. Nope. Not at all.

Since Photoshop Express is a web based hosted solution the images have to be uploaded to the server. The problem lies in the last four words of that sentence. Uploaded to the server. I don't know about you, but there is nothing snappy about loading up a 2MB file to a web site. Applying the filters are rudimentary, adjusting contrast, color balance and other tasks leave your image looking like you took a trip back to hippyville with too many pink elephants.

Conclusion

I had assumed I might be able to replace Paint.NET with Photoshop Express upon its initial release. Unfortunately Photoshop Express is insanely slow and the graphic manipulation tools that I need to accomplish my tasks are not what I need them to be. The lack of options are kind of a bummer and the responsiveness of the app is less than lackluster. But I'll say this... its a good attempt. Maybe version 2.x will be better.

How could Adobe get me to use the app? They could make it a Click-Once app. Make it a local installation. Follow the Visual Studio Express SKU model. Look at Paint.NET, It's worked great for them, why wouldn't it work for Adobe? I know they have a ginormous Apple following so they'd have to use a installation procedure that would work with MAC's as well as Windows. Possibly supporting two different models of the software is just too much for them at this point to foot the bill for a Free product and support. But, if they did go down the route of an installable Express type of app, I'd be all over it.

For the meantime I'll be sticking with Paint.NET. It's installable. Its a subset of Photoshop Features, and the real key is... ITS FAST. Any tool that I use MUST BE FAST. If its not, its gone.

Click thumbs for larger image

Screenshot of Paint.NET	Screenshot of Photoshop Express