Subscribe to our mailing list

* indicates required
Close

Thursday, March 26, 2009

Twice-as-fast isn't good enough

I've been involved in a number of performance tuning efforts over the years, and one thing I've noticed over and over again is how hard it is to get anyone to notice a 2x speed increase.

I mentioned this to an acquaintance once, a programmer of great skill whose opinion I value. He nodded and said matter-of-factly: "A twofold speedup is not a worthwhile performance gain."

I was struck by the finality of his statement. In fact, I questioned it. He challenged me back:

"What's something you do every day on the computer that takes a significant amount of time?" he quizzed me.

"Well, rotating a large image in Photoshop takes a long time," I said. (Bear in mind, this was in the early 1990s, when most Macs ran at 10 MHz.)

"How long does that take?" John asked.

"It can easily take 45 seconds," I said.

"Okay, so if it took 22 seconds, would it change your life?"

I stopped and thought. I could see where he was going. He was right, of course. Reducing a 45-second job to 20 seconds, or even ten (a speedup of more than four-fold), would not materially affect my productivity or my quality of life; I would still be spending way too much time babysitting the machine, waiting for it to finish a fairly simple operation.

My friend made the case that when people have to spend more than five seconds waiting for the machine to finish doing something, it becomes an issue. He cited some research (by HP? I can't remember) to the effect that if more than five seconds elapses with no visual indication of anything changing, most people start to worry that the machine might be locking up. Mind you, this was back when desktop computers were single-tasking and most applications lacked a "progress bar" for lengthy operations. (Photoshop was an exception.) You couldn't switch contexts and go do something else in another open program. You had to wait patiently and hope the operation finished normally. On the Mac, you had the spinning "beachball cursor," which you hoped would eventually stop spinning. Sometimes it didn't.

I asked my friend what kind of performance improvement he considered "worthwhile." He said: "An order of magnitude."

I thought about it. He was right. The lengthy operations that drove me crazy on a regular basis tended to be under a minute in length (anything longer than that meant taking a coffee break), and obviously, shortening a 60-second operation to 6 seconds would be a quality-of-life boon (it would positively impact productivity), whereas shortening it to 30 seconds would make little difference.

In the late 1990s, I bought an aftermarket CPU upgrade for one of my Macs. It boosted overall processing speed by a factor of two-and-a-half. I hardly noticed. My PageMaker file saves went from two minutes to one. My Photoshop image-rotates that previously took 30 seconds, took about 15. The misery factor went down 50%. But I was still miserable.

So nowadays, when I hear someone (like Sun, with its ZFS file system) bragging about a 100% speedup of this-or-that operation because of realtime compression (or whatever), I just snicker. In this business, a two-fold speedup (of nearly anything) buys you 18 months in Moore time; then you're back where you started.

Don't delude yourself. In very few cases will your life be changed by a two-fold speedup of anything computer-related. If your business plan (for a new technology) is predicated on speed, and the promised performance boost is less than a factor of ten (a full order of magnitude), rethink what you're doing. "Twice as fast" is not a competitive advantage. Not by a long shot.

18 comments:

  1. A counterexample: Your game runs at 15 FPS. You upgrade your system and your game runs at 30 FPS now. Was it worth, or should it run with 150 FPS?

    ReplyDelete
  2. Surely what was shown was not that you need a larger improvement, but that improvements are more noticeable when they cross certain barriers.

    "If your business plan (for a new technology) is predicated on speed, and the promised performance boost is less than a factor of ten (a full order of magnitude), rethink what you're doing."

    What? How about a shift from 10 hours to 5? Or a shift from 10 hours to 2? Take something that would take more than one business day and make it less than that.

    Another example, take something that would take a work week and get it to run in a day. You think that's unimportant?

    "Okay, so if it took 22 seconds, would it change your life?"

    It depends on how often I'm doing that thing. It's entirely possible that this is the case. That speed improvement might just push it past the barrier that allows me to do something more important with it. Maybe now it's feasible to run larger batch scripts to run the program

    "The misery factor went down 50%. But I was still miserable."

    But less miserable, which is surely the most important thing.

    "In this business, a two-fold speedup (of nearly anything) buys you 18 months in Moore time; then you're back where you started."

    So with a bit of work we can get the same improvement as waiting a year and a half? Not doing this sounds *astonishingly* lazy.

    A simple rebuttal to the general idea is that if you improve it by a factor of two, then someone else does, and someone else does, you've got your order of magnitude.

    Please do not assume that my time is worthless when developing software I will use.

    Ian

    ReplyDelete
  3. I completely agree with the impression a user might have with a twice-as-fast solution.

    But I do not agree with the conclusion: twice-as-fast is not worth the effort/money.
    A software twice-as-fast means always beeing 18 month in the future compared to your competitors.
    In addtition, if it would not be worth to go for a X% speedup. You will probably never get a product being 10000 times faster - like it has happened with CPU/GPUs etc.

    ReplyDelete
  4. Moore's dead, man. Moore's dead.

    ReplyDelete
  5. I've always been of the opinion that "twice as fast" and "an order of magnitude faster" don't tell you anything about a user's expectations / needs, so I usually aim for either "fast enough to make people happy" or "so fast that I feel all warm and fuzzy inside".

    ReplyDelete
  6. get 4 2x speedups and you're at 16x...

    ReplyDelete
  7. See your previous post on optimizations by doing less work. If the 2x speedup is inside a loop, it's more valuable. If you're a photographer who rotates 12 images a day, no it's not a big speedup. If you're the Picassa server that does it tens of thousands of times a day, yeah it's probably useful. If it means the company can use 1/2 the hardware to provide the same performance to its userbase, it's worth it.

    ReplyDelete
  8. This is basically Big-O that we are talking about. If I half the time it takes for a Bubble Sort to run, it still takes forever. If I move to Heap Sort, then I am gettings somewhere. If I then moves from Heap Sort to Merg Sort, the gains just aren't as noticeable (assuming Merg is faster by a constant, but not in Big-O).

    ReplyDelete
  9. Yeah, but Blackdog has it right on. Bad Wolf, of course we're going to look for the major program structuring perf issues. But suppose your software is 100 man-months of development sunk into a distributed high capacity "heap sort". You have a deadline to meet. And the end user is experiencing 1500ms response times, and are going to leave if you ship like that. Sure, you could try switching to "quick sort" would be perhaps another 50 man-months of effort. There's just no time.

    Now say you have 4 people working for one month to fix the perf, and each finds a "measly" 2x speedup (in reality, you take every 5-15% you can get, but for sake of argument). In this miracle month, you've gone from notacable lag to <100ms response time.

    In the real world, you take every little improvement you can get because it gets you closer to shipping that much sooner. Starting over may not be an option.

    ReplyDelete
  10. 2x matters and also women DO care about penis size.

    ReplyDelete
  11. I think there is a far lager point missed here. Everyone is focused on the effect of a single user on their PC, there are a lot of servers running in datacenters consuming a much larger per CPU carbon/energy footprint. If you reduce CPU requirements by 2x you are saving a boatload of carbon/energy/$$.

    ReplyDelete
  12. I'm in 512 kbps broadband, and longing for 1 Mbps. That *would* change my life.

    ReplyDelete
  13. Getting another 30 seconds from a 60 second operation is a worthwhile improvement. What could you do with those 30 seconds? On a production level it may change the products / operations per day from perhaps 600 to 1200, which is relatively small, but the time you've gained is large.

    What else could you do with that time?

    ReplyDelete
  14. Counterexample: 100% speed improvement on code so that your regression suite takes 4 hours instead of 8 means someone only has to stick around till 9PM rather than 1AM. It happens dude...

    ReplyDelete
  15. If you could increase 2x the response time of all the queries made to my database server, my life would be definitely better.

    ReplyDelete
  16. While performance improvements are always important, choosing when to make the user wait is almost as important as how long you make them wait. Every user interface has a cadence to it. There are parts of the the process where the user expects to continue working without perceptible delay and others when they've reached some boundary condition where a delay is acceptable, maybe even expected. Good software designers know how to take advantage of this and budget computing time across the users expectations.

    ReplyDelete
  17. WTF? It's a matter of coding time, leverage, and mission-criticalness.

    If I'm writing arbitrage code in finance, a *2 speedup would be huge. Ditto if I'm writing blade server code -- you'd need half as many computers. Also if I'm writing compilers, because a vast horde will benefit from a 1% speedup.

    If nothing else, think about the power consumption!

    ReplyDelete

Add a comment. Registration required because trolls.