Subscribe to our mailing list

* indicates required
Close

Sunday, May 17, 2009

WolframAlpha fails to impress

I been fooling around with WolframAlpha (the much-ballyhooed intelligent search engine) yesterday and today, and all I can say is, it feels very alpha.

I tried every query I could think of to get the annual consumption of electricity in the U.S., and got nowhere. On the other hand, if you just enter "electricity," your first hit is Coulomb's law 2.0mC, 5.0mC, 250cm, your second hit is 12A, 110V, and your third hit is diode 0.6 V. Which seems (how shall I say?) pretty useless.

As it turns out, WolframAlpha is also extraordinarily slow most of the time and hasn't performed well in load tests. Apparently Wolfram's AI doesn't extend to figuring out how to make something scale.

I'll suspend judgment on WA for a while longer, pending further testing. But right now it looks and smells like the answer to a question nobody asked.

7 comments:

  1. It's always risky putting something out in alpha form. The possible queries outweigh the supported queries, so we are always going to have cases (lots of them) where something we want isn't answered. So tinkering is required with the query and the knowledge.

    In spite of that, I do think it is an intriguing tool - not one that will be my default search engine yet, however.

    In some domains it is delightful (weather being the most obvious), and in others it simply doesn't know anything.

    Does it have any links/relationships with CYC, I wonder?

    As far as scalability is concerned, that will become a huge issue if/when it goes mainstream. The amount of compute power required for a relatively complex google query will be minute compared with one of these. If we hazard a guess (and I have no data, just trying a number to see) that a wolframalpha query takes 2 orders of magnitude more compute power than a google query of similar complexity, then the production wolframalpha operation will cause the lights to dim in a major city.

    I'll definitely play with it some more though.....

    ReplyDelete
  2. I'm not sure I understood right, but "annual electricity consumption US" gives "3.892 trillion kilowatt hours per year (2007 estimate)" straight away, is this what you were looking for?

    Also, they announced they are preforming load tests right now during the weekend. Friday was a soft launch, serious business comes on Monday I think.

    ReplyDelete
  3. True, there are still gaps in its apparent knowledge. The demo videos were spectacular, it will probably take a while to live up to those in typical performance.
    Worfram did say it was a long term project.

    ReplyDelete
  4. Of course it fails to impress: it's an advanced calculator touted by bloggers and press as being a google/wikipedia killer. It difficult to use it properly when you don't even know what it really is.

    Also, I wonder... is "Alpha" the product's name or is it supposed to mean that it's not even in beta yet?

    ReplyDelete
  5. http://www10.wolframalpha.com/input/?i=US+Electricity+consumption

    It wasn't that hard ...

    ReplyDelete
  6. Anonymous5:25 PM

    Heh, like the above comments state, your blog fails to impress.

    I mean, aren't you the same person who doesn't know how to interview without sounding like a sarcastic jackass?

    Based on this blog's content, you should definitely be checking for AssertionErrors, man. Stuart Grimshaw's comment is a great example of why.

    ReplyDelete
  7. Let's wait and see then shall we? Once it is up to it's full operational capacity, it would be interesting to see how it performs. Being touted as a Google or Wikipedia killer will bring about high expectations. Thanks!

    ReplyDelete

Add a comment. Registration required because trolls.