Jeopardy: The IBM Challenge

[The GIGGAS]

Re: Jeopardy: The IBM Challenge

The problem is, via its algorithm's it is never wrong. But based on the english language (unless it's bilingual? ha.) it is quite capable of being wrong.

There's an example on this thread alone.

I'm more curious in its humanistic characteristics, meaning is it capable of formulating an answer instantaneously as opposed to never, or if their algorithm is far more advanced.

With the hardware used this thing would potentially just answer the question immediately or never answer.

You've confused me with your wording here. I imagine IBM would continue to refine the program behind Watson aiming for 100% correctness based on the English language first and foremost to become useful for customer uses, like medicine and others. I took your last comment to say that you would have to program error into the machine so it could be human... but that doesn't make sense to me, as it should always be right.

The way I can see it getting around the problem about immediately or not at all in a non-Jeopardy environment is to give the top x answers that meet some kind of confidence threshold, and let the human figure out what is the best one for the situation.

[Seattlefan1]

Re: Jeopardy: The IBM Challenge

Whatever the outcome, it should be interesting nonetheless; having the two top players in the show's history against an IBM computer makes for interesting game play and an interesting show as well.

[SqueakyD]

Re: Jeopardy: The IBM Challenge

You've confused me with your wording here. I imagine IBM would continue to refine the program behind Watson aiming for 100% correctness based on the English language first and foremost to become useful for customer uses, like medicine and others. I took your last comment to say that you would have to program error into the machine so it could be human... but that doesn't make sense to me, as it should always be right.

The way I can see it getting around the problem about immediately or not at all in a non-Jeopardy environment is to give the top x answers that meet some kind of confidence threshold, and let the human figure out what is the best one for the situation.

The way I read it is that he might be comparing that to something like video game AI, such as in sports games where athletes are initially programmed to 100 perfect abilites and reactions, but some quirks or "mental lapses" are then programmed in afterwards for the sake of gameplay/fun factor, etc... I agree with you that I don't think that's the case with Watson, however... It's more of Watson not learning the language yet and that there is still a limited amount of information and sources given to him. The potential is only as far as it's programmers can provide.

I would love to see if Watson can use that confidence threshold you mention and combine it into some kind of risk-taking that's used in game shows. Should he ring in at a 50-60 percent confidence rating for a $200 answer? $2000? How close are the other contestants scores? I really am interested in how he works the Final Jeopardy round, where wagering the right amount is so important.

[The GIGGAS]

Re: Jeopardy: The IBM Challenge

The way I read it is that he might be comparing that to something like video game AI, such as in sports games where athletes are initially programmed to 100 perfect abilites and reactions, but some quirks or "mental lapses" are then programmed in afterwards for the sake of gameplay/fun factor, etc... I agree with you that I don't think that's the case with Watson, however... It's more of Watson not learning the language yet and that there is still a limited amount of information and sources given to him. The potential is only as far as it's programmers can provide.

I would love to see if Watson can use that confidence threshold you mention and combine it into some kind of risk-taking that's used in game shows. Should he ring in at a 50-60 percent confidence rating for a $200 answer? $2000? How close are the other contestants scores? I really am interested in how he works the Final Jeopardy round, where wagering the right amount is so important.

I don't know where I was reading this, but he's not gonna ring in if his confidence is below 60%, I believe. I'm not sure at the threshold, but he just plain won't. I agree with you on the Final Jeopardy round.

[Pared]

Re: Jeopardy: The IBM Challenge

In the NOVA video Speedy posted, Watson answers after one of the human contestants answers. One of the humans answers incorrectly "What is Mosquito". Thereafter, Watson answers "What is Mosquito". While they apparently fixed the bug so that Watson does not repeat incorrect answers from other contestants, it does seem to indicate that Watson does not always answer the question instantaneously (though for almost all of the other examples I've seen of it answering questions, it seems they are instantaneous).

This is part of the problem. It's not so much a "bug" but an algorithm that would typically lead to the problem being solved in a finite way.

The way I read it is that he might be comparing that to something like video game AI, such as in sports games where athletes are initially programmed to 100 perfect abilites and reactions, but some quirks or "mental lapses" are then programmed in afterwards for the sake of gameplay/fun factor, etc... I agree with you that I don't think that's the case with Watson, however... It's more of Watson not learning the language yet and that there is still a limited amount of information and sources given to him. The potential is only as far as it's programmers can provide.

I would love to see if Watson can use that confidence threshold you mention and combine it into some kind of risk-taking that's used in game shows. Should he ring in at a 50-60 percent confidence rating for a $200 answer? $2000? How close are the other contestants scores? I really am interested in how he works the Final Jeopardy round, where wagering the right amount is so important.

You get it.

The final jeopardy round should definitely be interesting, that's for sure.

The 60% confidence is interesting. I didn't know that. That allows some room for failure.

[p_rushing]

Re: Jeopardy: The IBM Challenge

In the testing, it failed big time for some of the games. There were times where all 3 of the best answers where wrong. They have loaded basically everything they could onto the drives for it. If it doesn't understand the question and the language/wording, then it will get it wrong.

There aren't many real business applications for the actual Watson because the thing is just too massive for most companies to invest in, but the the programming is applicable on a smaller level for many analytical things business want to do.

[Blzer]

Re: Jeopardy: The IBM Challenge

I wonder if Watson is going to be handling "interviews" with Alex, like its history on how it was built and stuff (pre-programmed scripts or something like that), or if at least a designer is going to step up behind him at all during it. They obviously won't just skip it.

[goh]

Re: Jeopardy: The IBM Challenge

Watching the first minute I thought no one was going to be able to buzz in! I also wonder if it's capable of answering incorrectly.

Me too. I don't think the other 2 guys knew how fast they had to buzz in at first but they figured it out.

Awesome. Good to know.

That gives it a sense of humanity. However, if the engineers continually modify the algorithm to fix these "errors" you potentially have a system that can never be wrong,

...or answers 42 to everything.

[superjames1992]

Re: Jeopardy: The IBM Challenge

Well, it's finally coming on here! First time I've watched Jeopardy in a few years, but I'm excited. Go humans!

I think Watson will win, though.

[The GIGGAS]

Re: Jeopardy: The IBM Challenge

It's coming on here too. Go Watson! Go IBM! Go Artificial Intelligence!

[superjames1992]

Re: Jeopardy: The IBM Challenge

Tied neck-and-beck at $5,000 after the first day. Watson was mowing them down, then struggled near the end, including answering the question with the same wrong answer one of the contestants had just answered.

[The GIGGAS]

Re: Jeopardy: The IBM Challenge

Man, I hate that they stopped it. I want to watch more. Hell, I would have enjoyed a 2 hour special on this with two games of Jeopardy. Anyway, it makes sense that Watson could reproduce an answer, considering that he doesn't hear/see the other contestants.

It's going to get really interesting tomorrow. Although, it was really neat to see him take a Daily Double.

[Fresh Tendrils]

Re: Jeopardy: The IBM Challenge

I completely forgot about this when I got home. Anybody know where I can watch it?

[SPTO]

Re: Jeopardy: The IBM Challenge

Yeah it was pretty interesting especially when you saw that Watson gave the same wrong answer as one of the other contestants. I have to say for AI I think it's damn well impressive. I can't wait to see how it does tomorrow.

[JohnnytheSkin]

Re: Jeopardy: The IBM Challenge

Me too. I don't think the other 2 guys knew how fast they had to buzz in at first but they figured it out.

...or answers 42 to everything.

God I hope more people got this. Well played, sir...well played.