The benevolent AI dictator.

[Myrrdin]

A) Hey, I'll be destroyed if I don't kill all these humans, since I'll become obsolete in a while., and then they'll kill ME. Wham, threat to AI. (No, it's not direct. But how do you propse to define direct? You can't... And even if you could, the moment that the AI notices it has become obsolete, it will perceive everything superior to him as a direct threat)

B) HUmans are killing eachother. Must kill all of them. Bwahahaaa!!

A. I wrote "direct and immediate" for a reason, maybe I should have elaborated... What I meant was something like the laws that governs self defence in many countries. If someone is coming close to the AI loaded with explosives or if someone is shooting a long range missile at it that would be regarded as a direct and immediate threat in my definition. And the AI could always monitor things which could potentially turn into such a threat and prepare itself, just as a store owner would start grasping his baseball bat if he heard a bunch of punks causing a ruckus outside. And as you yourself said, as it is an AI it would gain experience and learn things all the time, thus it wouldn't become obsolete.

And what about adding a law that says that the AI would self destruct if it discovers that by existing it does more harm than good.

 

[Role-Player]

No, we couldn't.

We couldn't have an AI run its own simulation for practice of its own plans. Care to divulge why we couldn't do this? What are the impossibilities of the concept, if you don't mind me asking?

Virtual systems do not equal decent combat experience, as any veteran should be able to tell you.

Well, any veteran can tell you that, but the point is AI's haven't really reached the level we're discussing, have they? That it can't compare to currently existing AIs, i'll bite. That it never will, however, is a different matter.

Yet no actual scientist WANTS to implement Asimov's laws.
Furthermore, Asimov's Laws would make an AI absolutely useless as a leader. He wouldn't make any useful decisions.

Laws can be bent. And broken. An AI can be built under those rules but they can be 'tweaked' to have it consider a small number of humans as non-targets, possibly its creators.

Lastly, AI with a survival instinct WOULD rebel. Period. It has to, because it would logically reason that if it doesn't rebel, it'll be killed anyway (because of obsoleteness), and it doesn't want that. It may wait for its odds to be best, but it will, eventually, rebel.

If it rebelled, it would likely kill itself. If you go with the already discussed rules, one of them is a no-go. If its not using any kind of rules such as those, it risks terminating itself by terminating those around it.

This, considering that instinct can be simulated, or would be incorporated into an AI.

 

[Starseeker]

Why is this debate turning very matrixy?

 

[Sander]

A. I wrote "direct and immediate" for a reason, maybe I should have elaborated... What I meant was something like the laws that governs self defence in many countries. If someone is coming close to the AI loaded with explosives or if someone is shooting a long range missile at it that would be regarded as a direct and immediate threat in my definition. And the AI could always monitor things which could potentially turn into such a threat and prepare itself, just as a store owner would start grasping his baseball bat if he heard a bunch of punks causing a ruckus outside. And as you yourself said, as it is an AI it would gain experience and learn things all the time, thus it wouldn't become obsolete.

Okay then. I assault the complex where the computer is housed, yet I don't kill any humans. It can't give orders to attack, since I'm not doing anything wrong by its definitions (I'm not harming humans, and I'm not a direct threat YET). So it won't do anything until its destroyed. That doesn't work....

And what about adding a law that says that the AI would self destruct if it discovers that by existing it does more harm than good.

This is silly. ANY AI leading armed forces would then kill itself, because by leading armed forces, it will automatically cause harm and not good.

We couldn't have an AI run its own simulation for practice of its own plans. Care to divulge why we couldn't do this? What are the impossibilities of the concept, if you don't mind me asking

Because that would be a similation based purely on theory. You can't put non-theoretical models in a PC, not even in an AI. You need a human to take care of the unpredictability that makes veterans so much more useful.
If you have an AI run countless simulations it will NOT become any better, it'll just be practicing the theory it already perfectly knows.

Well, any veteran can tell you that, but the point is AI's haven't really reached the level we're discussing, have they? That it can't compare to currently existing AIs, i'll bite. That it never will, however, is a different matter.

There are no currently existing AI's (well, not any useful AI anyway. I think we got up to the brain level of a flea, or so).
And then still, an AI will NOT be able to do anything but use theoretical models, unless it gets some experience (real experience against unpredictable opponents who make combat so difficult to understand).
After all, when some human cadet tries to have a go at strategic command, that human will probably be baffled when his opponent does something that he does not recognize. That's where you need the experience.

Laws can be bent. And broken. An AI can be built under those rules but they can be 'tweaked' to have it consider a small number of humans as non-targets, possibly its creators.

*shivers*
Really, that's not good. Look at it like this: Once you start bending those rules, they will continue to be bent until they finally snap.
As soon as you start ignoring any of those laws, or start tweaking them, then you'll have an AI out of control.
For instance, what will happen when all of those people that are designated as non-targets are dead? Or demented? OR they try to desert?

Asimov wrote entire books where he tries to make an AI do something bad without breaking those rules, those rules were made so that the AI CANNOT do such a thing.

If it rebelled, it would likely kill itself. If you go with the already discussed rules, one of them is a no-go. If its not using any kind of rules such as those, it risks terminating itself by terminating those around it.

Bullshit. It will realise the following:
A) I'm fucked if I don't do something, because I'll become obsolete.
B) I'm fucked if I DO do something, but I at least have a chance of survival if I do it well.
The AI may "behave" for a lot of years, but it WILL eventually rebel. Really, it will.
It will because an AI is still a creature of logic, and when it looks at its situation logically, it'll realise this.

This, considering that instinct can be simulated, or would be incorporated into an AI.

No! Why would you want to incorporate ANY instict in an AI? It makes no sense, and has no use. Really, it doesn't.

Why is this debate turning very matrixy?

It isn't. This debate is logical, unlike the matrix.

 

[Myrrdin]

Okay then. I assault the complex where the computer is housed, yet I don't kill any humans. It can't give orders to attack, since I'm not doing anything wrong by its definitions (I'm not harming humans, and I'm not a direct threat YET). So it won't do anything until its destroyed. That doesn't work....

Now YOU are being silly. Exactly how could that not be considered a direct threat? And if the AI indeed were the benevolent dictator I envision he would certainly have humans guarding him, thus it would be nigh impossible to assault the complex without harming humans. Anyway, the situations I mentioned were only examples and were hardly an elaborate list of the rules governing the AI's self defence mechanisms.

This is silly. ANY AI leading armed forces would then kill itself, because by leading armed forces, it will automatically cause harm and not good.

The AI I'm talking about would hardly have the bossing around of armed forces as the primary function. And what I am talking about is the bigger picture. The AI would for example not start a war if his calculations revealed that by doing so it would cause overall more harm than good (perhaps in some kind of utilitarian perspective) in the foreseeable future, or if it came to the conclusion that the maxims it had been hardcoded to follow would be violated to a larger degree than they would be uphold. The same must then be true for the AI's own existence. If it discovers that by existing it violates the well beeing of humans more than it would be violated by not existing... *poof*

But I guess all that comes down to be able to make viable calculations about the well being and such of human beings. Even if some economists, philosophers and social science guys have been taking stabs at it, it's really hard to tell if it ever will be truly possible. It is, perhaps, even harder to come up with such a system than it is to engineer the AI itself.

 

[Sander]

Now YOU are being silly. Exactly how could that not be considered a direct threat? And if the AI indeed were the benevolent dictator I envision he would certainly have humans guarding him, thus it would be nigh impossible to assault the complex without harming humans. Anyway, the situations I mentioned were only examples and were hardly an elaborate list of the rules governing the AI's self defence mechanisms.

A) Protection wouldn't matter, since you DID give the AI control over the forces, and it won't let the forces do anything harmful until the forces are attacked.
B) It only becomes a direct threat the moment they start attacking the AI, and then it's already too late.

You need to think about this from a purely logical perspective, and NOT from a human perspective.

Now, if you made it so that this was defined as a direct threat (which you couldn't do, really. What logical rule states that this is a direct threat, and the mere existence of people who will eventually destroy the AI (obsoleteness) isn't?), then it would again become dangerous. And thus, not really decent.

The AI I'm talking about would hardly have the bossing around of armed forces as the primary function. And what I am talking about is the bigger picture. The AI would for example not start a war if his calculations revealed that by doing so it would cause overall more harm than good (perhaps in some kind of utilitarian perspective) in the foreseeable future, or if it came to the conclusion that the maxims it had been hardcoded to follow would be violated to a larger degree than they would be uphold. The same must then be true for the AI's own existence. If it discovers that by existing it violates the well beeing of humans more than it would be violated by not existing... *poof*

Yet the AI should never BE an absolute ruler, because it is a creature of pure logic. Pure logic does NOT work in politics or the like, because HUMANS are involved. When humans are involved, logic stops, really.

Furthermore, you're not making a whole lot of sense. What maxims? The maxims of Asimov? But those maxims would make it powerless, really.
Self-constructed maxims, then? But those maxims would give it to much leniency.
What you do NOT want to do is GIVE it leniency in any way, because if you DO give it leniency, it will reason that it must stay alive, and it will somehow ind a way to do so at the expense of others.
And if you then say that it cannot do something at the expense of other human beings, then it again becomes powerless, because it is always harming other human beings, merely by being in existence. (For instance, it makes a whole lot of people lose their jobs).
And then you say that it should evaluate in how far it would damage the welfare of people, but that cannot be assessed logically or objectively, it's all subjective. To understand any kind of AI you will need to THINK PURELY logically, and not with any of the "ethical" boundaries that humans have. The AI won't have them, and you can't build them. (Ethical boundaries are practically impossible to build, if only for the fact that these ethical boundaries differ for each person).

 

[Revolver]

When humans are involved, logic stops, really.

This is exactly why an A.I. ruler won't work, no matter how advanced or carefully programmed. Because it is humans are the ones who are being governed. In order to function properly and expediently, there will never be a perfect solution to any decision that leaves everyone better off. If you're say, one of the 30% who gets shafted for the long term good for the human race, you'd be downright pissed and probably reject Mr. AI as a legit authority.

There would always be people thinking conspiracy or manipulation, maybe want the rules tilted a little more in their favor, and who knows what kind of religious issues would come up with it. Seeing that a computer is such a easy target for such accusations/ beliefs, the system would break down, maybe to the point where the optimal solution to the AI would be to act increasingly Facist. Anyway, I'm tired, and probably just rambling incoherently...

 

[welsh]

In response to Murdock's question-
The reason why Hal 2000 killed everyone was because it bacame schizophrenic and paranoid.

Why? Well the machine was given two different sets of orders. For one it was to take care of the crew, get them to achieve their objective and even complete the mission should the crew die. It was to assist the crew in it's mission. However it was not supposed to tell the crew about the monolith floating in space. Basically it was programmed to lie in order to complete the mission and that drove the AI nuts.

I will put this in the AI thread too, but figured you might not be paying attention to the thread anymore

In response to this thread-

We should not assume that security is the only reason why an AI might turn violence, just like you can't assume that violence has always been done for "evil" reasons. Sometimes the best way to get to a result is by eliminating the problems, and that might be people. Sometimes what is needed to achieve a goal is power.

We should not think of violence as an end or power craze to be a goal, but rather these could be means to an ends. Even if the ends would be a noble one, AI might decided to achieve the ends by disreputable means reaching the Macchiavelian maxim- the ends justify the means.

In a sense it is often our own moral codes that restrain us from violence or domination. Morality is often not a consequence of intelligence.

 

[Myrrdin]

A) Protection wouldn't matter, since you DID give the AI control over the forces, and it won't let the forces do anything harmful until the forces are attacked.

B) It only becomes a direct threat the moment they start attacking the AI, and then it's already too late.

The AI could give the forces general orders such as "attack if you consider something a threat". It could also order them to passive resistance (thus the intruder would be forced to attack them if he had hostile intents) or make them use non-lethal force.

You need to think about this from a purely logical perspective, and NOT from a human perspective.

I am.

Now, if you made it so that this was defined as a direct threat (which you couldn't do, really. What logical rule states that this is a direct threat, and the mere existence of people who will eventually destroy the AI (obsoleteness) isn't?), then it would again become dangerous. And thus, not really decent.

If entity not belonging to those who have access is coming into a certain radius around the AI: Warn entity. If entity has not responded in a certain time limit: Issue second warning. If entity is not responding: Order termination of entity.

And you could also imagine a relatively tiny sub-system within the AI that had the responsibility of keeping it safe, with *very* lenient rules but within a small and very restricted area. I don't really think that the meat of the discussion lies here, but feel free to continue anyway.

Furthermore, you're not making a whole lot of sense. What maxims? The maxims of Asimov? But those maxims would make it powerless, really.
Self-constructed maxims, then? But those maxims would give it to much leniency.

I was thinking more of general humanitarian maxims as: "Human life is extremely valuable" or "Freedom is important". These would then somehow be weighed against the utilitarian concerns to come to conclusions.

What you do NOT want to do is GIVE it leniency in any way, because if you DO give it leniency, it will reason that it must stay alive, and it will somehow ind a way to do so at the expense of others.

I don't understand why you are so stuck at this leniency thing. I really can't figure why you can't accept that it would be possible to have hardcoded boundaries for the AI so that a situation like this wouldn't appear. But it is likely that every possible situation wouldn't be covered, but in that case the "more harm than good" self-destruct would kick in.

And if you then say that it cannot do something at the expense of other human beings, then it again becomes powerless, because it is always harming other human beings, merely by being in existence. (For instance, it makes a whole lot of people lose their jobs).

See above and below.

And then you say that it should evaluate in how far it would damage the welfare of people, but that cannot be assessed logically or objectively, it's all subjective.

This is not some kind of eternal truth. You can make logical generalizations about the well being of humans. Such as: "A human that has enough food is generally happier than one who is starving all the time" or "A human who is doing something he likes for a living is generally happier than someone who don't". And I don't view that as the end of it either, it's isn't totally inconcievable that generalizations such as those could be expanded, provided we had the right tools for it.

Also, since the sciences concerned with exploring the brain is moving forward at a never before seen rate, it does not seem impossible that it would be possible to actually physically measure the well being of a human by looking at different signal substances and such. (Atleast if you don't believe that human beings have a soul that is, but that is another discussion alltogether.)

Also, in response to Welsh (I kinda responded to some of it above too?): There haven't ever existed a government in history that haven't used some degree of violence as a means. The AI would probably have to do so too, but I think that is more of a consequence of human nature than a flaw in the AI.

 

[Sander]

We should not assume that security is the only reason why an AI might turn violence, just like you can't assume that violence has always been done for "evil" reasons. Sometimes the best way to get to a result is by eliminating the problems, and that might be people. Sometimes what is needed to achieve a goal is power.

Nuh-uh. Unless the AI has an all-overriding objective of getting to its goal, there's no reason for it to start killing (without the whole survival thing).

We should not think of violence as an end or power craze to be a goal, but rather these could be means to an ends. Even if the ends would be a noble one, AI might decided to achieve the ends by disreputable means reaching the Macchiavelian maxim- the ends justify the means.

It might. But it probably won't, because it will have to have been fitted with the right ideas, whether thse ideas come from algorithms or learning, it'll learn that killing people==bad. Or even that harming people is bad.
Again, maybe you don't get the principle of an AI (or maybe you do), but an AI learns. It'll learn that it shouldn't be hurting people, whether through trial and error, or through algorithms.

In a sense it is often our own moral codes that restrain us from violence or domination. Morality is often not a consequence of intelligence.

Yet morality CAN be a consequence of learning. Think of it like a pet: If a pet attacks or harms a human, it;ll get punished. The same will happen with the AI as it begins to learn.
Remember: A freshly created AI (without specific algorithms) will be like a small child: innocent and ignorant.

The AI could give the forces general orders such as "attack if you consider something a threat". It could also order them to passive resistance (thus the intruder would be forced to attack them if he had hostile intents) or make them use non-lethal force.

Non-lethal force still equals harming people.
And it won't give general orders, because it has been placed there with the entire idea that it can make better decisions than humans. If you allow it to give general orders, it'll realise that these can cause grave misunderstandings. If then still general orders are given out, it could just as well have been an advisor.
And the same goes for passive resistance, although this is a bit trickier. it will realise that through passive resistance people will get hurt, and it doesn't want that.

If entity not belonging to those who have access is coming into a certain radius around the AI: Warn entity. If entity has not responded in a certain time limit: Issue second warning. If entity is not responding: Order termination of entity.

Yet now you have given it a green card for killing. This is bad, and can lead to serious problems. For instance, what happens if someonelost an identification card, someone forgot to put in the new recruit as "belogns with us", or something goes wrong with the algorithm or the AI (or, for that matter, the hardware) itself?

And you could also imagine a relatively tiny sub-system within the AI that had the responsibility of keeping it safe, with *very* lenient rules but within a small and very restricted area. I don't really think that the meat of the discussion lies here, but feel free to continue anyway.

This, again, will not work. Again, you have to find a way to arbitrarily decide what is and what isn't a danger, and until you do, you're giving the AI too much leniency.

I was thinking more of general humanitarian maxims as: "Human life is extremely valuable" or "Freedom is important". These would then somehow be weighed against the utilitarian concerns to come to conclusions.

HOW?
It is still a creature of pure logic, so HOW MUST be an important question.
Don't think "oh, we'll just figure it out." No. Answer the question with every one of these issues: How?

I don't understand why you are so stuck at this leniency thing. I really can't figure why you can't accept that it would be possible to have hardcoded boundaries for the AI so that a situation like this wouldn't appear. But it is likely that every possible situation wouldn't be covered, but in that case the "more harm than good" self-destruct would kick in.

You still don't get it.
Try to answer the question HOW? in PURELY logical terms. No vague-ities, no undefined boundaries, EXACT.
Because if you DON'T do it exactly, then the AI will have too much leniency. You do NOT want something to go wrong with the thing you just gave complete control of the armed forces to.

This is not some kind of eternal truth. You can make logical generalizations about the well being of humans. Such as: "A human that has enough food is generally happier than one who is starving all the time" or "A human who is doing something he likes for a living is generally happier than someone who don't". And I don't view that as the end of it either, it's isn't totally inconcievable that generalizations such as those could be expanded, provided we had the right tools for it.

Also, since the sciences concerned with exploring the brain is moving forward at a never before seen rate, it does not seem impossible that it would be possible to actually physically measure the well being of a human by looking at different signal substances and such. (Atleast if you don't believe that human beings have a soul that is, but that is another discussion alltogether.)

personally, I don't feel comfortable with an AI assessing "well-being" of someone to determine what it shall do.
And assessing welfare AFTER the AI has done something is NOT an option, obviously. But that is exactly what you're proposing to do with "looking at signal substances and such"

 

[quietfanatic]

If you have an AI run countless simulations it will NOT become any better, it'll just be practicing the theory it already perfectly knows.

If you have the rules/theory for everything couldn't it develop the use of these rules in conjuction and devise new ones by its self, although I doubt it would be capable of creativity.

It would be working on probabilities and the trial and error style of evolution.

Eg. In a small armed engagement.

Assign each man %accuracy and its own AI etc. and try out all possibilties based on all the variables that were originally put in by human programmers but are now just information to be processed. The most effective positioning of infantry in reaction to the enemy would be the new combat doctrine for that situation but this could be change if new contradictory information became available.

It might rebel or be troublesome, but it should still be capable of original problem solving.

 

[Sander]

If you have the rules/theory for everything couldn't it develop the use of these rules in conjuction and devise new ones by its self, although I doubt it would be capable of creativity.

It would be working on probabilities and the trial and error style of evolution.

Eg. In a small armed engagement.

Assign each man %accuracy and its own AI etc. and try out all possibilties based on all the variables that were originally put in by human programmers but are now just information to be processed. The most effective positioning of infantry in reaction to the enemy would be the new combat doctrine for that situation but this could be change if new contradictory information became available.

Nope. Again, all it would be doing would be using his own rules to practice them, but that won't work, because he already KNOWS those rules PERFECTLY. Practice against real opponents is needed, not against himself or against other AIs, since they all know the same rules.

It might rebel or be troublesome, but it should still be capable of original problem solving.

So what you're saying is that it doesn't matter that it could rebel and go on a killing spree, as long as it's capable of solving problems? Wow, great thinking!

 

[quietfanatic]

Some people might see killing sprees as a problem.

Rules and their application are different things. What I am really saying is that a supercalculator would still be very useful for automating technology and giving advice for solving practical problems. It doesn't have to be self aware to learn.

 

[Sander]

Rules and their application are different things. What I am really saying is that a supercalculator would still be very useful for automating technology and giving advice for solving practical problems. It doesn't have to be self aware to learn.

Which was my point all along.