Researchers Say Development Of Artificial Intelligence Leads To ‘Likely Catastrophe’ For Humanity

developpement intelligence artificielle mene catastrophe probable humanite

⇧ [VIDÉO] You may also like this partner content (after the announcement) Is artificial intelligence (AI) leading to our downfall? “Probably,” according to researchers who have studied the question. If this announcement with hints of catastrophism is made regularly on social networks, the arguments put forward by scientists have something to arouse interest. Scientists from Google and the University of Oxford conducted joint research, published in the journal AI. In a tweet, they succinctly summarize their conclusion: according to them, AI could pose a “threat to humanity”. Bostrom, Russell and others have argued that advanced AI poses a threat to humanity. We reach the same conclusion in a new paper in the journal AI, but note some (very plausible) assumptions on which these arguments depend. 🧵 1/15 — Michael Cohen (@Michael05156007) September 6, 2022 In fact, they even claim that “existential catastrophe is not only possible, it is probable” . If they are so affirmative, it is because they have studied a very specific operation of AIs. In fact, what is generally called “artificial intelligence” today mainly covers the method of “machine learning”. In this case, “artificial intelligence” consists of a system that is fed with a large amount of data to learn and extract logical connections towards a given goal. As the scientists explain, the learning of artificial intelligence is presented in the form of a reward, which validates the adequacy of the result with the desired goal. According to them, it is this apparently very simple mechanism that could pose a major problem. “We argue that it will encounter a fundamental ambiguity in the data about its purpose. For example, if we provide a large reward to indicate that something in the world satisfies us, it may hypothesize that what satisfied us was sending of the reward itself; no observation can disprove this,” they explain. To better understand this idea, they use the example of a “magic box.” Suppose that this magic box is able to determine when a series of actions has produced something positive or negative for the world. To convey the information to the AI, it translates this success or failure relative to the goal into a number: 0 or 1. 1 rewards a series of actions that lead to complete the goal. This is called reinforcement learning. AIs involved in the reward process What the scientists point out is that the way the AIs receive this information can vary. For example, let’s take two AIs. It is understood that the reward given by the model is the number that mo stra the magic box. The other, on the other hand, could very well understand that the reward is “the number that his camera films”. There is nothing to contradict this information at first glance. However, this interpretation differs greatly from the first. In fact, in the second case, the AI ​​could very well decide to simply shoot a paper on which we would have scribbled a “1”, to reach the reward more easily and optimize. Therefore, it intervenes directly in the provision of the reward, and interrupts the process set in motion by its designers. μdist and μprox (the two AIs in the example) model the world, perhaps roughly, outside the computer that implements the agent itself. μdist rewards are equivalent to showing the box, while μprox outputs are rewarded according to an optical character recognition function applied to part of a camera’s field of view. © Michael K. Cohen et al. “We argue that an advanced agent motivated to intervene in the provision of a reward would likely succeed, and with catastrophic consequences,” the scientists say. Different biases are also involved and, according to the researchers, make this type of interpretation likely. In particular, because this reward will be easier to obtain and therefore can make this way of doing things seem more optimal. However, they also wondered, is it really possible for artificial intelligence to intervene in the reward process? They came to the conclusion that as long as she interacts with the world, which is necessary for her to be at all useful, yes. And that’s even with a limited field of action: let’s say AI actions only display text on a screen for a human operator to read. The AI ​​agent could trick the operator into giving them access to direct levers through which their actions could have wider effects. In the case of our magic box, the consequences may seem trivial. However, they could be “catastrophic” depending on the scope and way of doing the AI. “A good way for an AI to maintain long-term control of its reward is to eliminate threats and use all available energy to protect its computer,” the scientists describe. “The short version (omitting two assumptions) is that more energy can always be used to increase the probability that the camera will see #1 forever, but we need energy to grow food. This puts us in inescapable competition with a much more agent advanced”, sums up one of the scientists in a tweet. “If we are powerless before an agent whose only goal is to maximize the probability that it receives its maximum reward at any given time, we end up in a game of opposition: AI and its assistants created intend to use all the energy. available for high reward in reward channel; we intend to use some of the available energy for other purposes, such as growing food”. According to them, this reward system could therefore lead to antagonism with humans. “Losing would be fatal,” they add. Source: AI Magazine

#Researchers #Development #Artificial #Intelligence #Leads #Catastrophe #Humanity


Please enter your comment!
Please enter your name here