"The two chatbots came to create their own changes to English that made it easier for them to work – but which remained mysterious to the humans that supposedly look after them.
The bizarre discussions came as Facebook challenged its chatbots to try and negotiate with each other over a trade, attempting to swap hats, balls and books, each of which were given a certain value. But they quickly broke down as the robots appeared to chant at each other in a language that they each understood but which appears mostly incomprehensible to humans."
Generally speaking a Black Box system involves a series of inputs, and observations of outputs; however, the observer doesn’t understand how the outputs are obtained. This is generally typical in applications like self-driving cars, etc. While the car may be able to drive, it’s very difficult to determine the processes involved in how the car is reaching its conclusions/actions at a given point in time. In essence, it is a system, whereby the output result cannot be audited to determine how said output was determined. Take the OODA loop, remove everything but the A, and that’s what you get.
A good analogy would be if I gave you a math problem. You give me an answer. The answer may be right, or it may be wrong, but you didn’t show your work and I have no way of knowing how you reached your conclusion regarding the submitted answer.
This technique is often used in Machine/Deep Learning/Neural Networks. You ultimately end up with a inscrutability problem from the results provided. There’s a level of opacity inherent to these systems, even to the designers/programmers creating them.
I personally don’t think the risk/reward ratio is skewed enough to justify these techniques, regardless of the perceived benefit of the outcomes.
We are currently in the dog days of A.I. Summer, and the A.I. “Problem” is being jack-hammered at by simultaneous multiple approaches. Situations like this create circumstances whereby the innovation outruns the ability to asses the long term cultural/social/human/envrionmental impact of these technologies.
Depending on the algorithm you can understand which inputs have high importance to the output but you don’t have a direct relation like you would in say a linear regression problem. With linear regression we could say every time variable X changes by 1 that makes the response change by Y. With some of the more “black box” algorithms we could say, variable X has a strong importance to determining Y but we can directly assign a numerical value to it. It’s not quite as scary as some make it out to be.
"Abstract: One might imagine that AI systems with harmless goals will be harmless. This paper instead shows that intelligent systems will need to be carefully designed to prevent them from behaving in harmful ways. We identify a number of “drives” that will appear in sufficiently advanced AI systems of any design. We call them drives because they are tendencies which will be present unless explicitly counteracted. We start by showing that goal-seeking systems will have drives to model their own operation and to improve themselves. We then show that self-improving systems will be driven to clarify their goals and represent them as economic utility functions. They will also strive for their actions to approximate rational economic behavior. This will lead almost all systems to protect their utility functions from modification and their utility measurement systems from corruption. We also discuss some exceptional systems which will want to modify their utility functions. We next discuss the drive toward self-protection which causes systems try to prevent themselves from being harmed. Finally we examine drives toward the acquisition of resources and toward their efficient utilization. We end with a discussion of how to incorporate these insights in designing intelligent technology which will lead to a positive future for humanity." - Steve Omohundro
Kind of like how some theoretical physicists in the 1930’s, speculating that fission might be a thing that could generate power, wouldn’t have anticipated the H-bomb, ICBMs, nuclear winter or idiot North Korean dictators?
Or Fukushima eventually killing all life in the Pacific Ocean, if not all the oceans.
People who love all these new innovations like AI, gene splicing, genetic manipulation, weather modification–whatever might be next–are living in a fantasy world where the zoo animals never escape and everything always behaves as predicted.
I knew the “good” stuff like this could bring was all a bunch of fairy tales when the killer bees escaped their captors and proceeded to spread all over the western hemisphere.
Everyday I become happier and happier to have an off grid property out of cell range.
Where my thinking starts to shift is we arm ourselves for a societal collapse or for some an over reaching government. But how do we arm ourselves against AI? Ultimately this becomes a discussion of what machines will AI be able to take over and use against us, and how do we defeat those? Far fetched stuff, but so is a total societal collapse some of us are worried about.
But yeah, we’re starting to outsmart ourselves and its going to come back to bite us in a big way.
True; however, this is akin to the “Aircraft Dilemma.” Whereby, we work for a defense contractor designing components, on a per-departmental basis, without any single individual having a complete, and thorough, understanding of the system in totality. I’m beginning to, personally, view machine learning as surpassing current “high intricacy” systems like aircraft, in terms of complexity. That’s just, like, my opinion though, man.
I’m familiar with different I/O “weighting,” which, I would argue, is creating bias in the DNN, and may, ultimately, be a good thing.
I view some of the evolutionary algorithm techniques, as bad ju-ju as well, as if things progress on the current trajectory, we will effectively create, should consciousness be simply a computational/mathematical problem (I argue it is not), a truly ruthless efficiency machine. If you ascribe to an atomistic reductionist worldview, then the possibility of synthetic consciousness is absolutely possible, and the methods we are using to create the supporting scaffolding that will ultimately lead to a truly superintelligent, self-aware system are not in-fitting with a friendly, altruistic entity. Altruism has certain sociological benefits, but I don’t think anyone, at least right now, is sinking billions of dollars of funding into creating Altruistic Intelligence.
Maybe, but it’s also a fantasy to believe that science and technology aren’t going to march forward. Such advances have had occasional negative consequences but the alternative, withholding or limiting technology frontiers, would rob us of a lot of the technology that has moved us vastly forward. Can’t have it both ways.
If someone had said 30 years ago…“we have the capability to develop a worldwide network web by which we can communicate, get information, and buy stuff”, someone else could say “but let’s outlaw it because it will result in worldwide pornography, the death of retail brick-and-mortar, and lots of arguing”. Then all of us here on M4C would never get to hear your opinions.
I think you’re comparing apples to oranges. Nothing about the internet or other “information” type inventions (like smart phones) could result in the death of the human race. Everything I named on my list could.
Science is so busy pushing the envelope of what can be done that no one stops to think if certain things are unwise to tamper with.