The AI Revolution: Superintelligence, Immortality, Extinction

[quadz]

Part 1: The AI Revolution: The Road to Superintelligence

Part 2: The AI Revolution: Our Immortality or Extinction









For me, an interesting article overall.

(I suspect some physical limits relating to information processing, and impediments like the General Halting Problem (and perhaps the existence of NP-complete problems), may place some constraints on the rate of takeoff from AGI to ASI... And I also suspect the 'quality' of intelligence may not be able to be ratcheted up to the tremendous multipliers postulated in the article; -- I don't know what sort of law might govern that, but I'd be surprised if there didn't work out to be some sort of diminishing returns, where even a superintelligence accustomed to designing the next smarter version of itself might begin finding it increasingly difficult to make incremental gains. Nevertheless, I've little doubt there's room to be many times smarter than humans before hitting those sorts of limits, if they exist.)

[[BTF]Defiant!]

Biological intelligence creating an independent intelligence which can self-improve seems inevitable.  For when, it is tough to estimate the order of complexity of the AGI or ASI problem. 

As our intelligence improves our wisdom and willingness to change remains slow.  Expect human kind will splinter with some wanting to retain pure biology and others not.  Independent of that, an ASI would be "bored" with us overall -- similar to how we would be bored communicating with a tree.

One nit in the article.. the quote "There are no hard problems, only problems that are hard to a certain level of intelligence." is provably wrong.  There are problems which are uncountably infinite thus are literally unsolvable with infinite time and resources.  (See Touring Machine and Cantor's diagonalization argument).