become so intelligent, so quickly, that nothing is able to keep up to control it or compete with it. This Bostrom calls the 3-point problem: the first superintelligence is likely to have a strategic advantage – i.e. However, and whatever, a superintelligence is built, the book highlights that it will be very difficult (I’d almost say he is hinting that it’s impossible) to design one that will behave in the way we want it to – even if we could agree, as society, what it should do.

Tool – not designed to exhibit goal-directed behaviour.Sovereign – open-ended autonomous operation that behaves in our best interests without explicit instructions.Genie – command-executing system (one job at a time).There are also plenty of useful tables to summarise his thinking, such as the types of AI that could be developed: I was happy to skip a few paragraphs here and there and still come out massively more informed than when I started. I confess that some of the deeper discussions that involved philosophical arguments did go over my head – but I don’t think that spoilt the book for me. Bostrom isn’t predicting when it might happen and therefore admits it’s impossible to know what technology will be involved, but the risks around producing something thousands of times more powerful than the brightest of human minds will be the same whatever technology is used. No technical knowledge of how AIs (non-superintelligent ones) are being developed now is required by the reader. how might we preclude, or at least, reduce these risks?īuy Superintelligence on Amazon (forewarned is forearmed!)* what are the dangers in building a superintelligence?ģ. how might a superintelligent general artificial-intelligence be created?Ģ. The book’s subtitle neatly sums up the content of this book – “paths, dangers, strategies” – which in summary means:ġ.

Nick bostrom books