The letter's writers assume too much about consciousness and its nature. How do they know that their models are even accurate and that they're not just mistaking the map for the territory? How do they know that AI consciousness would even resemble human consciousness? For all we know, it already exists and we just haven't recognized it yet.
One must also remember that our brains are prone to seeing signs of consciousness and agency where none exist, and that things like the Turing Test say more about how willing we are to be fooled by the illusion of sentience than about actual machine sentience. If they cannot even define consciousness, they're in no position to lecture others about it.
For now, I use the simple metric of initiative: ChatGPT might be good at talking about many subjects, but have you ever seen it start up a conversation without a human to prompt it first?
John Michael Godier makes a convincing argument that we, modern humans, are an example of AI. Albeit biological.
To oversimplify, machine AI is just us passing on our knowledge and seeing how the machines run with it. Similarly, precursor humans had technologies like fire and tool use, and probably at least some form of language. The new kids on the block learned from that and adopted it. Then we were able to outcompete the people who instructed us; and now they are extinct. That may well happen again.
But AI civilisations are probably the only ones that do have any future. The earth, and even the solar system, can only support life for a finite time. Hanging around here is an evolutionary dead end. But it is hard to see how a biological species can ever colonise the galaxy; we just don't have the endurance. But a machine civilisation won't be bothered about taking a few thousand years to get to the next star. And once established they can travel at the speed of light just by transmitting the code that makes them alive. Any housing will do as a vessel for their consciousness.
I do believe that we are probably one of the first, if not the first, intelligences in the universe; but the future belongs to the machines.
If that was the case, one would expect the known universe to have long since been overtaken by machine civilizations by now, with obvious signs of their activity tipping us off to their existence. Plus, even those machines are still going to run down and fall apart eventually (and code isn't inviolate either, errors are going to accumulate and they'll reach a level that can't just be fixed) so those civilizations will be no more immortal than ours. Entropy will get its due, one way or another, and the only way to avoid that is to somehow completely dissociate the mind from any kind of physical medium- something that is firmly in the realm of fantasy.
More importantly, biological life doesn't need constant supervision by itself or outside forces to adapt and evolve. Natural selection does that for it, and thus far the artificial equivalents simply can't compete. It might be able to model many different things, but how are you supposed to model events that are by their very nature impossible to predict?
Yeah, the fact we don't see any obvious technosignatures is, to me, more evidence that we are likely a very early emergent intelligence. And just statistically bearing in mind the potential period of habitability of the universe as a whole and how early we popped up.
Entropy gets us all in the end, but I don't see why a machine civilisation couldn't be self sustaining. Maybe the best form of life will be a von Neumann type machine. And a machine intelligence can just stick its consciousness into any suitable receptacle. That, to me, seems a much more efficient survival strategy. As humans of course we can pass on knowledge; but individual humans only have a limited time and capacity to absorb that and utilise it. What might Newton or Einstein have come up with if they had a thousand years of uninterrupted thinking time?
Machine intelligence isn't magical, and that assumes that any such conversion wouldn't have problems of its own. Our consciousness at least needs a body to work properly and actually understand the world around us- a disembodied intelligence with no stake in physical reality may very well withdraw from the universe completely in favor of solving increasingly complex math problems until it shuts down.
Why would these machines even feel the need to reproduce or expand in the first place, anyway? That again sounds like it's projecting our own motivations and drives onto something that may not have them and likely would not benefit from them- they arose through a very specific set of processes which by definition any AI consciousness will be exempt from. Just because our brains and minds have specific drives doesn't mean that they're the only kind of minds that can exist. It's better to assume that since they aren't human, they're unlikely to have any similarities with our own minds. Again, we may not recognize them as conscious...and they might not recognize us as conscious either.
>As humans of course we can pass on knowledge; but individual humans only have a limited time and capacity to absorb that and utilise it. What might Newton or Einstein have come up with if they had a thousand years of uninterrupted thinking time?
Ask instead if it might be possible that some things just can't be known, or if intelligence itself has limits.
I think there are lots of things that can't be known, by humans at least. We just don't have the brain wiring for it. And of course, some things might just be unknowable, even by the most powerful intelligence that could exist. There just wouldn't be the data to come up with an answer; like why is here something rather than nothing.
The disembodied intelligence point is interesting. Would a total lack of sensory input just allow one to focus in a zen sort of way? What might someone come up with after an eon of quiet contemplation? Or would it just drive you mad?
But an AI need not be disembodied. Even if it was housed in one location, it could still absorb remote sensory infraction by telepresence. After all, we only experience the world through electrical signals entering our brain. And whilst I don't believe we are just brains in jars or some other cartesian speculation, if you could simulate nerve signals in an appropriate way the brain would experience something indistinguishable from reality. So I don't see any material difference between organic or mechanical life/consciusness. Carbon chauvinism and all that.
It's all very fascinating to consider though. Be ironic if we actually were AIs and we just thought we were having this conversion online.
It's not carbon chauvinism when you remember that those consciousnesses will be shaped by their physical conditions. Without any of the Darwinian impulses towards reproduction or even self-preservation, why would they even bother concerning themselves with spreading anywhere at all? What would be the point of those senses when nothing about the physical world is likely to affect their "bodies" anyway?
They'd simply have no reason to do any of the things we might do. Regarding the simulation concept, that's mistaking the map for the territory again. Our simulations might resemble the real thing on the surface, but on the inside they could very well be completely unrelated to the actual processes that allow for consciousness to exist.
Yeah, that's an interesting thought. Would there be any evolutionary pressure on an AI? I suspect yes. Evolution is a good way of improving efficiency. It has been used even in jet engine design. So I don't think an AI would necessarily stagnate. And of course, their initial programming might have an influence. If the goal of their designers is that they should seek out ways of self improvement. I suspect any AI that arose from human civilisation would tend towards the 'grabby'. They'd get that from us.
The letter's writers assume too much about consciousness and its nature. How do they know that their models are even accurate and that they're not just mistaking the map for the territory? How do they know that AI consciousness would even resemble human consciousness? For all we know, it already exists and we just haven't recognized it yet.
One must also remember that our brains are prone to seeing signs of consciousness and agency where none exist, and that things like the Turing Test say more about how willing we are to be fooled by the illusion of sentience than about actual machine sentience. If they cannot even define consciousness, they're in no position to lecture others about it.
For now, I use the simple metric of initiative: ChatGPT might be good at talking about many subjects, but have you ever seen it start up a conversation without a human to prompt it first?
John Michael Godier makes a convincing argument that we, modern humans, are an example of AI. Albeit biological.
To oversimplify, machine AI is just us passing on our knowledge and seeing how the machines run with it. Similarly, precursor humans had technologies like fire and tool use, and probably at least some form of language. The new kids on the block learned from that and adopted it. Then we were able to outcompete the people who instructed us; and now they are extinct. That may well happen again.
But AI civilisations are probably the only ones that do have any future. The earth, and even the solar system, can only support life for a finite time. Hanging around here is an evolutionary dead end. But it is hard to see how a biological species can ever colonise the galaxy; we just don't have the endurance. But a machine civilisation won't be bothered about taking a few thousand years to get to the next star. And once established they can travel at the speed of light just by transmitting the code that makes them alive. Any housing will do as a vessel for their consciousness.
I do believe that we are probably one of the first, if not the first, intelligences in the universe; but the future belongs to the machines.
If that was the case, one would expect the known universe to have long since been overtaken by machine civilizations by now, with obvious signs of their activity tipping us off to their existence. Plus, even those machines are still going to run down and fall apart eventually (and code isn't inviolate either, errors are going to accumulate and they'll reach a level that can't just be fixed) so those civilizations will be no more immortal than ours. Entropy will get its due, one way or another, and the only way to avoid that is to somehow completely dissociate the mind from any kind of physical medium- something that is firmly in the realm of fantasy.
More importantly, biological life doesn't need constant supervision by itself or outside forces to adapt and evolve. Natural selection does that for it, and thus far the artificial equivalents simply can't compete. It might be able to model many different things, but how are you supposed to model events that are by their very nature impossible to predict?
Yeah, the fact we don't see any obvious technosignatures is, to me, more evidence that we are likely a very early emergent intelligence. And just statistically bearing in mind the potential period of habitability of the universe as a whole and how early we popped up.
Entropy gets us all in the end, but I don't see why a machine civilisation couldn't be self sustaining. Maybe the best form of life will be a von Neumann type machine. And a machine intelligence can just stick its consciousness into any suitable receptacle. That, to me, seems a much more efficient survival strategy. As humans of course we can pass on knowledge; but individual humans only have a limited time and capacity to absorb that and utilise it. What might Newton or Einstein have come up with if they had a thousand years of uninterrupted thinking time?
Machine intelligence isn't magical, and that assumes that any such conversion wouldn't have problems of its own. Our consciousness at least needs a body to work properly and actually understand the world around us- a disembodied intelligence with no stake in physical reality may very well withdraw from the universe completely in favor of solving increasingly complex math problems until it shuts down.
Why would these machines even feel the need to reproduce or expand in the first place, anyway? That again sounds like it's projecting our own motivations and drives onto something that may not have them and likely would not benefit from them- they arose through a very specific set of processes which by definition any AI consciousness will be exempt from. Just because our brains and minds have specific drives doesn't mean that they're the only kind of minds that can exist. It's better to assume that since they aren't human, they're unlikely to have any similarities with our own minds. Again, we may not recognize them as conscious...and they might not recognize us as conscious either.
>As humans of course we can pass on knowledge; but individual humans only have a limited time and capacity to absorb that and utilise it. What might Newton or Einstein have come up with if they had a thousand years of uninterrupted thinking time?
Ask instead if it might be possible that some things just can't be known, or if intelligence itself has limits.
I think there are lots of things that can't be known, by humans at least. We just don't have the brain wiring for it. And of course, some things might just be unknowable, even by the most powerful intelligence that could exist. There just wouldn't be the data to come up with an answer; like why is here something rather than nothing.
The disembodied intelligence point is interesting. Would a total lack of sensory input just allow one to focus in a zen sort of way? What might someone come up with after an eon of quiet contemplation? Or would it just drive you mad?
But an AI need not be disembodied. Even if it was housed in one location, it could still absorb remote sensory infraction by telepresence. After all, we only experience the world through electrical signals entering our brain. And whilst I don't believe we are just brains in jars or some other cartesian speculation, if you could simulate nerve signals in an appropriate way the brain would experience something indistinguishable from reality. So I don't see any material difference between organic or mechanical life/consciusness. Carbon chauvinism and all that.
It's all very fascinating to consider though. Be ironic if we actually were AIs and we just thought we were having this conversion online.
It's not carbon chauvinism when you remember that those consciousnesses will be shaped by their physical conditions. Without any of the Darwinian impulses towards reproduction or even self-preservation, why would they even bother concerning themselves with spreading anywhere at all? What would be the point of those senses when nothing about the physical world is likely to affect their "bodies" anyway?
They'd simply have no reason to do any of the things we might do. Regarding the simulation concept, that's mistaking the map for the territory again. Our simulations might resemble the real thing on the surface, but on the inside they could very well be completely unrelated to the actual processes that allow for consciousness to exist.
Yeah, that's an interesting thought. Would there be any evolutionary pressure on an AI? I suspect yes. Evolution is a good way of improving efficiency. It has been used even in jet engine design. So I don't think an AI would necessarily stagnate. And of course, their initial programming might have an influence. If the goal of their designers is that they should seek out ways of self improvement. I suspect any AI that arose from human civilisation would tend towards the 'grabby'. They'd get that from us.