Here is another question. Is it or will it be eventually possible to know what one is thinking, i.e., to read his mind, by reading his brain?
For example, the contents of thoughts, such as the propositions that can be expressed by thoughts, are infinitely varied: 2 + 2 = 4, 3 + 3 = 6, …, even uncountable, but the number of possible brain states is finite. Like, read my brain to know which number in [0, infinity) I am thinking of. How can you do that?
One may have better luck trying to divine the will of the gods by studying the entrails of a chicken.
Is he deluding himself? Are men and machines equally unfree in this sense?
I do not believe that free will exists, though I believe that we should behave as if it does.
Can a machine do what Smith claims is the human exclusive privilege?
That's what I've been claiming about the future.
it must be possible to discover utils
I say that we can measure dopamine and serotonin responses, but this hardly tells us which is more important, and whether they are important at all to the concept of utility. Say that eating an apple releases 10 units of dopamine and 10 units of serotonin. How does this compare to something which releases 5 units of dopamine and 15 units of serotonin? How can we prove that one is more important than the other? We can't.
Is it or will it be eventually possible to know what one is thinking, i.e., to read his mind, by reading his brain?
Is it a possibility.
the contents of thoughts, such as the propositions that can be expressed by thoughts, are infinitely varied
In general, yes. The potential is infinite.
the number of possible brain states is finite
Roughly 2^(100 billion) for any given snapshot of time. Yet there is a problem in this analysis:
1) You neglect that new neurons can be created and the system is not set in stone. You may create new neurons as your brain adapts to its environment. In fact, this is what happens (at the very least in childhood, but new evidence suggests that it also happens later on in life).
2) Just because you have a set number of neurons doesn't mean that they can do a limited amount of calculations. For example, neurons can be "re-purposed." Their reaction to stimulus can change over the course of life, so a neuron that once accounted for function X can change to function Y.
And analogous thing actually happens with computers. Just because a computer has only 1 GB of RAM it doesn't mean that computers are limited to one use of Photoshop and then you must buy a new computer because the RAM is full. No - memory gets reused and re-purposed. In a finite amount of memory you can express any piece of information which fits into the memory. But you can then erase it and fit something else in. In the same way, the human brain has a limited potential to store info (if we assume temporarily that no new neurons are created), but may think of as many thoughts (which by themselves fit in the brain, whatever that means) as it wants as long as they are not in there at the same time. And, as I explained before, new neurons do get created.
3) Ultimately, since the universe itself is limited in the number of states it may assume, then yes, at the extreme the brain is limited. It may not, for example, turn into a potato at any given moment. But I don't think that you want to take it to such an extreme.
Argument from ignorance.
So, "importance" is only gauged by an individual subjective experience. So, there are some aspects of experience, such as whether an apple is preferred to an orange, that are not reducible to "physics."
My point is: your argument implies that for every thought, there is a brain state uniquely representing that thought. But there is infinity of possible thoughts, and infinity is not "an inconceivably large finite number like 2^(100 billion)"; it's infinity. So, how can there be a one-to-one correspondence between every thought and a brain state?
You argument amounts to saying that a person never has 2^(100 billion) thoughts at the same time; well, that's true, right now I may be having only 2 or 3 in the back of my mind. So, then, the same brain state will be able to represent numerous, in fact, also an infinity of different thoughts, just as 1GB of RAM can house any software program. In which case it is impossible to read the mind by reading the brain.
P.S. IE 9 doesn't seem to work with this forum too well.
Is the following scenario possible?
I weigh eating an apple with eating an orange and decide to eat the apple. You scan my brain with your futuristic technology and tell me that according to super-scientific "physics," I "really" chose the orange! Well, orange it is then!
Not freedom of the will; freedom of the mind to assume, at the beginning of any chain of reasoning, anything it pleases.
The intellect’s key ability is to learn, to derive outputs from inputs. The inputs for machines are finite in number. Their “premises,” based on which they “reason,” are pre-programmed into them. The inputs for humans are infinite; we can assume anything we like.
(The mind is free to assume X or to assume ¬X. But it is not free to escape the consequences of extending belief to either X or ¬X and acting on that commitment.)
How about intuition, defined by Merriam-Webster as “the power or faculty of attaining to direct knowledge or cognition without evident rational thought and inference.” Things like that do happen, if we have a word for it!
Not every way of obtaining new knowledge can be programmed. Like, a guy had a near-death experience, and he met someone on the other side and says he knew right away without being told that the guy was freaking Moses! Simulate that!
Found this randomly on StumbleUpon 1 minute after reading your last post and found it relevant:
http://www.stumbleupon.com/su/25wpM3/faculty.washington.edu/eloftus/Articles/sciam.htm
Dude, the NDE is just an example. You are free to "assume" it to be whatever you please.
The fact that sometimes memories deceive you does not mean that all memories are false or, for that matter, that all memories regarding near-death experiences are false. I mean, look, NDEs are one of the most heavily researched subjective phenomena of them all. But who cares. If you are willing to go so far as to say that people never intuit anything, fine. But you will probably find lots of people disagreeing with you. There are more things in heaven and earth, etc.
What about appreciation and creation of beauty? There are a lot of people who are very good at knowing what is beautiful, and I'm pretty sure that very few of them are gay. Do you think that what is beautiful is a function? You plug in a few inputs, crunch some numbers, and out comes a great symphony. Oh, what the hell. Maybe it is a function. But until you come up with one, be more humble.
Here's what you might do. Talk to these guys. They've apparently taught a computer which faces the masses found attractive, competent, trustworthy, etc. The computer then evaluated new faces based on the initial knowledge. Here's the thing: if the algorithm can generate brand-new hitherto unknown beautiful faces, then I'll admit that there is a science to beauty. Or rather, of course, there is a science to beauty: e.g., symmetry, proportion (the Golden Ratio, perhaps), harmony, organic unity, etc. But is there a human element to it, as well, that's not rule-bound?
I am not saying any computers today can do a half-decent job at it. I am saying that in the future, it's likely we'll have said computers. I estimate before 150 years from now we will have something very close.