Sometimes, it can be difficult to tell, even for those who had the experience. Paul, (speaking of his own experience), said,

2 I knew a man in Christ above fourteen years ago, (whether in the body, I cannot tell; or whether out of the body, I cannot tell: God knoweth;) such an one caught up to the third heaven.

3 And I knew such a man, (whether in the body, or out of the body, I cannot tell: God knoweth;)

4 How that he was caught up into paradise, and heard unspeakable words, which it is not lawful for a man to utter.

I myself have had visions that were full body encompassing and I couldn't tell whether they were in the flesh or in the spirit. But, after considering the facts I determined that these experiences could not have been physical. For example, in one vision, I was walking next to my missionary companion in a field. I had a vision in which I moved great distances at rapid speeds. During the vision, I saw my own body walking next to my companion. I had no awareness of my physical body. I couldn't see what it saw or hear what it heard. I had a body I was in that was flying around doing and seeing things. I was fully aware of my body flying around. I saw what it saw, heard what it heard and only had awareness of my experience in that body. When the vision ended, I found myself back in my physical body, walking next to my companion in the field. The entire vision happened in less time than it takes to complete a step. My companion didn't see me leave his side. And so, based on the evidence I must conclude that it was not in the flesh. But from the experience itself, without the clues and facts outside the experience, I would have thought I was in the flesh. I have had several experiences like that. So, I understand what Paul was saying.

So, when the 3 witnesses of the Gold Plates say that they both saw and handled the plates, I understand that it could be physical or spiritual. You have to look at other clues because it can be difficult to tell, even for the person having the experience.

Here are a few quotes from the witnesses that indicate that they themselves discovered their experience was spiritual, not physical.

The printer of the Book of Mormon, John H. Gilbert recounted asking Martin Harris,

"Martin, did you see those plates with your naked eyes?" Martin looked down for an instant, raised his eyes up, and said, "No, I saw them with a spiritual eye."

Another account records:

"Did you see the plates, and the engraving on them with your bodily eyes?"

Harris replied, "Yes, I saw them with my eyes,—they were shown unto me by the power of God and not of man."

"But did you see them with your natural,—your bodily eyes, just as you see this pencil-case in my hand?"

Harris replied, "Why I did not see them as I do that pencil-case, yet I saw them with the eye of faith; I saw them just as distinctly as I see any thing around me."

David Whitmer wrote that it was a vision in the spirit and gave similar examples from the scriptures to compare with:

"In regards to my testimony to the visitation of the angel, who declared to us three witnesses that the Book of Mormon is true, I have this to say: Of course we were in the spirit when we had the view, for no man can behold the face of an angel, except in a spiritual view, but we were in the body also, and everything was as natural to us, as it is at any time. Martin Harris, you say, called it 'being in vision.' We read in the Scriptures, Cornelius saw, in a vision, an angel of God. Daniel saw an angel in a vision; also in other places it states they saw an angel in the spirit. A bright light enveloped us where we were, that filled at noon day, and there in a vision, or in the spirit, we saw and heard just as it is stated in my testimony in the Book of Mormon. I am now passed eighty-two years old, and I have a brother, J. J. Snyder, to do my writing for me, at my dictation." [Signed] David Whitmer.

He also said,

"In June, 1829, the Lord called Oliver Cowdery, Martin Harris, and myself as the three witnesses, to behold the vision of the Angel..."

Some people falsely assume that if the experience is not physical, then it was not valid. We can see examples in the scriptures where multiple people are present physically, but because the event was spiritual and seen and heard only with the spiritual eyes and ears, it was not witnessed by everyone physically present.

When Elisha and his servant were surrounded by a physical Syrian army, the servant was afraid and asked Elisha what they should do. The servant couldn't see the spiritual army protecting Elisha until Elisha prayed for his servant's eye to be opened. And even then, the Syrians still could not see the spiritual army.

16 And he answered, Fear not: for they that be with us are more than they that be with them.

17 And Elisha prayed, and said, Lord, I pray thee, open his eyes, that he may see. And the Lord opened the eyes of the young man; and he saw: and, behold, the mountain was full of horses and chariots of fire round about Elisha.

Stephen saw a vision of the heavens opened while an angry mob was surrounding him. They did not see what he saw. They hauled him out of the city and stoned him to death immediately after he said,

56 And said, Behold, I see the heavens opened, and the Son of man standing on the right hand of God.

7 And I Daniel alone saw the vision: for the men that were with me saw not the vision...

Balaam rode a donkey which could see the angel when he could not. Balaam was angry with the ass for refusing to walk forward and Balaam was beating the donkey with his staff. The donkey spoke to Balaam, (clearly not physically, but spiritually) and told Balaam that he wasn't walking to protect him. Then, Balaam's eyes were opened to see what was not physical.

31 Then the Lord opened the eyes of Balaam, and he saw the angel of the Lord standing in the way, and his sword drawn in his hand: and he bowed down his head, and fell flat on his face.

The spiritual is real, and in the spirit we can see, hear, touch, taste, and smell just as we can in the physical. So, it is difficult to explain when someone asks, and those unfamiliar with the experience misunderstand it as physical.When my dad took his family and siblings to the first Las Vegas Temple Open house, his sister Amy saw a line of people touring up high around the edges of the celestial room on a walkway built against the wall. But she couldn’t see the stairs to get up there or how to enter that line. After the tour, she asked other’s including my dad about it. My dad saw it, but few others saw it. It didn’t physically exist. And most of those who went on the tour didn't see it. Those who did see it assumed it was physical because they didn’t realize they were seeing it with their spiritual eyes.I could share many examples from my personal life as well, however, the point I wanted to make is that spiritual truth does not require physical reality. And it can be difficult to tell when an experience is spiritual rather than physical. This can be true even when you are the person who experienced it.