First you understand what Linga is. Linga is a symbol.
Why are genitals also called Linga? That is because it is the symbol through which one can know whether a baby is a male or female. When a baby is born you see only one area to identify the gender. So it is an identification symbol.
Lord Shiva is manifest all over the Universe, so how does one identify or relate to Him? This is why in the ancient days, the wise sages would place a Pind or a round or oval stone piece and identify that with Lord Shiva.
Thus, the Linga and the Yoni (here referring to the base of the Shiva Linga on which the stone or Linga rests) was kept because by this the male and female were identified.
Now, how will you describe the Lord of the Universe who is formless?
In the ancient age, there was no form of Lord Shiva holding a Trishul (Trident), or anything like that. In ancient days, there was only a Pind (a stone) that was kept and then by chanting mantras, the Chaitanya Shakti (energy pervading all of the Divine Consciousness) was awakened and manifested in the Pind. So this is how it was worshiped.
It was only much later that idols were created.
Then what happened? On the Pind people started painting eyes, a face, etc. The custom of painting a face on the Pind was not there earlier.
In the same way even Lord Vishnu was worshiped by performing pooja of his feet (referred to as Paada in Sanskrit).
If you go to Gaya (second largest city of Bihar, India), people there will say, ‘Vishnu Paada’, only the feet of Lord Vishnu are worshiped.
It is said that more important than the idol, is the Yantra (the diagrammatic representation of the deity in the form of symbols).
Every deity has a Yantra and a Mantra dedicated to him or her, and the procedure or rituals of worship of the deity is known as Tantra.
The idol does not get its spiritual strength until its Yantra is installed. And the Yantra does not have any power until it is empowered by the chanting of the Mantra.
That is why in every temple a Yantra is installed first, and then on top of that the idol is established. This is done to invoke a deep sense of devotion in those who visits the temple.
Even earlier, in the Sanatan Dharma (referring to an earlier name for Hinduism) there were no idols or idol worship as such, but only Havans (ritual in which making offerings into a consecrated fire is the primary action) were performed and Lord Shiva’s pind would be placed to establish his presence. That was it, nothing else would be done.
It was only later that the custom of installing idols came up.
Lord Ganesh was seen in a supari (a small betel nut), Lord Shiva was seen in a Pind and Devi (Mother Divine) was worshiped in a Kalash (a holy pot filled with water) with a coconut on top of it. This was the procedure as per the Sanatan Dharma.
Even today there is no significance of worshiping an idol without the Kalash. The water from the Kalash is poured over the idol. This is the custom.
Now, why did the practice of having idols begin?
This was because by seeing the idol a feeling of devotion would arise from within.
The other reason is that when Buddhists and Jains made their temples they would place such beautiful idols at the altar. So then those who followed Sanatan Dharma felt that they should also do something like this. So they also followed the same and began to establish different idols of Lord Vishnu, Lord Rama and Lord Krishna.
You will not find any mention of a practice of installing idols for worship in the Bhagavad Gita or the Ramayana.
Only the Shiva Linga (Shankar Linga) was installed. That is why only the Shiva Linga was there in the ancient period, which was worshiped by Lord Krishna, Lord Rama and everyone else.
Do you know, the stone in the Holy Kaaba (referring to the holy shrine in Mecca, Saudi Arabia) is also Lord Shiva?
There is a shloka (verse) in the Bhavishya Puran, about the three footsteps of Lord Vishnu (in his fifth incarnation as Vamana, a dwarf Brahmin). The first step of the Lord was in Gaya and the second step was in Mecca.
Much before the coming of Prophet Mohammed, people used to go on a pilgrimage to Mecca. That is why the pilgrims go and kiss the stone there and they circumambulate it seven times. It is done the same way in Shiva temples where people wear unstitched white clothes and go and worship the stone.
This is exactly like customs that have been practiced since the ancient times. So there are remarkable similarities in how worship is done in Gaya, and in Mecca: the same kind of stone is installed and worshiped, circumambulations are performed in the same way, and same kind of unstitched clothes are worn.
All these are linked and connected somewhere or the other.