Allahu akbar! Are you suggesting the Prophet's ﷺ closest companions didn't understand the Qur'an? Are you questioning their faith after Allah has revealed that He is pleased with them as they are pleased with Him? The companions are the ones who preserved the knowledge of the religion, they're the reason we even have scholars today.
The Prophet ﷺ said:
"I was sent to join the ties of kinship, to break the idols, and to proclaim the oneness of Allah so that nothing be associated with Him." (Muslim 832)
And 'Ali (Allah be pleased with him) said to Abu'l-Hayyaj al-Asadi:
"Should I not send you on the same mission as Allah's Messenger ﷺ sent me? Do not leave an image without obliterating it, or a high grave without levelling it." (Muslim 969)
Breaking the idols is an act of piety not limited to the prophets at all. The companions used to break the idols wherever they came upon them, from Persia to Africa. And in this they did not misunderstand anything; wasn't it the Messenger ﷺ himself who sent Jarir to destroy the false deity at Dhul Khalasa, so he went out and destroyed it? (Bukhari 3823)
As for what the scholars said. Here's Ibn Al-Qayyim (Allah's mercy on him):
"It is not allowed to leave places of Shirk and tawagheet (false gods) even for one day after having the ability to destroy them. These are signs of Kufr and Shirk, and this is the peak of evil. So it is never allowed to acknowledge them if you have the power. And this is the ruling for the tombstone which is on the grave which is taken as taghoot, worshipped besides Allah. And it is also the ruling for stones which people go to, to seek barakaah, kissing them and making offerings to them. It is not allowed to leave anything of them on the earth when you have the ability to remove them. A lot of these are at the same level as Lat, Uzza, Manat or are even considered more than these to those people who worship them. Wallahu musta'an." (Zad Al-Ma'ad 3/506)