There seems to be a permanent state of conflict between religious authority and individual spirituality (and this applies to all religions I know of). Authority values conformity, control, and stability, whereas mystics have invariably been individualists who have challenged the established structures and have encouraged religious experiment, even serendipity.
Invariably the individualists have been isolated, excluded, and disparaged by the authorities, sometimes excommunicated and imprisoned. (Galileo is an obvious example.) Bear in mind that Chasidism was excommunicated twice. And the great Moshe Luzzatto, whose book Mesilat Yesharim (The Path of the Righteous) is a yeshivah "set text", was banned too, by little minds.
Without a doubt, individuals have gone overboard occasionally. Shabetai Tzvi and Jacob Frank were rebels too far. We have had plenty of false messiahs, fake gurus, and even corrupt egomaniacs. But religious life without challenges is boring, stifling. Ultimately it heads to a dead end out of which only mystical revolutions like those of the Essenes, the Kabalists, or the Chasidim can free it. Then they themselves lose the dynamic; they become fossilized and structured, become the establishment, hunt out revolutionaries within, impose conformity of thought and behavior, and the cycle repeats again.
To me, the role of the Biblical priesthood typifies establishment. It was hereditary, privileged, and protected from reality. Historically, in both commonwealths, it lost its sense of mission and spiritual leadership. The first priest was Aharon, a good man, slow to anger and a peacemaker. But when it came to taking the lead, he couldn't. The worshippers of the golden calf manipulated him and he seemed to have given way far too easily. He offered no resistance. He sought political compromise. After all, he would have argued, he represented the whole of the community. He had responsibility. He was a diplomat to Moshe’s dynamic leadership. It was not for him be innovative or take risks.
One example of leadership, Chur, was left jointly in charge with Aharon, according to the Midrash, tried to stop those who wanted a golden calf, but was killed. Aharon was the cautious papal representative of the Hochhuth play, the representative rabbi who fears taking a stand in case he alienates, or the president of a commercial company, bank, or institution who has to convey confidence and stability, and show a steady hand on the tiller.
The prophet, on the other hand, had an entirely different role. The prophet was not appointed; he rose by dint of his own personality. Incidentally and significantly, there are women prophets in the Bible but no women priests.
The prophet was God's voice on earth. The message was the essence, not the presentation. There were, of course, great prophetic poets too. But more often the prophet was a mystic feeling the presence of God, driven by powers beyond his control, caught between suffering and ecstasy, the wild charismatic living in caves, on the run from authority, preaching challenging messages and pointing to new directions.
Moshe was unique because he alone combined both elements. He did not fit into any society completely. He was something of an outcast amongst Jews, resented because he could do things they could not, because he was fearless. The Egyptians regarded him as a threat, one of theirs who had been educated by them and given all the privileges, then used all that knowledge and turned against them. He was a stranger amongst the Midianites. He was also a spiritual visionary who withdrew into the desert to meditate and find his God and disappeared up the mountain in a haze of Divine encounter. Very reluctantly, and only under duress, did he take on a leadership role. Even then he continually begged God to relieve him of the burden and to let someone else take over, even to kill him and put him out of his misery.
Moshe was indeed a prophet who was forcibly harnessed into a leadership role, at which he turned out to be very good. He was a unique combination, which s of course why he remains the greatest of all spiritual figures. Ultimately he came to recognize that a people needed a constitution, a clearly defined framework, and a team of specialists to deal with the complex needs of a whole society. He took risks with his charges and put his job on the line. He even challenged God and yet retained his close, intense, and profound relationship with Him. His persona defines the characteristics of Judaism that differentiates us.
He and Aharon were so different and yet they represented these two contradictory paradigms of Jewish leadership. Aharon was the safe one who could not stop the rebels. Moshe was the radical who could.
The fact is that in most areas of human life we need both. Relationships need responsibility and obligation, but without love and passion they become dry and uninspiring. Businesses need complicity officers, accountants, and lawyers to make sure everything is done according to the book and unwise risks are not taken. But without visionaries, brilliant and sometimes crazy innovators, a business soon loses its edge, becomes petrified and fails.
The same is true of religion. Authority provides continuity, safeguards, and comfort. But it cannot see the exception. It cannot deal with the individual or the rebel. Too often authority loses sight of the essential message and ironically misleads the mass down the wrong alleyways. Without creativity and without challenge all authorities retreat behind bureaucracy and safety and they end up driving too many marginal people away.
Prophets can become loose canons and, with the exception of Moshe (and Eliyahu on Mount Carmel), prophets in Judaism have no authority to intervene in law. Yet without creativity and passion, religion atrophies and bores instead of inspiring. Woe to the generation whose prophets are silenced.