The scriptures about the ancient kingdoms of Israel are illustrative for answering this question. They were originally one kingdom through the reigns of Kings David and Solomon, but after Solomon’s reign, they divided into the Northern Kingdom of Israel and the Southern Kingdom of Judah.
The Kingdom of Judah, who governed the temple of Solomon and had official priests and a line of authority, were idolatrous by worshiping their leaders. Jeremiah proclaimed that Jehovah himself pointed this out and said that when they prophesied, pretending it was from Him, it was more like it was from Baal instead:
The priests said not, Where is the Lord? and they that handle the law knew me not: the pastors also transgressed against me, and the prophets prophesied by Baal, and walked after things that do not profit. (Jeremiah 2:8).
The Kingdom of Israel rebelled against Solomon’s royal line and worshiped in high places and in groves. They were idolatrous by worshiping the hosts of heaven instead of God alone:
And they left all the commandments of the Lord their God, and made them molten images, even two calves, and made a grove, and worshipped all the host of heaven, and served Baal. And they caused their sons and their daughters to pass through the fire, and used divination and enchantments, and sold themselves to do evil in the sight of the Lord, to provoke him to anger. Therefore the Lord was very angry with Israel, and removed them out of his sight: there was none left but the tribe of Judah only. (2 Kings 17:16-18).
Eventually, the Kingdom of Judah gave up all pretenses, and after an insincere and feigned return to Jehovah worship during King Josiah’s reign, they became involved in full blown idolatry by worshiping the hosts of heaven as well:
¶The Lord said also unto me in the days of Josiah the king, Hast thou seen that which backsliding Israel hath done? she is gone up upon every high mountain and under every green tree, and there hath played the harlot. And I said after she had done all these things, Turn thou unto me. But she returned not. And her treacherous sister Judah saw it. And I saw, when for all the causes whereby backsliding Israel committed adultery I had put her away, and given her a bill of divorce; yet her treacherous sister Judah feared not, but went and played the harlot also. And it came to pass through the lightness of her whoredom, that she defiled the land, and committed adultery with stones and with stocks. And yet for all this her treacherous sister Judah hath not turned unto me with her whole heart, but feignedly, saith the Lord. And the Lord said unto me, The backsliding Israel hath justified herself more than treacherous Judah. (Jeremiah 3:6-11).
In both cases, God rejected them when they got it wrong, including the Jews when their temple worship missed the mark (you only have to skim Isaiah, Jeremiah, or Ezekiel and other prophets to catch that theme). But, He also supported them both when they got it right, even though the Deuteronomists and King Josiah’s reformers in the Southern Kingdom of Judah modified the sacred record to reflect a condemnation of all worship in high places outside of the central temple in Jerusalem (see this scripture search for “high places” to notice the parenthetical insertions with the pattern: “But the high places were not taken away” throughout the book of the 2 Kings, for starters). The hypocrisy of Judah condemning Israel for worshiping outside of the temple manifested itself by Judah turning fully to idolatry themselves. They were just as bad, and eventually they stopped covering it up with the cloak of the official religion laid down by Moses.
The fact that God called prophets in the Northern Kingdom of Israel, most notably Elijah with his worship at an altar out in the open at Mount Carmel (see 1 Kings 18), proves that God recognizes worship outside of the temples built by religious establishments and the priestly class. The Kingdom of Israel had no temple, so all of its prophets were unofficial in terms of “authorized” temple worship.
Joseph Smith explained how both forms of worship–in a temple and on the mountain top–could be appropriate:
I preached in the grove on the keys of the Kingdom, Charity &c The keys are certain signs and words by which false spirits and personages may be detected from true, which cannot be revealed to the Elders till the Temple is completed–The rich can only get them in the Temple–the poor may get them on the Mountain top as did Moses. The rich cannot be saved without Charity, giving to feed the poor when and how God requires as well as building. There are signs in heaven, earth, and hell, the Elders must know them all to be endowed with power, to finish their work and prevent imposition. The devil knows many signs but does not know the sign of the Son of Man, or Jesus. No one can truly say he knows God until he has handled something, and this can only be in the Holiest of Holies. (Manuscript History of the Church, 1 May 1842, Sunday Morning, Grove, Ehat, Andrew F., and Cook, Lyndon W., Words of Joseph Smith, pp.120-121)
Furthermore, Alma in the Book of Mormon asserts:
Behold thy brother hath said, What shall we do?—for we are cast out of our synagogues, that we cannot worship our God. Behold I say unto you, do ye suppose that ye cannot worship God save it be in your synagogues only? And moreover, I would ask, do ye suppose that ye must not worship God only once in a week? I say unto you, it is well that ye are cast out of your synagogues, that ye may be humble, and that ye may learn wisdom; for it is necessary that ye should learn wisdom; for it is because that ye are cast out, that ye are despised of your brethren because of your exceeding poverty, that ye are brought to a lowliness of heart; for ye are necessarily brought to be humble. (Alma 32:9-12).
So therefore, the LDS are much like the Kingdom of Judah, and the residue that have left the LDS Church to worship on their own are much like the Kingdom of Israel. Both scenarios can have excesses and insincere, unaccepted worship, as well as sincere and true worship. So, contrary to the prejudices of many active Latter-day Saints, non-members can legitimately worship outside of LDS temples, and still worship appropriately in the spirit of the temple.
On February 22, 1844, the Prophet Joseph Smith mentioned worship on the mountaintops when he instructed concerning an exploration of the West for a settlement of the Saints. He said:
I want every man that goes to be a king and a priest. When he gets on the mountains he may want to talk with his God (Documentary History of the Church, vol. 6, p. 224).
The Lord cuts a path between the extremists of the two by seeking those who will worship Him in spirit and in truth (see John 4:23), whether they be in the temple or on the mountains, or at a home altar. In “spirit” because His word is spirit, as the mind of God lays an unseen path before the believer to follow; and in “truth” because the faith and action of the believer brings the realities of the spirit (or the mind of God) into existence in the natural world when they elect to follow His commands. The realities of the spirit are only real if God has truly ordained it as opposed to it coming from a frenzied mind and heated imagination, and only if the believer has followed God’s words and guidance correctly.
And then received ye spirits which ye could not understand, and received them to be of God; and in this are ye justified? Behold ye shall answer this question yourselves; nevertheless, I will be merciful unto you; he that is weak among you hereafter shall be made strong. . . . And again, he that receiveth the word of truth, doth he receive it by the Spirit of truth or some other way? If it be some other way it is not of God. (D&C 50:15-16,19-20). And truth is knowledge of things as they are, and as they were, and as they are to come. (D&C 93:24).