Black Mushroom - This spawns in Cathedral Level 1, Act 1
Shinbone - Spawns randomly in the fireplace in Leoric's Manor, Act 1
Liquid Rainbow - Spawns in the Mysterious Cave in the Mysterious Chest in the Oasis, Act 2. Save the old man and he will open the chest.
Wirts Bell is sold by a merchants in the city, Act 2.
Gibbering Gemstone - A random drop from Chiltara in the Caverns of Frost, Act 3. It's in the Fields of Slaughter, Ballistae fight area. If you get Ice Caverns exit out try again. The mob is random.
The Plans for the Staff of Herding drops randomly from Izual, Act 4
Then have 50k gold and teach Haedrig how to build the staff
Between New Tristram and Old Tristram, you will find a dead cow. Talking to the Ghost of the Cow King will grant you entry to Whimsyshire.