Colored stones are used to make the Rainbow Galleon, the fastest boat in the game. There is only one stone for each of the seven colors of the rainbow. The stones are hidden randomly around the map.
Red- on the only island that has sand, on the underwater corner
Orange - Little mine with only copper and iron. Go behind the island and it's near the iron ore.
Yellow - In the big mine. Looks like sulfur but with no stud.
Green - On the only island with hemp.
Blue - On the island with copper and iron. Very big and has a hilly part. On the shore somewhere behind the hill
Indigo - Search the islands in the wiki and it's at plateau's island.
Purple - On a small sand sqaure in the water near the L shaped island with berries