You took the North Coast Route (via Matane)?
The campground is on road 132 (boulevard Griffon) at the entrance of the village of Anse-au-Griffon as you leave Rivière-au-Renard. In Rivière-au-Renard, be careful, you need to turn left in order to stay on the 132. If you simply follow the road, it becomes the 197 and you will head in the wrong direction.
South side (via Percé)?
The quickest route after passing downtown Gaspé is to follow the 197, reach Rivière-au-Renard and turn right at the 132.
Staying on the 132 will be more scenic and will bring you to the Forillon peninsula, as well as the villages of Cap-des-Rosiers and Anse-au-Griffon. You will find the campground 3 km from the centre of Anse-au-Griffon (where the harbour is located).