I walked around the city today, and there is not really much to see.
I decided to walk here and the fort is not impressive, but you get a good view of the city. There was some military members of there having a meeting in formation, and I didn't want to get too close and bother them, so I left...
This small fort is mostly a ruin, and hosts beneath it one of the city's reservoirs. There are a few derelict buildings, but that is all. Right next to the military base, Fort Ceperou is best used as the premier vantage point in Cayenne. Seeing it takes 5 minutes if you're slow.
Find the steps if you don't have a car. It's not a strenuous walk and well worth it for the views. The turret is in disrepair and the door off it's hinges but there was no litter and a pleasant breeze at the top.