Ddrowe, I did not notice your post until today, or I'd have replied much sooner.
Yes, relic-hunting on National Park (or state park) property is VERY illegal.
NW GA is a good area for civil war relic digging. My suggestion about "where to start" is that you get into contact with some of the nearby relic-hunting guys. I know there used to be a civil war relic hunting club in Chattanooga, and another club in Dalton GA. Also, there's still a club in the Marietta GA area (North GA Relic Hunters), but attending that club's meetings would be a bit of a drive from Elijay.
Also, there is a good-sized civil war relic Show-and-Sale at Dalton GA, in a couple of weeks. That would be a good place to contact the local relic-club guys -- some of whom are kindhearted about welcoming a dad-and-son team into the hobby, freely sharing information and relic-hunting advice.
When you arrive at the show, talk to the guys who run the show.
I do not have the time-&-place info on the Dalton relic-show handy, but if you can't find it on the internet, let me know.
Regards,
Pete [P.C. George]