It probably is the south side. What you are describing to me sounds like rotten logs. They will typically turn dark first. Tap on them with a hammer to see if they sound hollow. If not, sand them then try to blend in with the rest of the home with satin. It may take several coats. DO NOT fill all of the ”tiny cracks”.