If your dog is already house trained, the first step is to discuss the problem with your vet. It may be a health problem, wherein your dog can't control its behavior. If they're younger, it could also be that your dog is trying to establish its territory by marking it.
Either way, your dog isn't understanding that it should be defecating and urinating in a specific area. After all, it's not instinctive for dogs to relieve themselves in a certain place. The only place they won't do it is where they sleep.
So, patience is key. American Humane says never to rub a dog's nose in urine or feces or punish the dog for an accident, as it will instill fear. Instead, reward the dog whenever they do relieve themselves in the appropriate area. Having a feeding schedule will also help you both better understand certain times when your dog needs to go outside.