In the Find/Change dialog, click on the button to the right of the "Find what:" field, then pick "any page number." Usually by choosing the GREP tab, you can put a pipe character ("|") between parts of the expression to match any of the parts delimited by the pipe, so ~#|~x should search for page numbers and section markers in the same search. Sometimes this doesn't seem to work right, though, so you may want to add your own and make sure it finds it. GREP searching for "\x18" also find page numbers, and "\x19" section markers, and "\x18|\x19" for both.
There are other things that could mess you up, though, like indexes, chapter numbers, etc.