improve logic for tex-in-latex-p

[dankessler]

Do checking similar to how markdown mode is checked: specifically check the text property, since an overlay could cover it up (e.g., hl-line-mode masks things). Also check for 'tex-math face (defined in built-in latex-mode).