I cannot find a clear regexp to exclude indexing files from a whole domain
includde URls: http://example.com/.*
Exludeed URls for crawling: http://example.com/./.jpg
What is the right regexp to exclude indexing files like .jpg .css .php in a whole domain?
For php, it contains request parameters.
Thnx a lot @marevol . Another question is how can I exclude an entire URL which contains a specific word? Words like category or tag
What would be the correct regexp to exclude category from the above URL?
This is Java Regular Expression.