(from github.com/looper976)
I am trying to exclude the crawling of web results that look like the following: (feed)
http://site.net/category/reset/build/feed
http://site.net/category/reset/build/feed/
But I don’t want to exclude pages that have the word feed in the middle of the URL:
http://site.net/category/feeding/build/
I am trying to use something like this /feed$
but it does not appear to be working. Do I have to have the regex match the entire URL or just the specific pattern I am after?
Thanks,