encodeTitle() runs page titles through java.net.URLEncoder, which escapes everything except letters, digits and . - * _. So encodedTitle, and the url that is derived for Wikipedia dumps, turn / into %2F, : into %3A and ( ) into %28 %29. This applies to both source=xml and source=cirrus, since both go through the same code.
MediaWiki builds its own page URLs with wfUrlencode(), which keeps ; @ $ ! * ( ) , / ~ : literal. See mediawiki/includes/GlobalFunctions.php at b74e52aaad657f46afb3a925de704bc5565f2a69 · wikimedia/mediawiki · GitHub . The generated links therefore don’t match the wiki’s canonical URLs (Mercury_%28planet%29 instead of Mercury_(planet), Talk%3AFoo instead of Talk:Foo). Subpage links break completely: /index.php/Foo%2FBar returns 404 on Apache’s default config (AllowEncodedSlashes Off), which is how most self-hosted MediaWiki installs run.
I’ve opened a pull request that fixes this:
- #27 ( fix: encode article titles the way MediaWiki does by ram-fabian-heinrich · Pull Request #27 · codelibs/fess-ds-wikipedia · GitHub ):
after percent-encoding,encodeTitle()undoes exactly the escapeswfUrlencode()undoes.& ? # % + = ' "and non-ASCII characters stay percent-encoded, as in MediaWiki. Tests cover the kept and escaped characters, subpages, and the derived Wikipedia URL. Existing installs get the new links on their next crawl.
Motivation
We index our internal MediaWiki through its XML export and build the result URL from encodedTitle in the handler script. Every search result for a subpage (Foo/Bar) led to a 404, and titles with brackets or namespaces got URLs that differ from the ones the wiki shows. We currently work around it with a chain of .replace() calls in the handler script, but since every MediaWiki encodes its URLs this way, I think the plugin should produce them directly. I’m happy to adjust the PR.