Explore support for robots meta and canonical tag

[blite]

Should explore support for robots meta tags. Robots seems straightforward, but canonical tag might be thorny probably just needs an attempt to see what the real issues are.

[willhardy]

This would be a nice thing to add I think. Here are some useful values for such a tag:

• NoIndex
• NoFollow
• NoArchive
• NoSnippet (only google?)
• NoODP (partial support)
• NoYDIR (partial support, not google)
• NoImageIndex (only google)
• NoTranslate (only google)
• Unavailable_After (only google)

[thedrow]

If you could show me the format I'd be happy to contribute a patch