Add support for "robots" meta-tag

With this change, cgit will start to generate the "robots" meta-tag, using
a default value of "index, nofollow".

The default value can be modified with a new cgitrc variable, "robots".

Signed-off-by: Lars Hjemli <hjemli@gmail.com>

cgitrc

@@ -8,6 +8,11 @@
 #nocache=0
 
 
+## This variable can be used to override the default value for "robots"
+## meta-tag. If unset, the meta-tag isn't generated.
+#robots=index, nofollow
+
+
 ## Set allowed snapshot types by default. Can be overridden per repo
 # can be any combination of zip/tar.gz/tar.bz2/tar
 #snapshots=0