feat: disable search engine indexing via configuration

[l3pr-org]

Developer Certificate of Origin

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved.

PR

Resolves #221

This PR allows users to allow/deny their OTS instance from being indexed by search engines (that abide by robots.txt). Default behavior is to deny indexing for all user agents on all paths. Unless explicitly set (disableSearchIndex: false), a robots.txt file with the below contents will be served on /robots.txt

User-agent: *
Disallow: /

I git pulled my repo and ran with no issues. Everything compiled with no errors and I was able to go run . --config config-file.yaml with disableSearchIndex : false to change /robots.txt back to a 404 error.

Wiki must be updated with the new configuration option if this PR is merged.