🚫🔍 feat: disallow search indexing

[berry-13]

Summary

with DISALLOW_SEARCH_INDEXING, you can decide whether to allow or prevent search engines from indexing LibreChat

Change Type

• [x] New feature (non-breaking change which adds functionality)

Testing

true

false

Test Configuration:

Checklist

• [X] My code adheres to this project's style guidelines
• [X] I have performed a self-review of my own code
• [X] I have commented in any complex areas of my code
• [X] I have made pertinent documentation changes
• [X] My changes do not introduce new warnings
• [X] I have written tests demonstrating that my changes are effective or that my feature works
• [X] Local unit tests pass with my changes
• [X] Any changes dependent on mine have been merged and published in downstream modules.