Index alias not accepted

[ghost]

Describe the bug

Ill preface this by saying this may not be a bug just my understanding of how fscrawler works.

I have made an alias for my index as i have ilm policy and rollover indexes setup, the alias is 'movies-index-alias' which points to movies-index-000001, when i curl or use dev tools in elastic I have no issue. When i try to set 'movies-index-alias' as the index name in _settings.yaml and run i get the below error in the log:

Job Settings

name: "movies"
fs:
  url: "/Users/user/Downloads"
  ocr:
    language: "eng"
  checksum: "SHA-1"
elasticsearch:
  nodes:
  - url: "http://localhost:9200"
  //alias: "movies-index-alias" //I tried this as a hail mary
  index: "movies-index-alias"
  username: "x"
  password: "x"
  pipeline: "ingest-pipeline"

Logs

11:56:12,296 ^[[32mINFO ^[[m [f.p.e.c.f.c.BootstrapChecks] Memory [Free/Total=Percent]: HEAP [92.9mb/2gb=4.54%], RAM [48.7mb/8gb=0.6%], Swap [813.2mb/3gb=26.47%].
11:56:12,550 ^[[32mINFO ^[[m [f.p.e.c.f.FsCrawlerImpl] Starting FS crawler
11:56:13,008 ^[[32mINFO ^[[m [f.p.e.c.f.c.v.ElasticsearchClientV7] Elasticsearch Client for version 7.x connected to a node running version 7.8.0
11:56:13,149 ^[[32mINFO ^[[m [f.p.e.c.f.c.v.ElasticsearchClientV7] Elasticsearch Client for version 7.x connected to a node running version 7.8.0
11:56:13,262 ^[[33mWARN ^[[m [f.p.e.c.f.c.v.ElasticsearchClientV7] failed to create index [movies-index-alias], disabling crawler...
11:56:13,262 ^[[1;31mFATAL^[[m [f.p.e.c.f.c.FsCrawlerCli] We can not start Elasticsearch Client. Exiting.
org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=invalid_index_name_exception, reason=Invalid index name [movies-index-alias], already exists as alias]
        at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:176) ~[elasticsearch-7.11.1.jar:7.11.1]
        at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1900) ~[elasticsearch-rest-high-level-client-7.11.1.jar:7.11.1]
        at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1877) ~[elasticsearch-rest-high-level-client-7.11.1.jar:7.11.1]
        at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1634) ~[elasticsearch-rest-high-level-client-7.11.1.jar:7.11.1]
        at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1606) ~[elasticsearch-rest-high-level-client-7.11.1.jar:7.11.1]
        at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1573) ~[elasticsearch-rest-high-level-client-7.11.1.jar:7.11.1]
        at org.elasticsearch.client.IndicesClient.create(IndicesClient.java:134) ~[elasticsearch-rest-high-level-client-7.11.1.jar:7.11.1]
        at fr.pilato.elasticsearch.crawler.fs.client.v7.ElasticsearchClientV7.createIndex(ElasticsearchClientV7.java:228) ~[fscrawler-elasticsearch-client-v7-2.7-SNAPSHOT.jar:?]
        at fr.pilato.elasticsearch.crawler.fs.client.v7.ElasticsearchClientV7.createIndex(ElasticsearchClientV7.java:640) ~[fscrawler-elasticsearch-client-v7-2.7-SNAPSHOT.jar:?]
        at fr.pilato.elasticsearch.crawler.fs.client.v7.ElasticsearchClientV7.createIndices(ElasticsearchClientV7.java:473) ~[fscrawler-elasticsearch-client-v7-2.7-SNAPSHOT.jar:?]
        at fr.pilato.elasticsearch.crawler.fs.service.FsCrawlerDocumentServiceElasticsearchImpl.createSchema(FsCrawlerDocumentServiceElasticsearchImpl.java:62) ~[fscrawler-core-2.7-SNAPSHOT.jar:?]
        at fr.pilato.elasticsearch.crawler.fs.FsCrawlerImpl.start(FsCrawlerImpl.java:118) ~[fscrawler-core-2.7-SNAPSHOT.jar:?]
        at fr.pilato.elasticsearch.crawler.fs.cli.FsCrawlerCli.startEsClient(FsCrawlerCli.java:307) [fscrawler-cli-2.7-SNAPSHOT.jar:?]
        at fr.pilato.elasticsearch.crawler.fs.cli.FsCrawlerCli.main(FsCrawlerCli.java:283) [fscrawler-cli-2.7-SNAPSHOT.jar:?]
        Suppressed: org.elasticsearch.client.ResponseException: method [PUT], host [http://localhost:9200], URI [/movies-index-alias?master_timeout=30s&timeout=30s], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"invalid_index_name_exception","reason":"Invalid index name [movies-index-alias], already exists as alias","index_uuid":"_na_","index":"movies-index-alias"}],"type":"invalid_index_name_exception","reason":"Invalid index name [movies-index-alias], already exists as alias","index_uuid":"_na_","index":"movies-index-alias"},"status":400}
                at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:326) ~[elasticsearch-rest-client-7.11.1.jar:7.11.1]
                at org.elasticsearch.client.RestClient.performRequest(RestClient.java:296) ~[elasticsearch-rest-client-7.11.1.jar:7.11.1]
                at org.elasticsearch.client.RestClient.performRequest(RestClient.java:270) ~[elasticsearch-rest-client-7.11.1.jar:7.11.1]
                at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1621) ~[elasticsearch-rest-high-level-client-7.11.1.jar:7.11.1]
                at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1606) ~[elasticsearch-rest-high-level-client-7.11.1.jar:7.11.1]
                at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1573) ~[elasticsearch-rest-high-level-client-7.11.1.jar:7.11.1]
                at org.elasticsearch.client.IndicesClient.create(IndicesClient.java:134) ~[elasticsearch-rest-high-level-client-7.11.1.jar:7.11.1]
                at fr.pilato.elasticsearch.crawler.fs.client.v7.ElasticsearchClientV7.createIndex(ElasticsearchClientV7.java:228) ~[fscrawler-elasticsearch-client-v7-2.7-SNAPSHOT.jar:?]
                at fr.pilato.elasticsearch.crawler.fs.client.v7.ElasticsearchClientV7.createIndex(ElasticsearchClientV7.java:640) ~[fscrawler-elasticsearch-client-v7-2.7-SNAPSHOT.jar:?]
                at fr.pilato.elasticsearch.crawler.fs.client.v7.ElasticsearchClientV7.createIndices(ElasticsearchClientV7.java:473) ~[fscrawler-elasticsearch-client-v7-2.7-SNAPSHOT.jar:?]
                at fr.pilato.elasticsearch.crawler.fs.service.FsCrawlerDocumentServiceElasticsearchImpl.createSchema(FsCrawlerDocumentServiceElasticsearchImpl.java:62) ~[fscrawler-core-2.7-SNAPSHOT.jar:?]
                at fr.pilato.elasticsearch.crawler.fs.FsCrawlerImpl.start(FsCrawlerImpl.java:118) ~[fscrawler-core-2.7-SNAPSHOT.jar:?]
                at fr.pilato.elasticsearch.crawler.fs.cli.FsCrawlerCli.startEsClient(FsCrawlerCli.java:307) [fscrawler-cli-2.7-SNAPSHOT.jar:?]
                at fr.pilato.elasticsearch.crawler.fs.cli.FsCrawlerCli.main(FsCrawlerCli.java:283) [fscrawler-cli-2.7-SNAPSHOT.jar:?]
11:56:13,269 ^[[32mINFO ^[[m [f.p.e.c.f.FsCrawlerImpl] FS crawler [movies] stopped
11:56:13,270 ^[[32mINFO ^[[m [f.p.e.c.f.FsCrawlerImpl] FS crawler [movies] stopped

Expected behavior

It would just insert into the index the alias points to.

Versions:

• OS: [MacOS-10.15.6]
• Version [ES-7.8.1]

Attachment

N/A

[dadoonet]

I'm making this a feature request. I'd actually like to support Index Templates instead of manually trying to create index settings. That's somewhat connected to #643.

Writing to an alias is always a good thing so I think this is something we should support OOTB.

[ghost]

Data streams is the newer alternative to aliases, am I correct in assuming the same issues would affect the use of data streams?

[dadoonet]

That sounds correct to me.