Ingest fails to get a connection

[DuncanRowland]

I have added a couple of resources in a couple of different ways (dr0,dr1 below), but the ingest command fails to get a cassandra connection (as immediately below). Nodetool seems to indicate the cluster is OK (Up&Normal, but I'm no expert here)...

root@atlas-instance-1:/var/log/cassandra# nodetool status           
Note: Ownership information does not include topology; for complete information, specify a keyspace
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Owns   Host ID                               Token                                    Rack
UN  127.0.0.1  42.61 KB   100.0%  4e9989b3-50c6-42c0-8baf-2b3e12e74dc5  0fab72bb6a3f06de62c0c6b17fe8947f         `rack1`

biocache> list
 --------------------------------------------------------------------------------------------
 | name                          | uri                                                | uid |
 |------------------------------------------------------------------------------------------|
 | Lund Botanical Museum (LD)    | http://ala-demo.org/collectory/ws/dataResource/dr1 | dr1 |
 | The ARC Rare Species Database | http://ala-demo.org/collectory/ws/dataResource/dr0 | dr0 |
 --------------------------------------------------------------------------------------------

biocache> ingest -dr dr1
2017-02-12 18:12:34,529 INFO : [IngestTool] - Ingesting resource uid: dr1
2017-02-12 18:12:34,529 INFO : [IngestTool] - Loading: dr1
2017-02-12 18:12:34,617 INFO : [DataLoader] - Darwin core archive loading
2017-02-12 18:12:34,659 INFO : [DataLoader] - Downloading zip file from http://ala-demo.org/collectory/upload/1486922963828/aab0cf80-0c64-11dd-84d1-b8a03c50a862.zip
2017-02-12 18:12:34,664 INFO : [DataLoader] -  Content-Disposition: attachment;filename=aab0cf80-0c64-11dd-84d1-b8a03c50a862.zip
2017-02-12 18:12:34,664 INFO : [DataLoader] - Downloaded. File size: 
2017-02-12 18:12:34,664 INFO : [DataLoader] - Extracting ZIP /data/biocache-load/dr1.zip
2017-02-12 18:12:34,666 INFO : [DataLoader] - Archive extracted to directory: /data/biocache-load/dr1
2017-02-12 18:12:34,666 INFO : [DataLoader] - File last modified date: Sun Feb 12 18:12:34 UTC 2017
2017-02-12 18:12:34,667 INFO : [DataLoader] - Loading archive: /data/biocache-load/dr1 for resource: dr1, with unique terms: List(gbif:gbifID), stripping spaces:  false, incremental: false,  testing: false
2017-02-12 18:12:34,964 INFO : [DataLoader] - The current institution codes for the data resource: Set()
2017-02-12 18:12:34,964 INFO : [DataLoader] - The current collection codes for the data resource: Set()
2017-02-12 18:16:34,982 ERROR: [CommonsBackedPool] - Failed to get a connection within the configured wait time because there are no available nodes. This possibly indicates that either the suspension strategy is too aggressive or that your cluster is in a bad way.
2017-02-12 18:16:34,982 ERROR: [CassandraPersistenceManager] - Failed to get a connection within the configured max wait time.
org.scale7.cassandra.pelops.exceptions.NoConnectionsAvailableException: Failed to get a connection within the configured max wait time.
	at org.scale7.cassandra.pelops.pool.CommonsBackedPool.getConnectionExcept(CommonsBackedPool.java:345)
	at org.scale7.cassandra.pelops.Operand.tryOperation(Operand.java:77)
	at org.scale7.cassandra.pelops.Operand.tryOperation(Operand.java:66)
	at org.scale7.cassandra.pelops.Selector.getColumnOrSuperColumnFromRow(Selector.java:311)
	at org.scale7.cassandra.pelops.Selector.getColumnFromRow(Selector.java:330)
	at org.scale7.cassandra.pelops.Selector.getColumnFromRow(Selector.java:306)
	at org.scale7.cassandra.pelops.Selector.getColumnFromRow(Selector.java:292)
	at au.org.ala.biocache.persistence.CassandraPersistenceManager.get(CassandraPersistenceManager.scala:167)
	at au.org.ala.biocache.dao.OccurrenceDAOImpl.getUUIDForUniqueID(OccurrenceDAOImpl.scala:168)
	at au.org.ala.biocache.dao.OccurrenceDAOImpl.createOrRetrieveUuid(OccurrenceDAOImpl.scala:157)
	at au.org.ala.biocache.load.DwCALoader.getUuid(DwCALoader.scala:147)
	at au.org.ala.biocache.load.DwCALoader.loadArchive(DwCALoader.scala:256)
	at au.org.ala.biocache.load.DwCALoader$$anonfun$load$1.apply(DwCALoader.scala:120)
	at au.org.ala.biocache.load.DwCALoader$$anonfun$load$1.apply(DwCALoader.scala:111)
	at scala.collection.immutable.List.foreach(List.scala:318)
	at au.org.ala.biocache.load.DwCALoader.load(DwCALoader.scala:111)
	at au.org.ala.biocache.load.Loader.load(Loader.scala:166)
	at au.org.ala.biocache.load.IngestTool$.ingestResource(IngestTool.scala:101)
	at au.org.ala.biocache.load.IngestTool$$anonfun$main$1.apply(IngestTool.scala:54)
	at au.org.ala.biocache.load.IngestTool$$anonfun$main$1.apply(IngestTool.scala:51)
	at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
	at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
	at au.org.ala.biocache.load.IngestTool$.main(IngestTool.scala:51)
	at au.org.ala.biocache.cmd.CMD2$.main(CMD2.scala:103)
	at au.org.ala.biocache.cmd.CMD2.main(CMD2.scala)
2017-02-12 18:16:34,983 ERROR: [DataLoader] - Failed to get a connection within the configured max wait time.
org.scale7.cassandra.pelops.exceptions.NoConnectionsAvailableException: Failed to get a connection within the configured max wait time.

[JeGoi]

Hello, It's old but anyway. I'm not sure but we had a similar issue.

19:04:01,796 INFO : [DataLoader] - The current institution codes for the data resource: Set()
19:04:01,797 INFO : [DataLoader] - The current collection codes for the data resource: Set()
19:08:01,865 ERROR: [CommonsBackedPool] - Failed to get a connection within the configured wait time because there are no available nodes. This possibly indicates that either the suspension strategy is too aggressive or that your cluster is in a bad way.
19:08:01,868 ERROR: [CassandraPersistenceManager] - Failed to get a connection within the configured max wait time.
org.scale7.cassandra.pelops.exceptions.NoConnectionsAvailableException: Failed to get a connection within the configured max wait time.
	at org.scale7.cassandra.pelops.pool.CommonsBackedPool.getConnectionExcept(CommonsBackedPool.java:345)

For us, during Cassandra installation, the database was not really created. To test, if it's the same issue

$ cassandra-cli
Connected to: "Biocache" on 127.0.0.1/9160
Welcome to Cassandra CLI version 1.2.19

Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.

[default@unknown] use occ;
Keyspace 'occ' not found.

For us, the occ database was not created. You can do it by hand by using the same template as here https://github.com/AtlasOfLivingAustralia/ala-install/blob/master/ansible/roles/biocache-db/files/cassandra/cassandra-schema.txt and the same command line as in https://github.com/AtlasOfLivingAustralia/ala-install/blob/master/ansible/roles/biocache-db/tasks/main.yml#L26

Many thanks to Dave Martin. Regards