ice Key does not exist error when getLastMillis called

Key does not exist error when getLastMillis called

Open sojharo opened this issue 9 years ago • 13 comments

I am having this error and my data is not shown on the GUI.

    | Downloading: joda-time/joda-time/maven-metadata.xml
    May 22, 2015 12:20:11 AM java.util.prefs.WindowsPreferences <init>
    WARNING: Could not open/create prefs root node Software\JavaSoft\Prefs at root 0x80000002. Windows RegCreateKey| Running Grails application
    2015-05-22 00:21:51,823 [main] INFO  http11.Http11NioProtocol  - Initializing ProtocolHandler ["http-nio-8080"]

    2015-05-22 00:21:52,019 [main] INFO  net.NioSelectorPool  - Using a shared selector for servlet write/read
    2015-05-22 00:21:52,104 [main] INFO  core.StandardService  - Starting service Tomcat
    2015-05-22 00:21:52,109 [main] INFO  core.StandardEngine  - Starting Servlet Engine: Apache Tomcat/8.0.20
    2015-05-22 00:21:52,885 [localhost-startStop-1] INFO  core.StandardContext  - Unable to set the web application
     class loader property [clearReferencesStatic] to [false] as the property does not exist.
    2015-05-22 00:21:52,891 [localhost-startStop-1] INFO  core.StandardContext  - Unable to set the web application
     class loader property [clearReferencesStopThreads] to [false] as the property does not exist.
    2015-05-22 00:21:52,894 [localhost-startStop-1] INFO  core.StandardContext  - Unable to set the web application
     class loader property [clearReferencesStopTimerThreads] to [false] as the property does not exist.
    2015-05-22 00:21:52,904 [localhost-startStop-1] INFO  core.StandardContext  - Unable to set the web application
     class loader property [clearReferencesHttpClientKeepAliveThread] to [true] as the property does not exist.
    2015-05-22 00:21:53,024 [localhost-startStop-1] INFO  startup.ContextConfig  - No global web.xml found
    2015-05-22 00:21:53,934 [localhost-startStop-1] INFO  [localhost].[/ice]  - No Spring WebApplicationInitializer
     types detected on classpath
    2015-05-22 00:21:54,218 [localhost-startStop-1] INFO  [localhost].[/ice]  - Initializing Spring root WebApplicationContext
    2015-05-22 00:23:29,424 [localhost-startStop-1] INFO  context.GrailsConfigUtils  - [GrailsContextLoader] Grails
     application loaded.
    2015-05-22 00:23:31,307 [localhost-startStop-1] INFO  BootStrap  - Starting ice...
    2015-05-22 00:23:42,172 [localhost-startStop-1] INFO  processor.ProcessorConfig  - starting up...
    2015-05-22 00:23:42,342 [localhost-startStop-1] INFO  processor.BillingFileProcessor  - poller thread for com.netflix.ice.processor.BillingFileProcessor started...
    2015-05-22 00:23:42,344 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - poller starting...
    2015-05-22 00:23:42,391 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - trying to list objects in billing bucket jawaid using assume role, and external id
    2015-05-22 00:23:42,651 [localhost-startStop-1] INFO  basic.BasicManagers  - trying to find new tag group and data managers...
    2015-05-22 00:23:45,150 [localhost-startStop-1] INFO  basic.BasicManagers  - poller thread for com.netflix.ice.basic.BasicManagers started...
    2015-05-22 00:23:45,178 [com.netflix.ice.basic.BasicManagers] INFO  basic.BasicManagers  - poller starting...
    2015-05-22 00:23:45,365 [com.netflix.ice.basic.BasicManagers] INFO  basic.BasicManagers  - trying to find new tag group and data managers...
    2015-05-22 00:23:45,365 [localhost-startStop-1] INFO  [localhost].[/ice]  - Initializing Spring FrameworkServlet 'grails'
    2015-05-22 00:23:45,574 [main] INFO  http11.Http11NioProtocol  - Starting ProtocolHandler ["http-nio-8080"]
    | Server running. Browse to http://localhost:8080/ice
    2015-05-22 00:23:45,768 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - found 8 in billing bucket jawaid
    2015-05-22 00:23:45,846 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file 999304081095-aws-billing-csv-2015-05.csv
    2015-05-22 00:23:45,848 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file 999304081095-aws-billing-detailed-line-items-with-resources-and-tags-2015-05.csv.zip
    2015-05-22 00:23:45,850 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file 999304081095-aws-cost-allocation-2015-05.csv
    2015-05-22 00:23:45,969 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file aws-programmatic-access-test-object
    2015-05-22 00:23:46,060 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file mnt/
    2015-05-22 00:23:46,062 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file mnt/ice_processor/
    2015-05-22 00:23:46,100 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file mnt/ice_reader/
    | Error 2015-05-22 00:23:46,640 [com.netflix.ice.processor.BillingFileProcessor] ERROR processor.BillingFileProcessor  - Error reading from file lastProcessMillis_2015-05
    Message: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: 5759BD7A76830F63)
        Line | Method
    ->> 1077 | handleErrorResponse in com.amazonaws.http.AmazonHttpClient
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |    725 | executeOneRequest   in     ''
    |    460 | executeHelper . . . in     ''
    |    295 | execute             in     ''
    |   3710 | invoke . . . . . .  in com.amazonaws.services.s3.AmazonS3Client
    |   1146 | getObject           in     ''
    |   1018 | getObject . . . . . in     ''
    |    658 | getLastMillis       in com.netflix.ice.processor.BillingFileProcessor
    |    672 | lastProcessTime . . in     ''
    |    151 | poll                in     ''
    |     50 | doWork . . . . . .  in com.netflix.ice.common.Poller
    |     28 | access$000          in     ''
    |     88 | run . . . . . . . . in com.netflix.ice.common.Poller$1
    ^    745 | run                 in java.lang.Thread
    2015-05-22 00:23:46,879 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - trying to download 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip...
    2015-05-22 00:23:47,968 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - downloaded 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip
    2015-05-22 00:23:47,971 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - processing 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip...
    2015-05-22 00:23:48,087 [com.netflix.ice.processor.BillingFileProcessor] INFO  basic.BasicAccountService  - created account 999304081095.
    2015-05-22 00:23:48,365 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - done processing 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip
    2015-05-22 00:23:48,370 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - cut hours to 487
    2015-05-22 00:23:48,749 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - archiving results for 2015-05-01T00:00:00.000Z...
    2015-05-22 00:23:48,811 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - archiving tag data...
    2015-05-22 00:23:48,848 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.TagGroupWriter  - creating TagGroupWriter for \tagdb_all
    2015-05-22 00:23:49,066 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.TagGroupWriter  - tagdb_all uploading to s3...
    | Error 2015-05-22 00:23:49,092 [com.netflix.ice.processor.BillingFileProcessor] ERROR processor.BillingFileProcessor  - Error polling
    Message: null
       Line | Method
    ->> 246 | upload     in com.netflix.ice.common.AwsUtils
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |    73 | archive    in com.netflix.ice.processor.TagGroupWriter
    |   412 | archive .  in com.netflix.ice.processor.BillingFileProcessor
    |   235 | poll       in     ''
    |    50 | doWork . . in com.netflix.ice.common.Poller
    |    28 | access$000 in     ''
    |    88 | run . . .  in com.netflix.ice.common.Poller$1
    ^   745 | run        in java.lang.Thread
    2015-05-22 00:28:45,773 [com.netflix.ice.basic.BasicManagers] INFO  basic.BasicManagers  - trying to find new tag group and data managers...
    2015-05-22 00:28:49,164 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - trying to list objects in billing bucket jawaid using assume role, and external id
    2015-05-22 00:28:49,672 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - found 8 in billing bucket jawaid
    2015-05-22 00:28:49,697 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file 999304081095-aws-billing-csv-2015-05.csv
    2015-05-22 00:28:49,702 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file 999304081095-aws-billing-detailed-line-items-with-resources-and-tags-2015-05.csv.zip
    2015-05-22 00:28:49,703 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file 999304081095-aws-cost-allocation-2015-05.csv
    2015-05-22 00:28:49,708 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file aws-programmatic-access-test-object
    2015-05-22 00:28:49,709 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file mnt/
    2015-05-22 00:28:49,712 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file mnt/ice_processor/
    2015-05-22 00:28:49,714 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file mnt/ice_reader/
    | Error 2015-05-22 00:28:50,055 [com.netflix.ice.processor.BillingFileProcessor] ERROR processor.BillingFileProcessor  - Error reading from file lastProcessMillis_2015-05
    Message: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: 823E6091BDCF4D64)
        Line | Method
    ->> 1077 | handleErrorResponse in com.amazonaws.http.AmazonHttpClient
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |    725 | executeOneRequest   in     ''
    |    460 | executeHelper . . . in     ''
    |    295 | execute             in     ''
    |   3710 | invoke . . . . . .  in com.amazonaws.services.s3.AmazonS3Client
    |   1146 | getObject           in     ''
    |   1018 | getObject . . . . . in     ''
    |    658 | getLastMillis       in com.netflix.ice.processor.BillingFileProcessor
    |    672 | lastProcessTime . . in     ''
    |    151 | poll                in     ''
    |     50 | doWork . . . . . .  in com.netflix.ice.common.Poller
    |     28 | access$000          in     ''
    |     88 | run . . . . . . . . in com.netflix.ice.common.Poller$1
    ^    745 | run                 in java.lang.Thread
    2015-05-22 00:28:50,064 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - trying to download 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip...
    2015-05-22 00:28:51,165 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - downloaded 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip
    2015-05-22 00:28:51,169 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - processing 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip...
    2015-05-22 00:28:51,242 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - done processing 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip
    2015-05-22 00:28:51,875 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - cut hours to 487
    2015-05-22 00:28:51,882 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - archiving results for 2015-05-01T00:00:00.000Z...
    2015-05-22 00:28:51,883 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - archiving tag data...
    2015-05-22 00:28:51,888 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.TagGroupWriter  - creating TagGroupWriter for \tagdb_all
    2015-05-22 00:28:51,908 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.TagGroupWriter  - tagdb_all uploading to s3...
    | Error 2015-05-22 00:28:51,911 [com.netflix.ice.processor.BillingFileProcessor] ERROR processor.BillingFileProcessor  - Error polling
    Message: null
       Line | Method
    ->> 246 | upload     in com.netflix.ice.common.AwsUtils
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |    73 | archive    in com.netflix.ice.processor.TagGroupWriter
    |   412 | archive .  in com.netflix.ice.processor.BillingFileProcessor
    |   235 | poll       in     ''
    |    50 | doWork . . in com.netflix.ice.common.Poller
    |    28 | access$000 in     ''
    |    88 | run . . .  in com.netflix.ice.common.Poller$1
    ^   745 | run        in java.lang.Thread
    2015-05-22 00:33:47,726 [com.netflix.ice.basic.BasicManagers] INFO  basic.BasicManagers  - trying to find new tag group and data managers...
    2015-05-22 00:33:51,926 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - trying to list objects in billing bucket jawaid using assume role, and external id
    2015-05-22 00:33:52,548 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - found 8 in billing bucket jawaid
    2015-05-22 00:33:52,560 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file 999304081095-aws-billing-csv-2015-05.csv
    2015-05-22 00:33:52,562 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file 999304081095-aws-billing-detailed-line-items-with-resources-and-tags-2015-05.csv.zip
    2015-05-22 00:33:52,564 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file 999304081095-aws-cost-allocation-2015-05.csv
    2015-05-22 00:33:52,565 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file aws-programmatic-access-test-object
    2015-05-22 00:33:52,567 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file mnt/
    2015-05-22 00:33:52,567 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file mnt/ice_processor/
    2015-05-22 00:33:52,568 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file mnt/ice_reader/
    | Error 2015-05-22 00:33:53,068 [com.netflix.ice.processor.BillingFileProcessor] ERROR processor.BillingFileProcessor  - Error reading from file lastProcessMillis_2015-05
    Message: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: 88DA31B978AA52DD)
        Line | Method
    ->> 1077 | handleErrorResponse in com.amazonaws.http.AmazonHttpClient
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |    725 | executeOneRequest   in     ''
    |    460 | executeHelper . . . in     ''
    |    295 | execute             in     ''
    |   3710 | invoke . . . . . .  in com.amazonaws.services.s3.AmazonS3Client
    |   1146 | getObject           in     ''
    |   1018 | getObject . . . . . in     ''
    |    658 | getLastMillis       in com.netflix.ice.processor.BillingFileProcessor
    |    672 | lastProcessTime . . in     ''
    |    151 | poll                in     ''
    |     50 | doWork . . . . . .  in com.netflix.ice.common.Poller
    |     28 | access$000          in     ''
    |     88 | run . . . . . . . . in com.netflix.ice.common.Poller$1
    ^    745 | run                 in java.lang.Thread
    2015-05-22 00:33:53,077 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - trying to download 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip...
    2015-05-22 00:33:54,222 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - downloaded 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip
    2015-05-22 00:33:54,223 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - processing 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip...
    2015-05-22 00:33:54,264 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - done processing 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip
    2015-05-22 00:33:54,332 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - cut hours to 487
    2015-05-22 00:33:54,338 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - archiving results for 2015-05-01T00:00:00.000Z...
    2015-05-22 00:33:54,342 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - archiving tag data...
    2015-05-22 00:33:54,343 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.TagGroupWriter  - creating TagGroupWriter for \tagdb_all
    2015-05-22 00:33:54,549 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.TagGroupWriter  - tagdb_all uploading to s3...
    | Error 2015-05-22 00:33:54,553 [com.netflix.ice.processor.BillingFileProcessor] ERROR processor.BillingFileProcessor  - Error polling
    Message: null
       Line | Method
    ->> 246 | upload     in com.netflix.ice.common.AwsUtils
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |    73 | archive    in com.netflix.ice.processor.TagGroupWriter
    |   412 | archive .  in com.netflix.ice.processor.BillingFileProcessor
    |   235 | poll       in     ''
    |    50 | doWork . . in com.netflix.ice.common.Poller
    |    28 | access$000 in     ''
    |    88 | run . . .  in com.netflix.ice.common.Poller$1
    ^   745 | run        in java.lang.Thread
    2015-05-22 00:38:50,148 [com.netflix.ice.basic.BasicManagers] INFO  basic.BasicManagers  - trying to find new tag group and data managers...
    2015-05-22 00:38:54,575 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - trying to list objects in billing bucket jawaid using assume role, and external id
    2015-05-22 00:38:55,205 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - found 8 in billing bucket jawaid
    2015-05-22 00:38:55,229 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file 999304081095-aws-billing-csv-2015-05.csv
    2015-05-22 00:38:55,234 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file 999304081095-aws-billing-detailed-line-items-with-resources-and-tags-2015-05.csv.zip
    2015-05-22 00:38:55,237 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file 999304081095-aws-cost-allocation-2015-05.csv
    2015-05-22 00:38:55,238 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file aws-programmatic-access-test-object
    2015-05-22 00:38:55,239 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file mnt/
    2015-05-22 00:38:55,240 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file mnt/ice_processor/
    2015-05-22 00:38:55,241 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - ignoring file mnt/ice_reader/
    | Error 2015-05-22 00:38:55,600 [com.netflix.ice.processor.BillingFileProcessor] ERROR processor.BillingFileProcessor  - Error reading from file lastProcessMillis_2015-05
    Message: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: FB6A8E8F0C2BAE49)
        Line | Method
    ->> 1077 | handleErrorResponse in com.amazonaws.http.AmazonHttpClient
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |    725 | executeOneRequest   in     ''
    |    460 | executeHelper . . . in     ''
    |    295 | execute             in     ''
    |   3710 | invoke . . . . . .  in com.amazonaws.services.s3.AmazonS3Client
    |   1146 | getObject           in     ''
    |   1018 | getObject . . . . . in     ''
    |    658 | getLastMillis       in com.netflix.ice.processor.BillingFileProcessor
    |    672 | lastProcessTime . . in     ''
    |    151 | poll                in     ''
    |     50 | doWork . . . . . .  in com.netflix.ice.common.Poller
    |     28 | access$000          in     ''
    |     88 | run . . . . . . . . in com.netflix.ice.common.Poller$1
    ^    745 | run                 in java.lang.Thread
    2015-05-22 00:38:55,620 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - trying to download 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip...
    2015-05-22 00:38:56,731 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - downloaded 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip
    2015-05-22 00:38:56,734 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - processing 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip...
    2015-05-22 00:38:56,752 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - done processing 999304081095-aws-billing-detailed-line-items-2015-05.csv.zip
    2015-05-22 00:38:56,759 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - cut hours to 487
    2015-05-22 00:38:56,761 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - archiving results for 2015-05-01T00:00:00.000Z...
    2015-05-22 00:38:56,762 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.BillingFileProcessor
    - archiving tag data...
    2015-05-22 00:38:56,807 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.TagGroupWriter  - creating TagGroupWriter for \tagdb_all
    2015-05-22 00:38:56,822 [com.netflix.ice.processor.BillingFileProcessor] INFO  processor.TagGroupWriter  - tagdb_all uploading to s3...
    | Error 2015-05-22 00:38:56,829 [com.netflix.ice.processor.BillingFileProcessor] ERROR processor.BillingFileProcessor  - Error polling
    Message: null
       Line | Method
    ->> 246 | upload     in com.netflix.ice.common.AwsUtils
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |    73 | archive    in com.netflix.ice.processor.TagGroupWriter
    |   412 | archive .  in com.netflix.ice.processor.BillingFileProcessor
    |   235 | poll       in     ''
    |    50 | doWork . . in com.netflix.ice.common.Poller
    |    28 | access$000 in     ''
    |    88 | run . . .  in com.netflix.ice.common.Poller$1
    ^   745 | run        in java.lang.Thread

Here is the snapshot of how dashboard looks.

ice-1

May 22 '15 13:05 sojharo

HI Sojharo, I`m passing through the same issue, Do you solve them?

Thanks.

Feb 01 '16 18:02 lbardelli

I think we had solved it. It was an old project and we had left work on this. I don't remember exactly how we had solved it. Sorry :(

Feb 01 '16 18:02 sojharo

Godness!!! I had the same problem. Could it be something related to S3 policy or something? T_T

Mar 09 '16 10:03 khoi-thinh

@thinhduckhoi

My problem was solved. It was the key and the secret key, I just removed the "space" on conf.

Feel free to any question.

Bardelli

Mar 09 '16 10:03 lbardelli

What do you mean by "remove the space" on conf? The ice.properties or which file? Ah sorry, this was the tutorial i followed. https://gist.github.com/diegopacheco/70285842a2a5dcb89904#file-ice-md I just had to provide key and secret key when run app, not in any configuration file.

Mar 09 '16 10:03 khoi-thinh

Follow my conf files.

I made comments with "#look"

Note, do you need two buckets, one for collect billing information and another to Ice work.

Other important thing it is "ice.highstockUrl" (in ice.properties) if you use https its needed host this java file on other place.

Try these.

ice.properties.txt

docker-compose.yml.txt

Mar 09 '16 11:03 lbardelli

Please, let us know if work.

Mar 09 '16 14:03 lbardelli

Thank you so much for supporting me! Seems like i don't have permission to see Billing report. So i was not be able to build this container. Every time i execute command, it said something like: You're not authorized....bla bla.... But my co-worker did it successfully! Worked like a charm!

Mar 10 '16 03:03 khoi-thinh

There was still some minor error which i don't know why it happened everytime running the app. For example:

1.2016-03-10 15:19:07,089 [localhost-startStop-1] INFO core.StandardContext - Unable to set the web application class loader property [clearReferencesStatic] to [false] as the property does not exist. 2016-03-10 15:19:07,091 [localhost-startStop-1] INFO core.StandardContext - Unable to set the web application class loader property [clearReferencesStopThreads] to [false] as the property does not exist. 2016-03-10 15:19:07,092 [localhost-startStop-1] INFO core.StandardContext - Unable to set the web application class loader property [clearReferencesStopTimerThreads] to [false] as the property does not exist. 2016-03-10 15:19:07,092 [localhost-startStop-1] INFO core.StandardContext - Unable to set the web application class loader property [clearReferencesHttpClientKeepAliveThread] to [true] as the property does not exist. 2016-03-10 15:19:07,104 [localhost-startStop-1] INFO startup.ContextConfig - No global web.xml found 2016-03-10 15:19:07,283 [localhost-startStop-1] INFO [localhost].[/ice] - No Spring WebApplicationInitializer types detected on classpath 2016-03-10 15:19:07,329 [localhost-startStop-1] INFO [localhost].[/ice] - Initializing Spring root WebApplicationContext

I checked the /ice/work directory and found there was one file called web.xml.tmp. So the first things came to my mind was changing the name of that file. Sadly, after changed web.xml.tmp into web.xml, the same error happened. You have any idea what was the reason?

Btw, i checked Icy and Billing & Cost Management Dashboard about the cost of Feb -2016 and they didn't match. In Icy: Aggregated 1049.18 $ while the AWS dashboard said: 1046.49 $. How did that happen?

Mar 10 '16 07:03 khoi-thinh

Please check your format off billing reports: Format of the report file name:

<AWS account number>-aws-billing-detailed-line-items-with-resources-and-tags-yyyy-mm.csv.zip

This it my police on bucket with the reports:

{ "Version": "2012-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::7777777777777:root" }, "Action": "", "Resource": "arn:aws:s3:::ice-billingreports" }, { "Sid": "", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::77777777777:root" }, "Action": "s3:PutObject", "Resource": "arn:aws:s3:::ice-billingreports/" } ] }

I hope help you.

Mar 10 '16 10:03 lbardelli

Basically even those error happened, ICE still showed graphics and figures as i accessed the IP:8080. I just want to make sure if those errors has effected the figure's result or not.

Mar 14 '16 01:03 khoi-thinh

Hi!

My ice.properties is similar to the one pasted above but the issue persists for me:

2016-09-15 13:13:45,354 [com.netflix.ice.processor.BillingFileProcessor] WARN s3.AmazonS3Client - Attempting to re-send the request to .s3-external-1.amazonaws.com with AWS V4 authentication. To avoid this warning in the future, please use region-specific endpoint to access buckets located in regions that require V4 signing.

| Error 2016-09-15 13:13:47,001 [com.netflix.ice.processor.BillingFileProcessor] ERROR processor.BillingFileProcessor - Error reading from file lastProcessMillis_2016-09

Message: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: )

Line | Method

->> 1077 | handleErrorResponse in com.amazonaws.http.AmazonHttpClient

| 725 | executeOneRequest in ''

| 460 | executeHelper . . . in ''

| 295 | execute in ''

| 3710 | invoke . . . . . . in com.amazonaws.services.s3.AmazonS3Client

| 1146 | getObject in ''

| 1018 | getObject . . . . . in ''

| 658 | getLastMillis in com.netflix.ice.processor.BillingFileProcessor

| 672 | lastProcessTime . . in ''

| 151 | poll in ''

| 50 | doWork . . . . . . in com.netflix.ice.common.Poller

| 28 | access$000 in ''

| 88 | run . . . . . . . . in com.netflix.ice.common.Poller$1

^ 745 | run in java.lang.Thread

2016-09-15 13:13:47,004 [com.netflix.ice.processor.BillingFileProcessor] INFO processor.BillingFileProcessor - trying to download /-aws-billing-detailed-line-items-2016-09.csv.zip...

| Error 2016-09-15 13:13:47,017 [com.netflix.ice.processor.BillingFileProcessor] ERROR processor.BillingFileProcessor - Error polling

Message: Not Found (Service: Amazon S3; Status Code: 404; Error Code: 404 Not Found; Request ID: )

Line | Method

->> 1077 | handleErrorResponse in com.amazonaws.http.AmazonHttpClient

| 725 | executeOneRequest in ''

| 460 | executeHelper . . . . . . in ''

| 295 | execute in ''

| 3710 | invoke . . . . . . . . . . in com.amazonaws.services.s3.AmazonS3Client

| 1010 | getObjectMetadata in ''

| 988 | getObjectMetadata . . . . in ''

| 275 | downloadFileIfChangedSince in com.netflix.ice.common.AwsUtils

| 175 | poll . . . . . . . . . . . in com.netflix.ice.processor.BillingFileProcessor

| 50 | doWork in com.netflix.ice.common.Poller

| 28 | access$000 . . . . . . . . in ''

| 88 | run in com.netflix.ice.common.Poller$1

^ 745 | run . . . . . . . . . . . in java.lang.Thread

Kindly let me know how to solve this. Theres no data in the UI , Working S3 Bucket and local ice server.

Sep 15 '16 11:09 vaniparidhyani

Hi @vaniparidhyani, It's seems something related with IAM permissions, maybe at the S3, or even on the Ec2.

Sep 19 '16 08:09 lbardelli

ice ice copied to clipboard

Key does not exist error when getLastMillis called

ice
ice copied to clipboard