Rnightlights
Rnightlights copied to clipboard
chrisvwn
Error while downloading nightlight data for 2021
Hi Chris,
I am unable to download the data for year 2021, its not redirecting to the source https://eogdata.mines.edu/nighttime_light/monthly/v10/2021/. can you please suggest the solution?
Hi @naiktejashvi ,
Apologies for the late response. Could you post the code you are running.
Hi Chris,
Thank you for your response. Please find the code as requested : library(Rnightlights) library(reshape2) library(lubridate) library(rgdal) library(httr)
#Rnightlights:::nlCleanup() #file.remove(list.files(path = getNlDir("dirNlTemp"),pattern = "*.html",full.names = T)) ctry <- "IND" highestAdmLevelStats <- getCtryNlData(ctryCode = ctry, admLevel = "admin3", nlType = "VIIRS.M", nlPeriods = nlRange("202101","202102","VIIRS.M"), nlStats = list("sum",na.rm=TRUE), ignoreMissing=FALSE)
Actually, it seems the pages have changed and the code will need to be updated to reflect this. However, I may not be able to fix this any time soon as this will likely be time consuming.
Hi Chris,
Thank you for the immediate response. please find the output as below:
Processing missing data: IND:VIIRS.M:VCMCFG:avg_rade9:202101:sum, VIIRS.M:VCMCFG:avg_rade9:202102:sum. This may take a while. Note: Set 'ignoreMissing=TRUE' to return only data found or 'ignoreMissing=NULL' to return NULL if not all the data is found 2021-08-10 15:52:59: **** START PROCESSING: ctryCodes=IND, admLevels=list(IND = "gadm36_IND_3"), nlTypes=VIIRS.M, configNames=VCMCFG, extensions=avg_rade9, multiTileStrategy=all, multiTileMergeFun=mean, removeGasFlaresMethod=VTM, nlPeriods=c("202101", "202102"), nlStats=list(list("sum", na.rm = TRUE)), custPolyPath=NULL, gadmVersion=3.6, gadmPolyType=shpZip, downloadMethod=auto, cropMaskMethod=rast, extractMethod=rast**** 2021-08-10 15:52:59: Downloading country polygons ... | | 0%2021-08-10 15:52:59: Downloading polygon: IND 2021-08-10 15:52:59: Downloading ctry poly: IND 2021-08-10 15:52:59: Downloading ctry shpZip: IND 2021-08-10 15:52:59: Polygon dir for IND:3.6 already exists |==============================================================================| 100% 2021-08-10 15:52:59: Downloading country polygons ... DONE 2021-08-10 15:53:00: **** PROCESSING nlType:VIIRS.M | configName: VCMCFG | extension: avg_rade9 | nlPeriod:202101**** 2021-08-10 15:53:00: Checking tiles required for VIIRS.M 202101 2021-08-10 15:53:00: IND: Stats missing. Adding tiles 2021-08-10 15:53:00: numTiles: 1, Required tiles: 75N060E 2021-08-10 15:53:00: Downloading tile (1/1): 2021013 2021-08-10 15:53:03: ** Tile not available on the NOAA page. Please manually check for the 202101: Tile3_75N060E: VCMCFG at https://eogdata.mines.edu/download_dnb_composites.html'. If it exists please report this as a bug ** 2021-08-10 15:53:03: Something went wrong with the tile downloads. Aborting ... 2021-08-10 15:53:03: **** COMPLETED PROCESSING :ctryCodes=IND, admLevels=gadm36_IND_3, nlTypes=VIIRS.M, configNames=VCMCFG, extensions=avg_rade9, multiTileStrategy=all, multiTileMergeFun=mean, removeGasFlaresMethod=VTM, nlPeriods=c("202101", "202102"), nlStats=list(list("sum", na.rm = TRUE)), custPolyPath=NULL, gadmVersion=3.6, gadmPolyType=shpZip, downloadMethod=auto, cropMaskMethod=rast, extractMethod=rast**** Error in data.table::fread(getCtryNlDataFnamePath(ctryCode = ctryCode, : File 'D:/Rnightlights/.Rnightlights/data/NL_DATA_IND_ADM3_GADM-3.6-SHPZIP.csv' does not exist or is non-readable.
Confirmed. The nightlights index page structure has changed. The package scrapes the upstream tile list to get a list of available tiles. This will take a while to fix as there are likely other major changes that need to be addressed.
Hi Chris,
Thanks for the update. Please let me know, if in future you update the package and are able to extract the same.