RGA
RGA copied to clipboard
artemklevtsov
as.request error
Hi, I am getting the following error when utilizing the RGA package in R:
Error in UseMethod("as.request") :
no applicable method for 'as.request' applied to an object of class "function"
Has something changed in the Google Analtyics API that is causing this issue?
Besides the RGA package, what other packages will need to be updated as well?
I have updated the packages and still get the error from time to time. Not sure what is causing it since the packages are updated.
Please post the traceback() output here. Also a code to reproduce the error would be helpful.
> traceback()
9: as.request(config)
8: request_build("GET", hu$url, as.request(config), ...)
7: GET(url, config = config, accept_json())
6: api_request(get_url(path), token)
5: get_mgmt(path, token)
4: get_profile(profile$accountId, profile$webPropertyId, profile$profileId,
token)
3: get_report("data/ga", query, token, fetch.by)
2: get_ga(profileId = "ga:xxxxxxx", start.date = stdate, end.date = enddate,
metrics = c("ga:sessions", "ga:pageviews", "ga:pageLoadTime",
"ga:timeonPage"), dimensions = c("ga:date", "ga:dimension13",
"ga:dimension14"), sort = sort, filters = filters, segment = NULL,
samplingLevel = NULL, start.index = NULL, max.results = NULL,
include.empty.rows = NULL, fetch.by = NULL, ga_token)
1: rbind(Opinion_Users2015 <- get_ga(profileId = "ga:xxxxxxxx",
start.date = stdate, end.date = enddate, metrics = c("ga:sessions",
"ga:pageviews", "ga:pageLoadTime", "ga:timeonPage"),
dimensions = c("ga:date", "ga:dimension13", "ga:dimension14"),
sort = sort, filters = filters, segment = NULL, samplingLevel = NULL,
start.index = NULL, max.results = NULL, include.empty.rows = NULL,
fetch.by = NULL, ga_token)[1, c(1:7)]) at #12
>
This is what happens when it throws another as.request error as well.
Batch processing mode enabled.
Fetch data by day: from 2015/10/01 to 2015/10/31.
|============================================================================| 100%
Error in vapply(x, function(x) as.numeric(x[[name]]), numeric(1)) :
values must be length 1,
but FUN(X[[1]]) result is length 0
In addition: There were 31 warnings (use warnings() to see them)
> traceback()
5: vapply(x, function(x) as.numeric(x[[name]]), numeric(1))
4: sum_by(pages, "sampleSize")
3: fetch_by(path, query, by, token)
2: get_report("data/ga", query, token, fetch.by)
1: get_ga(profileId = "ga:94268785", start.date = "2015/10/01",
end.date = "2015/10/31", metrics = c("ga:users", "ga:sessions",
"ga:pageviews", "ga:pageLoadTime", "ga:timeonPage"),
dimensions = c("ga:date", "ga:dimension13", "ga:dimension14"),
sort = "-ga:users", filters = "ga:dimension7==national",
segment = NULL, samplingLevel = "FASTER", start.index = 1,
max.results = 1, include.empty.rows = NULL, fetch.by = "day",
ga_token)
Try the latest git-version of the RGA package. Also make sure to ue the latest version of the curl and httr packages.
I have updated all packages and now I am getting this error as well.
traceback() 5: vapply(x, function(x) as.numeric(x[[name]]), numeric(1)) 4: sum_by(pages, "sampleSize") 3: fetch_by(path, query, by, token) 2: get_report("data/ga", query, token, fetch.by) 1: get_ga(profileId = "ga:94268785", start.date = "2015-8-01", end.date = "2015-8-31", metrics = c("ga:sessions", "ga:pageviews", "ga:pageLoadTime", "ga:timeonPage"), dimensions = c("ga:date", "ga:dimension13", "ga:dimension14"), sort = "-ga:pageviews", filters = "ga:dimension7==hockey", segment = NULL, samplingLevel = NULL, start.index = 1, max.results = 1, include.empty.rows = NULL, fetch.by = "day", ga_token)
I encountered this issue today as well. It runs for awhile before encountering the error
Batch processing mode enabled. Fetch data by day: from 2016-04-01 to 2016-09-27. |======================================= | 36% Error in UseMethod("as.request") : no applicable method for 'as.request' applied to an object of class "function"
Timlri,
I think that it may be something with the dates. If you attempt to do only 30days in batch mode by day the code seldom breaks. I then have been utilizing the rbind function to pull the data together into one matrix.
Thanks. If I simply re-run the script it often completes okay with the given date range. And if it does fail, it usually fails either earlier or later - not at the same point.
Guys, can I help you? Does someone have a reproducible example of code?
Here is the traceback code that I received from R.
Batch processing mode enabled. Fetch data by day: from 2015-04-01 to 2015-04-30. |===== | 7% Error in UseMethod("as.request") : no applicable method for 'as.request' applied to an object of class "function" In addition: Warning messages: 1: Only 1 observations out of 78 were obtained. Set max.results = NULL (default value) to get all results. 2: Only 1 observations out of 72 were obtained. Set max.results = NULL (default value) to get all results.
traceback() 15: as.request(config) 14: request_build("GET", hu$url, as.request(config), ...) 13: GET(url, config = config, accept_json()) 12: api_request(get_url(path, query), token) 11: get_data(path, query, token) 10: eval(expr, envir, enclos) 9: eval(expr, pf) 8: withVisible(eval(expr, pf)) 7: evalVis(expr) 6: capture.output(pages[[i]] <- get_data(path, query, token)) 5: withCallingHandlers(expr, message = function(c) invokeRestart("muffleMessage")) 4: suppressMessages(capture.output(pages[[i]] <- get_data(path, query, token))) 3: fetch_by(path, query, by, token) 2: get_report("data/ga", query, token, fetch.by) 1: get_ga(profileId = "ga:94268785", start.date = "2015-04-01", end.date = "2015-04-30", metrics = c("ga:sessions", "ga:pageviews", "ga:pageLoadTime", "ga:timeonPage", "ga:avgTimeOnPage", "ga:bounces", "ga:bouncerate"), dimensions = c("ga:date", "ga:dimension13", "ga:dimension14"), sort = "-ga:pageviews", filters = "ga:dimension7==books-and-literature", segment = NULL, samplingLevel = NULL, start.index = 1, max.results = 1, include.empty.rows = NULL, fetch.by = "day", ga_token)
And again.
Batch processing mode enabled. Fetch data by day: from 2015-06-01 to 2015-06-30. |================================================== | 67% Error in UseMethod("as.request") : no applicable method for 'as.request' applied to an object of class "function" In addition: There were 20 warnings (use warnings() to see them)
traceback() 15: as.request(config) 14: request_build("GET", hu$url, as.request(config), ...) 13: GET(url, config = config, accept_json()) 12: api_request(get_url(path, query), token) 11: get_data(path, query, token) 10: eval(expr, envir, enclos) 9: eval(expr, pf) 8: withVisible(eval(expr, pf)) 7: evalVis(expr) 6: capture.output(pages[[i]] <- get_data(path, query, token)) 5: withCallingHandlers(expr, message = function(c) invokeRestart("muffleMessage")) 4: suppressMessages(capture.output(pages[[i]] <- get_data(path, query, token))) 3: fetch_by(path, query, by, token) 2: get_report("data/ga", query, token, fetch.by) 1: get_ga(profileId = "ga:94268785", start.date = "2015-06-01", end.date = "2015-06-30", metrics = c("ga:sessions", "ga:pageviews", "ga:pageLoadTime", "ga:timeonPage", "ga:avgTimeOnPage", "ga:bounces", "ga:bouncerate"), dimensions = c("ga:date", "ga:dimension13", "ga:dimension14"), sort = "-ga:pageviews", filters = "ga:dimension7==books-and-literature", segment = NULL, samplingLevel = NULL, start.index = 1, max.results = 1, include.empty.rows = NULL, fetch.by = "day", ga_token)
I then run the code a few seconds to a minute later and it will work from time to time. Sometimes the code will break again as well.
Here is the traceback when the code executes properly.
Batch processing mode enabled. Fetch data by day: from 2015-04-01 to 2015-04-30. |===========================================================================| 100% There were 31 warnings (use warnings() to see them)
traceback() 15: as.request(config) 14: request_build("GET", hu$url, as.request(config), ...) 13: GET(url, config = config, accept_json()) 12: api_request(get_url(path, query), token) 11: get_data(path, query, token) 10: eval(expr, envir, enclos) 9: eval(expr, pf) 8: withVisible(eval(expr, pf)) 7: evalVis(expr) 6: capture.output(pages[[i]] <- get_data(path, query, token)) 5: withCallingHandlers(expr, message = function(c) invokeRestart("muffleMessage")) 4: suppressMessages(capture.output(pages[[i]] <- get_data(path, query, token))) 3: fetch_by(path, query, by, token) 2: get_report("data/ga", query, token, fetch.by) 1: get_ga(profileId = "ga:94268785", start.date = "2015-03-01", end.date = "2015-03-31", metrics = c("ga:sessions", "ga:pageviews", "ga:pageLoadTime", "ga:timeonPage", "ga:avgTimeOnPage", "ga:bounces", "ga:bouncerate"), dimensions = c("ga:date", "ga:dimension13", "ga:dimension14"), sort = "-ga:pageviews", filters = "ga:dimension7==local", segment = NULL, samplingLevel = NULL, start.index = 1, max.results = 1, include.empty.rows = NULL, fetch.by = "day", ga_token)
Could the issue be that RGA is not compatible with the newer versions of R such as 3.3.0 There were some issues with other packages and the interface. Has anyone tried rolling back to a previous version of R such as 3.2.5?
I got this error today for the first time. Been using RGA since more than a year. didn't change date format or the date range, neither did I update any package. Please confirm why this happens?
Hello? This keeps happening from time to time. Is this package being maintained?
Here's the code I have and the error happens from time to time: @artemklevtsov
# Load up the RGA package. This is the package that has the smarts to actually
# connect to and pull data from the Google Analytics API
Sys.setenv(HADOOP_CMD="/usr/bin/hadoop")
library(RGA)
library(rhdfs)
#Authentication
#options(httr_oob_default=TRUE)
client.id <- 'CLIENTID'
#Insert the client secret retrieved from the Google developer #console
client.secret <- 'client secret'
viewIds <- c( "ga:62814012","ga:124135466", "ga:132796646", "ga:131158887");
yesterdayDate <- Sys.Date() -1;
yesterdayDate <- as.character.Date(yesterdayDate)
print("YESTERDAY DATE")
print(yesterdayDate)
fileNames <- c();
print("going to authorize")
token <- authorize(client.id = "962653212381-l7bpufol4vbpvtruvh6sst84hru4on27.apps.googleusercontent.com", client.secret ="h6XYPURGo331v_NKmC16yU42", cache = "/home/rstudio/gaToken")
print("token in ga")
print(token)
maxDate <-"2017-07-01";
getLastImport = function(){
if(file.exists("/home/rstudio/maxDateGAConsolidated")){
mDate <- read.table("/home/rstudio/maxDateGAConsolidated", header=FALSE)
print(mDate$V1)
maxDate <<- mDate$V1
#maxDate <<- as.Date(maxDate) + 1;
print("max date is")
print(maxDate)
print("yesterday date is")
print(yesterdayDate)
if(maxDate >= as.Date(yesterdayDate)){
maxDate <<- yesterdayDate
stop(simpleError("Data has already been updated. You were trying to duplicate the data"))
}else{
maxDate <<- as.Date(maxDate) + 1;
return(maxDate)
}
}else{
print("max date dpes nt exist")
maxDate <<- as.Date(maxDate);
print("max date after increment")
print(maxDate)
return(maxDate)
}
}
getLastImport()
print(maxDate)
fetchData <- function(id) {
print("fetching data")
print("id")
print(id)
print("max date in fetch data")
print(maxDate)
assign(paste0("gaDataUser", substring(id, 4)), get_ga(profileId = id, start.date = maxDate ,end.date = "yesterday" , metrics = c("ga:transactions", "ga:users", "ga:newUsers","ga:percentNewSessions", "ga:sessionsPerUser"),dimensions = c("ga:date"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
print("firstfetch")
assign(paste0("gaDataSession", substring(id, 4)) , get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:sessions", "ga:bounces", "ga:bounceRate", "ga:sessionDuration", "ga:avgSessionDuration", "ga:hits"),dimensions = c("ga:date"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by = "day", token), envir = .GlobalEnv)
assign(paste0("gaDataSources", substring(id, 4)) , get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:transactions", "ga:organicSearches"),dimensions = c("ga:transactionId", "ga:fullReferrer", "ga:campaign", "ga:sourceMedium", "ga:keyword"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("gaDataEventTracking", substring(id, 4)) , get_ga(profileId =id, start.date = maxDate,end.date = "yesterday" , metrics = c( "ga:totalEvents", "ga:uniqueEvents", "ga:eventValue", "ga:avgEventValue", "ga:sessionsWithEvent", "ga:eventsPerSessionWithEvent"),dimensions = c("ga:eventCategory", "ga:eventAction", "ga:eventLabel", "ga:date"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by = "day", token), envir = .GlobalEnv)
assign(paste0("gaDataEcommerce",substring(id, 4)) , get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:transactions", "ga:transactionRevenue", "ga:revenuePerTransaction", "ga:transactionRevenuePerSession", "ga:transactionShipping", "ga:transactionTax", "ga:totalValue", "ga:itemQuantity"),dimensions = c("ga:transactionId", "ga:date"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by = "day", token), envir = .GlobalEnv)
assign(paste0("gaDataEcommerce2", substring(id, 4)) ,get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:transactions","ga:itemRevenue", "ga:productAddsToCart", "ga:productCheckouts", "ga:productListClicks", "ga:productListViews", "ga:quantityAddedToCart"),dimensions = c("ga:transactionId", "ga:date"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by = "day", token), envir = .GlobalEnv)
assign(paste0("gaDataEcommerce3", substring(id, 4)) ,get_ga(profileId = id, start.date =maxDate,end.date = "yesterday" , metrics = c("ga:quantityCheckedOut", "ga:productListCTR", "ga:productRevenuePerPurchase"),dimensions = c("ga:transactionId","ga:date"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by = "day", token), envir = .GlobalEnv)
assign(paste0("gaDataEcommerce4", substring(id, 4)) ,get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c( "ga:buyToDetailRate", "ga:cartToDetailRate", "ga:internalPromotionCTR", "ga:internalPromotionClicks", "ga:internalPromotionViews", "ga:uniquePurchases"),dimensions = c("ga:transactionId", "ga:date" ), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by = "day", token), envir = .GlobalEnv)
assign(paste0("gaDeviceData", substring(id, 4)) ,get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:transactions"),dimensions = c("ga:transactionId", "ga:browser", "ga:operatingSystem"," ga:mobileDeviceBranding", "ga:mobileDeviceModel", "ga:mobileDeviceInfo", "ga:deviceCategory"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("usersAndSessionsGrouped", substring(id, 4)) ,get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:users", "ga:sessions"),dimensions = c("ga:date", "ga:sourceMedium"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("pageViewsGroupedByPath", substring(id, 4)) ,get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:entrances","ga:entranceRate","ga:pageviews","ga:pageviewsPerSession", "ga:uniquePageviews", "ga:timeOnPage", "ga:avgTimeOnPage", "ga:exitRate"),dimensions = c("ga:pageTitle", "ga:date"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("adwordsCost", substring(id, 4)) ,get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:adCost"),dimensions = c("ga:date", "ga:deviceCategory"), sort = NULL, filters = "ga:sourceMedium==google / cpc;ga:campaign!@android;ga:campaign!@IOS;ga:campaign!@APPENG;ga:campaign!@UAC_ALL",segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("dataSourcesWithRevenue", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:transactions", "ga:organicSearches", "ga:transactionRevenue"),dimensions = c("ga:transactionId", "ga:fullReferrer", "ga:campaign", "ga:sourceMedium"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("landingpage", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:pageviews", "ga:uniquePageviews", "ga:transactions", "ga:transactionRevenue"),dimensions = c("ga:landingPagePath", "ga:sourceMedium", "ga:date"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("campaignWithDevice", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:transactions", "ga:transactionRevenue","ga:organicSearches"),dimensions = c("ga:transactionId", "ga:orderCouponCode", "ga:fullReferrer", "ga:campaign", "ga:sourceMedium", "ga:deviceCategory"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("versioning_mobile", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:users"),dimensions = c("ga:date", "ga:appVersion", "ga:screenName"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("versioning_withoutScreenNamemobile", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:users"),dimensions = c("ga:date", "ga:appVersion"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("mobile_campaign", substring(id, 4)), get_ga(profileId = id, start.date = maxDate ,end.date = "yesterday" , metrics = c("ga:users", "ga:newUsers","ga:sessions", "ga:avgSessionDuration", "ga:transactionRevenue"),dimensions = c("ga:date","ga:campaign","ga:sourceMedium", "ga:transactionId", "ga:deviceCategory"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("mobile_campaign_noTransactionId", substring(id, 4)), get_ga(profileId = id, start.date = maxDate ,end.date = "yesterday" , metrics = c( "ga:users", "ga:newUsers","ga:sessions", "ga:avgSessionDuration", "ga:transactionRevenue"),dimensions = c("ga:date","ga:campaign", "ga:sourceMedium", "ga:deviceCategory"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("hourlyScreenViews", substring(id, 4)), get_ga(profileId = id, start.date = maxDate ,end.date = "yesterday" , metrics = c( "ga:users"),dimensions = c("ga:date","ga:hour", "ga:screenName"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("UsersByHour", substring(id, 4)), get_ga(profileId = id, start.date = maxDate ,end.date = "yesterday" , metrics = c( "ga:users"),dimensions = c("ga:date","ga:hour"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("PDPByHour", substring(id, 4)), get_ga(profileId = id, start.date = maxDate ,end.date = "yesterday" , metrics = c("ga:users"),dimensions = c("ga:date","ga:hour","ga:screenName"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("impressions", substring(id, 4)), get_ga(profileId = id, start.date = maxDate ,end.date = "yesterday", metrics = c( "ga:productListViews","ga:productListClicks","ga:productListCTR","ga:productAddsToCart","ga:productCheckouts","ga:uniquePurchases"),dimensions = c("ga:productSku", "ga:date"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("pageBySourceMedium", substring(id, 4)), get_ga(profileId = id, start.date = maxDate ,end.date = yesterdayDate, metrics = c( "ga:uniquePageviews", "ga:avgTimeOnPage", "ga:pageviews","ga:entrances","ga:bounceRate","ga:exitRate","ga:pageValue"),dimensions = c( "ga:date", "ga:pagePath", "ga:sourceMedium"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("fastDispatch", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:bounceRate","ga:entrances", "ga:pageviews", "ga:uniquePageviews", "ga:avgTimeOnPage", "ga:exitRate"),dimensions = c("ga:date", "ga:pagePath", "ga:shoppingStage"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
if(id == "ga:124135466" | id == "ga:112973340"){
print("in if")
assign(paste0("visitsDataBySourceMediumAndDevice", substring(id, 4)) ,get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:users", "ga:sessions", "ga:newUsers", "ga:bounces"),dimensions = c("ga:date", "ga:sourceMedium", "ga:operatingSystem"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
}else{
print('in else')
assign(paste0("visitsDataBySourceMediumAndDevice", substring(id, 4)) ,get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:users", "ga:sessions", "ga:newUsers", "ga:bounces"),dimensions = c("ga:date", "ga:sourceMedium", "ga:deviceCategory"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
}
if(id == "ga:124135466" | id == "ga:132796646" | id == "ga:112973340"){
assign(paste0("appTracking", substring(id, 4)) ,get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:screenviews", "ga:uniqueScreenviews", "ga:screenviewsPerSession", "ga:timeOnScreen", "ga:avgScreenviewDuration"),dimensions = c("ga:screenName", "ga:date", "ga:operatingSystem"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("gaDataCampaignReportGroupDateCampaign", substring(id, 4)), get_ga(profileId = id, start.date = maxDate ,end.date = "yesterday" , metrics = c("ga:newUsers", "ga:users"),dimensions = c( "ga:date"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("gaDataCampaignReport", substring(id, 4)), get_ga(profileId = id, start.date = maxDate ,end.date = "yesterday" , metrics = c("ga:sessions","ga:newUsers","ga:percentNewSessions","ga:bounceRate","ga:avgSessionDuration","ga:pageviewsPerSession","ga:transactions","ga:transactionRevenue"),dimensions = c("ga:date"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("pageViewsGroupedByPathSourceMedium", substring(id, 4)) ,get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:screenviews", "ga:uniqueScreenviews", "ga:screenviewsPerSession", "ga:timeOnScreen", "ga:avgScreenviewDuration"),dimensions = c("ga:screenName", "ga:date", "ga:operatingSystem"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
# assign(paste0("versioning_mobile", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:users"),dimensions = c("ga:date", "ga:appVersion", "ga:screenName"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
# assign(paste0("versioning_withoutScreenNamemobile", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:users"),dimensions = c("ga:date", "ga:appVersion"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
}else{
assign(paste0("appTracking", substring(id, 4)) ,get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:entrances","ga:entranceRate","ga:pageviews","ga:pageviewsPerSession", "ga:uniquePageviews", "ga:timeOnPage", "ga:avgTimeOnPage", "ga:exitRate"),dimensions = c("ga:pagePath", "ga:date", "ga:deviceCategory"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("gaDataCampaignReportGroupDateCampaign", substring(id, 4)), get_ga(profileId = id, start.date = maxDate ,end.date = "yesterday" , metrics = c("ga:sessions","ga:newUsers", "ga:users"),dimensions = c("ga:campaign", "ga:date"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("gaDataCampaignReport", substring(id, 4)), get_ga(profileId = id, start.date = maxDate ,end.date = "yesterday" , metrics = c("ga:sessions","ga:newUsers","ga:percentNewSessions","ga:bounceRate","ga:avgSessionDuration","ga:pageviewsPerSession","ga:transactions","ga:transactionRevenue"),dimensions = c("ga:transactionId","ga:campaign","ga:sourceMedium"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("pageViewsGroupedByPathSourceMedium", substring(id, 4)) ,get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:entrances","ga:entranceRate","ga:pageviews","ga:pageviewsPerSession", "ga:uniquePageviews", "ga:timeOnPage", "ga:avgTimeOnPage", "ga:exitRate"),dimensions = c("ga:pagePath", "ga:date", "ga:deviceCategory", "ga:sourceMedium"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
}
assign(paste0("banner_ecommerce", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c( "ga:searchUniques", "ga:transactionRevenue", "ga:transactions", "ga:revenuePerTransaction", "ga:goalValueAllPerSearch"),dimensions = c("ga:date", "ga:searchKeyword"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("banner_ecommerce_devicecategory", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c( "ga:searchUniques", "ga:transactionRevenue", "ga:transactions", "ga:revenuePerTransaction", "ga:goalValueAllPerSearch"),dimensions = c("ga:date", "ga:searchKeyword", "ga:deviceCategory"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("campaign_sourcemedium", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:sessions", "ga:percentNewSessions", "ga:newUsers", "ga:bounceRate", "ga:pageviewsPerSession", "ga:avgSessionDuration", "ga:transactionsPerSession", "ga:transactions"),dimensions = c("ga:date", "ga:campaign", "ga:sourceMedium"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
#assign(paste0("internalSearch", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:sessions", "ga:percentNewSessions", "ga:newUsers", "ga:bounceRate", "ga:pageviewsPerSession", "ga:avgSessionDuration", "ga:transactionsPerSession", "ga:transactions"),dimensions = c("ga:date", "ga:searchKeyword"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("sitecontent_allpages", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:pageviews", "ga:uniquePageviews", "ga:avgTimeOnPage", "ga:entrances", "ga:bounceRate", "ga:exitRate","ga:pageValue"),dimensions = c("ga:date", "ga:pagePath"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("sitecontent_allpages_device_category", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:pageviews", "ga:uniquePageviews", "ga:avgTimeOnPage", "ga:entrances", "ga:bounceRate", "ga:exitRate","ga:pageValue"),dimensions = c("ga:date", "ga:pagePath", "ga:deviceCategory"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("top_events", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:totalEvents", "ga:uniqueEvents", "ga:eventValue", "ga:avgEventValue"),dimensions = c("ga:date","ga:eventCategory","ga:eventAction", "ga:eventLabel"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("top_eventsByDeviceCategory", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:totalEvents", "ga:uniqueEvents", "ga:eventValue", "ga:avgEventValue"),dimensions = c("ga:date","ga:eventCategory","ga:eventAction", "ga:eventLabel","ga:deviceCategory"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
assign(paste0("color_pdp_desktop_mobile", substring(id, 4)), get_ga(profileId = id, start.date = maxDate,end.date = "yesterday" , metrics = c("ga:totalEvents", "ga:uniqueEvents", "ga:eventValue", "ga:avgEventValue"),dimensions = c("ga:date","ga:eventAction", "ga:eventLabel", "ga:eventCategory"), sort = NULL, filters = NULL,segment = NULL, samplingLevel = NULL, start.index = NULL,max.results = NULL, include.empty.rows = NULL, fetch.by ="day", token), envir = .GlobalEnv)
fileNames <<-c(fileNames, c(paste0("gaDataUser", substring(id, 4)) ,paste0("gaDataSession", substring(id, 4)), paste0("gaDataSources", substring(id, 4)), paste0("gaDataEventTracking", substring(id, 4)), paste0("gaDataEcommerce", substring(id, 4)), paste0("gaDataEcommerce2", substring(id, 4)), paste0("gaDataEcommerce3", substring(id, 4)), paste0("gaDataEcommerce4", substring(id, 4)), paste0("gaDeviceData", substring(id, 4)), paste0("usersAndSessionsGrouped", substring(id, 4)), paste0("pageViewsGroupedByPath", substring(id, 4)), c(paste0("adwordsCost", substring(id, 4))),
paste0("visitsDataBySourceMediumAndDevice", substring(id, 4)), paste0("appTracking", substring(id, 4)),
paste0("gaDataCampaignReportGroupDateCampaign", substring(id, 4)),
paste0("gaDataCampaignReport", substring(id, 4)),
paste0("dataSourcesWithRevenue", substring(id, 4)),
paste0("pageViewsGroupedByPathSourceMedium", substring(id, 4)),
paste0("landingpage", substring(id, 4)),
paste0("banner_ecommerce", substring(id, 4)),
paste0("campaign_sourcemedium", substring(id, 4)),
paste0("sitecontent_allpages", substring(id, 4)),
paste0("sitecontent_allpages_device_category", substring(id, 4)),
paste0("banner_ecommerce_devicecategory", substring(id, 4)),
paste0("top_events", substring(id, 4)),
paste0("campaignWithDevice", substring(id, 4)),
paste0("versioning_mobile", substring(id, 4)),
paste0("versioning_withoutScreenNamemobile", substring(id, 4)),
paste0("mobile_campaign", substring(id, 4)),
paste0("mobile_campaign_noTransactionId", substring(id, 4)),
paste0("hourlyScreenViews", substring(id, 4)),
paste0("UsersByHour", substring(id, 4)),
paste0("PDPByHour", substring(id, 4)),
paste0("impressions", substring(id, 4)),
paste0("pageBySourceMedium", substring(id, 4)),
paste0("fastDispatch", substring(id, 4)),
paste0("top_eventsByDeviceCategory", substring(id, 4)),
paste0("color_pdp_desktop_mobile", substring(id, 4))
));
}
tapply(viewIds, 1:length(viewIds),fetchData);
#print(testtest)
writeToHDFS <- function(fileName){
print(fileName)
print(get(fileName))
hdfs.init()
print("write hdfs")
if(!is.null(get(fileName)) && !is.na(get(fileName))){
if(file.exists(paste0("/home/rstudio/", fileName))){
write.table(get(fileName), paste0("/home/rstudio/", fileName) , sep="," , append=TRUE, na = "NA", row.names=FALSE, col.names = FALSE)
}else{
write.table(get(fileName), paste0("/home/rstudio/", fileName) , sep="," , append=TRUE, na = "NA", row.names=FALSE, col.names = TRUE)
}
hdfs.put(paste0("/home/rstudio/", fileName), "/user/rstudio/")
}
#hdfs.close(modelfile)
print("WRITIING")
}
tapply(fileNames, 1:length(fileNames),writeToHDFS)
write(yesterdayDate, "/home/rstudio/maxDateGAConsolidated", append = FALSE)
print("THE END")
print("#################################################################################")
We execute this code through a cron job and the cron executes a second time if the first attempt was not successful. The error happened both the times today. Then, executed the script manually a third time and it worked.We really need a solution to this
Looking through the traceback output provided above, I would suggest looking for the bug somewhere around here:
14: request_build("GET", hu$url, as.request(config), ...)
13: GET(url, config = config, accept_json())
12: api_request(get_url(path, query), token)
