Getting StackOverflowError when running runBootstrap.jl

[smeyn]

I get this error here. Probably I missed something simple: Steps to reproduce:

• clone repo
• activate environment
• work in test dir
• include("runbootstrap.jl")

---

[ Info: ===> sorting:TableDataList refs:2 [ Info: ("try sort", "Model") ERROR: LoadError: StackOverflowError: Stacktrace: [1] topologicalSort(::Main.Bootstrap.Rest.Schema, ::Array{Main.Bootstrap.Rest.Schema,1}) at D:\steph\git\GCP.jl\src\bootstrap.jl:304 [2] topologicalSort(::Main.Bootstrap.Rest.Schema, ::Array{Main.Bootstrap.Rest.Schema,1}) at D:\steph\git\GCP.jl\src\bootstrap.jl:307 (repeats 18603 times) [3] convert(::Type{Expr}, ::Main.Bootstrap.Rest.Service) at D:\steph\git\GCP.jl\src\bootstrap.jl:807 [4] emitService(::Main.Bootstrap.Rest.Service, ::String) at D:\steph\git\GCP.jl\src\bootstrap.jl:920 [5] emitGcp(::Main.Bootstrap.Rest.Service) at D:\steph\git\GCP.jl\src\bootstrap.jl:331

[rana]

Hi,

You probably just want to consume GCP.jl. In which case, you would need to generate a credential file from Google and do something like the following in Julia

At the Julia command prompt add the GCP package

(@v1.4) pkg> add GCP
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
.
.
.

Then in your Julia file, reference the GCP package.

using GCP
using GCP.Bigquery

CredentialFilename = expanduser("~/secrets/your-project.json")

p = GCP.Project(CredentialFilename, ["cloud-platform"])
dataset = Bigquery.Dataset()
dataset.datasetReference = Bigquery.DatasetReference()
dataset.datasetReference.datasetId = "your-dataset"
res = p.bigquery.datasets.insert(dataset)

@info res

The steps you have outlined above are more for generating GCP API SDKs from definition files which is probably not what you are intending.

Hope that helps.

[smeyn]

Actually generating other SDK was what I was looking for. I wanted to add a Datastore SDK.

On Tue, 12 May 2020 at 12:19 pm, Rana Ian [email protected] wrote:

Hi,

You probably just want to consume GCP.jl. In which case, you would need to generate a GCP credential file from Google and do something like the following in Julia

At the Julia command prompt add the GCP package

(@v1.4) pkg> add GCP Updating registry at ~/.julia/registries/General Updating git-repo https://github.com/JuliaRegistries/General.git Resolving package versions......

Then in your Julia file, reference the GCP package.

using GCPusing GCP.Bigquery

CredentialFilename = expanduser("~/secrets/your-project.json")

p = GCP.Project(CredentialFilename, ["cloud-platform"]) dataset = Bigquery.Dataset() dataset.datasetReference = Bigquery.DatasetReference() dataset.datasetReference.datasetId = "your-dataset" res = p.bigquery.datasets.insert(dataset) @info res

The steps you have outlined are more for generating GCP SDKs from definition files which is probably not what you are intending.

Hope that helps.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rana/GCP.jl/issues/7#issuecomment-627067758, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXROPRHHOC32VV5G62XGJTRRCW43ANCNFSM4M6K5ZSA .

[smeyn]

Digging into it further it turns ouyt that the new API of bigquery introduces new data types where some of htem have circular references. The topological sort assumes a DAG, which is no longer given. I added a sentinel in to avoid this happening but this just kicks the can down the road. Julia does not like structs with circular cross references

[bjarthur]

i can reproduce. here's my stack trace:

julia> Emit.getEmit(:bigquery)
[ Info: emitGcp: start ---
[ Info: emitService: bigquery: start ---
[ Info: JobStatistics.reservationUsage NOTHING
[ Info: SetIamPolicyRequest.updateMask NOTHING
[ Info: TrainingOptions.labelClassWeights NOTHING
[ Info: ProjectList.projects NOTHING
[ Info: JobStatistics2.reservationUsage NOTHING
[ Info: QueryParameterType.structTypes NOTHING
[ Info: RangePartitioning.range NOTHING
[ Info: TableDataInsertAllRequest.rows NOTHING
[ Info: JobConfiguration.labels NOTHING
[ Info: ModelDefinition.modelOptions NOTHING
[ Info: TableDataInsertAllResponse.insertErrors NOTHING
[ Info: Model.labels NOTHING
[ Info: BqmlTrainingRun.trainingOptions NOTHING
[ Info: TableFieldSchema.categories NOTHING
[ Info: TableFieldSchema.policyTags NOTHING
[ Info: Table.labels NOTHING
[ Info: TableCell.v NOTHING
[ Info: QueryRequest.labels NOTHING
[ Info: JobConfigurationQuery.tableDefinitions NOTHING
[ Info: DestinationTableProperties.labels NOTHING
[ Info: Dataset.labels NOTHING
[ Info: Dataset.access NOTHING
[ Info: TrainingRun.startTime NOTHING
[ Info: QueryParameterValue.structValues NOTHING
ERROR: StackOverflowError:
Stacktrace:
 [1] topologicalSort(::Main.Bootstrap.Rest.Schema, ::Array{Main.Bootstrap.Rest.Schema,1}) at /groups/scicompsoft/home/arthurb/.julia/packages/GCP/qh4tD/src/bootstrap.jl:304
 [2] topologicalSort(::Main.Bootstrap.Rest.Schema, ::Array{Main.Bootstrap.Rest.Schema,1}) at /groups/scicompsoft/home/arthurb/.julia/packages/GCP/qh4tD/src/bootstrap.jl:307 (repeats 21637 times)
 [3] convert(::Type{Expr}, ::Main.Bootstrap.Rest.Service) at /groups/scicompsoft/home/arthurb/.julia/packages/GCP/qh4tD/src/bootstrap.jl:807
 [4] emitService(::Main.Bootstrap.Rest.Service, ::String) at /groups/scicompsoft/home/arthurb/.julia/packages/GCP/qh4tD/src/bootstrap.jl:919
 [5] emitGcp(::Main.Bootstrap.Rest.Service) at /groups/scicompsoft/home/arthurb/.julia/packages/GCP/qh4tD/src/bootstrap.jl:331
 [6] getEmit(::Symbol; kws::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /groups/scicompsoft/home/arthurb/.julia/packages/GCP/qh4tD/src/bootstrap.jl:321
 [7] getEmit(::Symbol) at /groups/scicompsoft/home/arthurb/.julia/packages/GCP/qh4tD/src/bootstrap.jl:317
 [8] top-level scope at REPL[3]:2