"Given typename StatisticsEntry was invalid" when creating types

[hashhar]

Steps:

1. Install Apache Atlas release-2.1.0-rc1 by building from source (not sure if it matters).
2. Run the Atlas quick start to create some sample data and types as bin/quick_start.py
3. Install amundsen-atlas-types from source (python3 setup.py install) and run the following from a Python shell:

from amundsenatlastypes import Initializer
  
init = Initializer()
init.create_required_entities(fix_existing_data=True)

4. This logs the following at the start but then is able to create other entities without any error.

Trying to create Column Entity                                            
Already Exists, updating Column Entity
Something wrong happened: HTTP request failed for PUT http://localhost:21000/api/atlas/v2/types/typedefs: Not found 404: {"errorCode":"ATLAS-404-
00-001","errorMessage":"Given typename StatisticsEntry was invalid"}       
Applied Column Entity Definition

Now trying to get the type shows:

$ curl -X GET http://localhost:21000/api/atlas/v2/types/structdef/name/StatisticsEntry -u admin:admin
{"errorCode":"ATLAS-404-00-001","errorMessage":"Given typename StatisticsEntry was invalid"}

Is there something I'm doing wrong or can someone else reproduce this?

[mgorsk1]

@hashhar thanks for filing this one. I am quite certain that 1.1.0 version of amundsenatlastypes doesn't have StatisticsEntry entity - are you sure you are using latest code ?

second reason why it might be following is incosistencies between Atlas sample data and our entities. can you confirm it doesn't work even if you omit step 2 (loading sample data) ?

[hashhar]

On master I can indeed see StatisticsEntry in structDefs at https://github.com/dwarszawski/amundsen-atlas-types/blob/master/amundsenatlastypes/schema/01_column_schema.json.

It works without error if I skip loading sample data. :+1:

[pPanda-beta]

I can confirm even without loading the sample data this problem exists.

'{"errorCode":"ATLAS-404-00-001","errorMessage":"Given typename StatisticsEntry was invalid"}'

And the logs from atlas

2020-07-08 08:47:04,778 WARN  - [pool-2-thread-2 - 070b577b-3d87-4020-863a-75d0ff57f8a5:] ~ No RelationshipDef defined between Table and Reader on attribute: Table.readers (AtlasTypeRegistry:258)
2020-07-08 08:47:04,789 ERROR - [pool-2-thread-2 - 070b577b-3d87-4020-863a-75d0ff57f8a5:] ~ graph rollback due to exception  (GraphTransactionInterceptor:167)
org.apache.atlas.exception.AtlasBaseException: Table.owner : attribute delete not supported
	at org.apache.atlas.repository.store.graph.v2.AtlasStructDefStoreV2.updateVertexPreUpdate(AtlasStructDefStoreV2.java:406)
	at org.apache.atlas.repository.store.graph.v2.AtlasEntityDefStoreV2.updateVertexPreUpdate(AtlasEntityDefStoreV2.java:336)
	at org.apache.atlas.repository.store.graph.v2.AtlasEntityDefStoreV2.updateByName(AtlasEntityDefStoreV2.java:212)
	at org.apache.atlas.repository.store.graph.v2.AtlasEntityDefStoreV2.update(AtlasEntityDefStoreV2.java:179)
	at org.apache.atlas.repository.store.graph.v2.AtlasEntityDefStoreV2.update(AtlasEntityDefStoreV2.java:44)
	at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.updateGraphStore(AtlasTypeDefGraphStore.java:1035)
	at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.updateTypesDef(AtlasTypeDefGraphStore.java:481)
	at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$$FastClassBySpringCGLIB$$5226c80b.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
	at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:82)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
	at org.apache.atlas.repository.store.graph.v2.AtlasTypeDefGraphStoreV2$$EnhancerBySpringCGLIB$$f4fc3ad8.updateTypesDef(<generated>)
	at org.apache.atlas.web.rest.TypesREST.updateAtlasTypeDefs(TypesREST.java:407)

Versions :

• Atlas : release-2.1.0-rc2
• Flavour: embedded berkeley-elasticsearch
• elasticsearch: 7.6.1
• BerkeleyJE : sleepycat je-18.3.12

[pPanda-beta]

Same happened while updating Table schema as well,

HTTP request failed for PUT http://172.21.255.199:21000/api/atlas/v2/types/typedefs: Bad request 400: {"errorCode":"ATLAS-400-00-00D","errorMessage":"Table.owner : attribute delete not supported"}

[dwarszawski]

@pPanda-beta where that Table.owner attribute comes from? Have you tried to run on the fresh instance of Atlas?

[pPanda-beta]

@dwarszawski - I use docker, every time I'm cleaning up the volumes its as fresh as a daisy.

May be this version is not supported by amundsen types : Atlas : release-2.1.0-rc2

[dwarszawski]

@pPanda-beta amundsen-atlas-types in version 1.1.0 was released before atlas-2.1.0-rc was created. Could you run it from master branch? Let me try to switch our instance to rc2 and try to reproduce this problem.