Projection without a root entity

[pelletier197]

trafficstars

Hello!

I don't know if it's a bug or if it's wished this way, but I find the issue to be a little problematic. My example requires to return the results from a custom query into a DTO projection, but without a root entity in the projection. Here is an example

Given this custom cypher query

        MATCH (department:Department)
        OPTIONAL MATCH (person:Person)-[:MEMBER_OF]->(department)
        RETURN department.id as departmentId, collect(DISTINCT person.id) as peopleIds 

And this projection DTO

data class PeopleInDepartmentProjection(
    val departmentId: String,
    val peopleIds: List<String>
)

I get the following error:

org.springframework.data.neo4j.core.mapping.NoRootNodeMappingException: Could not find mappable nodes or relationships inside Record<{departmentId: "b35541a7-fe96-4782-9d11-6ad42a6d83c8", peopleIds: []}> for org.springframework.data.neo4j.core.mapping.DefaultNeo4jPersistentEntity@baa6277
	at org.springframework.data.neo4j.core.mapping.DefaultNeo4jEntityConverter.read(DefaultNeo4jEntityConverter.java:103)
	at org.springframework.data.neo4j.core.mapping.DefaultNeo4jEntityConverter.read(DefaultNeo4jEntityConverter.java:66)
	at org.springframework.data.neo4j.core.mapping.Schema.lambda$getRequiredMappingFunctionFor$0(Schema.java:96)
	at 

I'm wondering if it's wished that you absolutely need a root object to project the results of a query?

FYI, I have this workaround for now

        MATCH (department:Department)
        OPTIONAL MATCH (person:Person)-[:MEMBER_OF]->(department)
        RETURN department, collect(DISTINCT person.id) as peopleIds 

and I updated the DTO accordingly

[michael-simons]

The behavior is a bit different than the old @QueryResult: The query result was basically an arbitrary record. The projection is based on a concrete root object, it's properties are projected.

I case the domain object for Department has a field departmentId, right? In this case above it's obvious to that departmendId should match the property of it… But: We need to figure out the base entity, and we use the root object for that.

I'll see whether there's a low hanging fruit to improve this.

[pelletier197]

Actually, department has a field id. But I want to name it departmentId in the result, since more than one entity have the field id (person has an id too).

[michael-simons]

In that case, I would consider using the Neo4jClient and provide a custom mapping function tbh.

See: https://docs.spring.io/spring-data/neo4j/docs/6.1.1/reference/html/#faq.custom-queries-and-custom-mappings 11.9.2

[pelletier197]

I see.. That's a bit unfortunate. If that's the only way to make this work, we'll do that. Is there a reason why projections enforce having a root object? Why wouldn't it be possible to map a query result to an arbitrary object? Cause it seems to me to be a pretty standard use-case.

A bit the same thing happens for parameters of queries. It used to be possible to pass objects as a parameter to an @Query function and those parameters where serialized as a MapValue recursively. Now, The framework only allow for primitive types.

[alxxyz]

@michael-simons we also have some use cases where @QueryResult fits better, it will be great if there will be something similar

[michael-simons]

Hey @alxxyz Please see my answer here: https://github.com/spring-projects/spring-data-neo4j/issues/2283#issuecomment-857523868

[alxxyz]

Thanks.

Collection<String> labels = neo4jClient.query("CALL db.indexes() YIELD name RETURN name")
					.fetchAs(String.class)
					.all();
labels.forEach(System.out::println);
			```

[pelletier197]

That kind of example with one field is simple, but we have much much more complex queries than that which makes it really inconvenient to use the neo4jClient directly. Here's a sample data class we map our query results to.

data class PersonFileIndicatorsResponse(
    val file: FileEntity,
    val classification: ClassificationEntity,
    val host: HostEntity,
    val indicators: List<PersonFileIndicatorResponse>,
    val rawScoreDelta: Double,
    val lastIndicatorTimestamp: Instant,
    val infoDelta: Long
)

data class PersonFileIndicatorResponse(
    val id: String,
    val timestamp: Instant,
    val reference: String,
    val name: String,
    val type: String,
    val event: String,
    val rawScore: Double,
    val infoDelta: Long,
    val qualifiers: List<Qualifier>,
    val patternGroupDetails: List<PersonFileIndicatorPatternGroupDetailsResponse>
)

data class PersonFileIndicatorPatternGroupDetailsResponse(
    val patternGroup: PersonFileIndicatorPatternGroup,
    val classification: ClassificationEntity,
    val infoDelta: Long,
    val rawScoreDelta: Double
)

data class PersonFileIndicatorPatternGroup(
    val id: String,
    val label: String,
)

You see, there's multiple level of nesting in there (3 actually). At some point, in our query, we had to collect using a Neo4j Map. Like this.

collect({
                    indicator: indicator,
                    eventQualifiers: eventQualifiers, 
                    infoDelta: infoDelta,
                    patternGroupDetails: patternGroupDetails
                }) as indicators

We basically just ended up converting all the fields recursively from the framework's internal objects to a Map<String, Any>. and then we used an object mapper to convert it to the data class above. It works, but it's inconvenient, and we had to do this for many queries in our applications.

I know this is really not standard, so I don't expect this to be fixed any time soon, but I feel like it would be more natural to the users to be able to map their query results directly to the corresponding object in their code :shrug:

[alxxyz]

I know this is really not standard, so I don't expect this to be fixed any time soon, but I feel like it would be more natural to the users to be able to map their query results directly to the corresponding object in their code Agree

@michael-simons what are the reasons to not support something similar to @QueryResult?

[michael-simons]

We give you the freedom and flexibility through the Neo4jClient to plug the mapping you want or need. A query result that arbitrary can chose and pick not only from one record but the whole subgraph returned has to many angles to. Also it is diametral opposed to the need to be record oriented when working in a reactive (streaming) fashion.

[alxxyz]

I expect it to work also with CrudRepository` as it is part of Spring Data, and to be able just to provide the query, and not to map the internal neo4j result in the application entity.

[michael-simons]

The thing is: Repositories in the DDD world are supposed to contain one specific thing and provide CRUD methods for that. That is what a Spring Data Repository does too. In addition, we can project the results, but based on the actual entity. So even considering this ticket would be a way away from that and we are not the biggest fans to do this.

We have seen enough projects where the repository is more a less a shallow client, just used to run queries and hope for the best. This is not our goal with SDN 6. If you need to run tons of arbitrary queries, do with the Neo4jClient and enjoy the integration with Spring transactions and use some mapping function catered to your needs.

Am 28.06.2021 um 14:25 schrieb alxxyz @.***>:

I expect it to work also with CrudRepository`, to be able just to provide the query, and not to map the internal neo4j result in the application entity.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/spring-projects/spring-data-neo4j/issues/2272#issuecomment-869640978, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEAQL2KUUXCHGK4BW7ARS3TVBS5PANCNFSM453MV5GQ.

[michael-simons]

Closing due to inactivity.

[thedeadferryman]

I'm experiencing a pretty similar problem now.

	@Query("""
		MATCH	(n:$DATASOURCE_LABEL:$ENTITY_LABEL)-[]-(x:$DATASOURCE_LABEL:$ENTITY_LABEL)
		WHERE	n.nodeId IN ${'$'}ids
				AND NOT (x.nodeId IN ${'$'}ids)
		RETURN	DISTINCT n.nodeId AS parentId, collect(x)
	""")

I could just go with Neo4jClient to execute this query, but the question is - do I need to map entities all by myself now? I mean, I do have a model for an entity of this type (x) and I want to see it converted to that type, but doing full manual mapping just to run a single query that is a little bit more complicated than 'get a list of nodes' seems quite inappropriate to me. Seriously, I just want to get a set of nodes grouped by some property, and I can't do that while still benefitting from automatic object mapping?

So the question is - if I can't do that with repositories and I can't do that with Neo4jTemplate (which seems to be really weird to me as we don't have any constraints like 'repository should return the one thing' for it). But can I at least somehow access the object mapping facilities that are used to map results in repositories and use it to map results I get from the Neo4jClient?

[meistermeier]

Yes, this is possible. Map the parentId by hand to the very object and leave the collection (to be more precise the elements within the collection) to the mapping function for this domain object. https://docs.spring.io/spring-data/neo4j/docs/current/reference/html/#neo4j-client.result-objects.mapping-functions Autowire/Inject the Neo4jMappingContext into the class that uses the Neo4jClient and you can do:

BiFunction<TypeSystem, MapAccessor, ExpectedClass> mappingFunction = neo4jMappingContext.getRequiredMappingFunctionFor(ExpectedClass.class);
neo4jClient.query("<yourQuery>")
  .bind(<your parameters>)
  .fetchAs(ExpectedClass.class)
  .mappedBy((TypeSystem t, Record record) -> {
    var parentId = record.get("parentId").asLong();
    var innerObjects = record.get("collect(x)").asList(innerObject -> mappingFunction.apply(t, innerObject));
    return new ExpectedClass(parentId, innerObjects);
})

please be aware that labels cannot be replaced with parameters in Neo4j.

[thedeadferryman]

Thank you for the answer, it worked perfect for me. Side note about labels - these are not Neo4j query parameters, but rather the interpolation. I'm using Kotlin which supports string interpolation and I decided to replace the labels with constants since they're used in many places (and not only in query annotations). This is also the reason why I need to have that weird ${'$'} escaping for actual neo4j parameters.