Tracking issue for Hydra problems

[ondratra]

Overview

Here we are attempting to keep an ongoing list of problems currently identified in Hydra, including how those problems impact other Atlas products.

:x: means product is not affected ✅ means product is affected.

Bugs

• [x] https://github.com/Joystream/hydra/issues/441
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:
• [x] https://github.com/Joystream/hydra/issues/399
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:
• [x] https://github.com/Joystream/hydra/issues/434
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:
• [ ] https://github.com/Joystream/hydra/issues/359 (related to https://github.com/Joystream/hydra/issues/438) maybe it's already fixed (at least partially)
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:
• [ ] https://github.com/Joystream/hydra/issues/362 + https://github.com/Joystream/hydra/issues/283 closely related
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:
• [x] https://github.com/Joystream/hydra/issues/420
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:
• [x] https://github.com/Joystream/hydra/issues/452 - inspect and confirm it's an issue and that it's concerning Olympia
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:
• [ ] https://github.com/Joystream/hydra/issues/455
  • Pioneer: :x:
  • Atlas: ✅
  • Colossus: :x:
  • Argus: :x:
• [ ] https://github.com/Joystream/hydra/issues/454
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:

Enhancements

(ordered by priority)

• [ ] https://github.com/Joystream/hydra/issues/444 + https://github.com/Joystream/hydra/issues/448 - partially done
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:
• [ ] https://github.com/Joystream/hydra/issues/443 - possibly done already, but needs a confirmation
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:
• [ ] https://github.com/Joystream/hydra/issues/396
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:
• [ ] https://github.com/Joystream/hydra/issues/435 - likely hard to fix
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:

Nice2have

• [ ] https://github.com/Joystream/hydra/issues/370
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:
• [ ] https://github.com/Joystream/hydra/issues/372
  • Pioneer: :x:
  • Atlas: :x:
  • Colossus: :x:
  • Argus: :x:

[kdembler]

Af far as I can tell only issue from this list that affects Atlas is https://github.com/Joystream/hydra/issues/455 - like mentioned previously, we had to implement couple workarounds and restrict the user functionality since filtering search results is really problematic. Other issues that Atlas team have noticed can be found here - https://github.com/Joystream/joystream/issues/2818

[Lezek123]

https://github.com/Joystream/hydra/issues/399:

• I don't think it does affect any of the projects directly, but it does affect the Giza/Olympia query node in general. We don't explicitly set ids for all entities, for example, in Giza: NodeLocationMetadata, StorageBucketOperatorMetadata, DistributionBucketFamilyMetadata (this can be easily fixed though)

https://github.com/Joystream/hydra/issues/434:

• Currently there is no Asset union anymore, but this issue has caused another DataObjectTypeUnknown workaround to be implemented in Giza mappings.

https://github.com/Joystream/hydra/issues/359:

• This was solved by flattening the Event interface to not include any relations, which also solved another issue, w.r.t. limited filtering depth (as it was not possible to filter by as deeply nested property as, for example, member->createdInEvent->inBlock->blockNumber anyway), so I don't think it's a blocker in any way now.

https://github.com/Joystream/hydra/issues/438:

• I think it still affects Pioneer, they currently workaround it by executing a separate query for each event type and then combining and sorting the results client-side, which is quite sub-optimal

https://github.com/Joystream/hydra/issues/362:

• Neither Giza nor Olympia mappings are using cascade for any operations currently, so it seems like the issue does not affect any project now.

https://github.com/Joystream/hydra/issues/283:

• I don't think that we still want to remove createdAt and updatedAt now that it's going to be set to the current blocktime, as implied by https://github.com/Joystream/hydra/issues/400. I think fields like createdById, updatedById, deletedById, deletedAt can be removed, however. (Although this is not blocking any progress currently)
• Edit: After testing https://github.com/Joystream/hydra/issues/400 in Hydra 3.1.0-alpha.13 I noticed that updatedAt (as opposed to createdAt) is not actually beeing correctly set to blocktime, so it probably should be fixed.

https://github.com/Joystream/hydra/issues/420:

• Giza: The issue does not affect the mappings, because there is no case of BigInt beeing inside a variant.
• Olympia: A workaround is currently in place

https://github.com/Joystream/hydra/issues/452:

• I don't remember ever running into this issue

https://github.com/Joystream/hydra/issues/455:

• Does not affect Argus/Colossus

https://github.com/Joystream/hydra/issues/454:

• The issue is sidestepped in Giza with "intermediate" entities like StorageBagDistributionAssignment and StorageBagStorageAssignment, but this workaround makes mapping and queries much more complex. Would be great if it was solved (I'll try to investigate it more)

https://github.com/Joystream/hydra/issues/444 (the deep filtering part):

• This is a nice to have fot both Giza (Argus, Colossus) and Olympia, currently we workaround this by either introducing additional client-side filtering or executing an additional query/queries.

https://github.com/Joystream/hydra/issues/448:

• Appears to be fixed in 3.1.0-alpha.13

https://github.com/Joystream/hydra/issues/443:

• This was fixed in Olympia by changing relationship field in ProposalDiscussionPost entity from thread to discussionThread (https://github.com/Joystream/joystream/blob/olympia/query-node/schemas/proposalDiscussion.graphql#L65), it's not an urgent issue, but can cause a lot of confusion if someone doesn't know about it.

https://github.com/Joystream/hydra/issues/396:

• I don't think we encountered this issue in Giza, but I had to comment-out some assertions in Olympia integration tests due to this issue and I think it's also causing problems in Pioneer

https://github.com/Joystream/hydra/issues/435:

• This is just an inconvinience / confusing behaviour, but not a blocker of any sort

What I would add to the list:

I noticed that in hydra 3.1.0-alpha.13 there is still an issue with sanitizeNullCharacter, which breaks the processor unless commented-out in node_modules/@joystream/hydra-processor/lib/db/subscribers.js:

 Cannot read property 'storage_bag_id' of undefined, stack: TypeError: Cannot read property 'storage_bag_id' of undefined
processor-mnt_1          |     at sanitizeNullCharacter (/joystream/node_modules/@joystream/hydra-processor/lib/db/subscribers.js:40:16)
processor-mnt_1          |     at sanitizeEntity (/joystream/node_modules/@joystream/hydra-processor/lib/db/subscribers.js:33:9)
processor-mnt_1          |     at SanitizationSubscriber.beforeInsert (/joystream/node_modules/@joystream/hydra-processor/lib/db/subscribers.js:11:9)

[jodator]

AFAICS the issues that were causing us troubles were: #444 & #396. However, as for now, we use a different query to get the deeply nested variants (#396) and I think that #444 isn't that present now (2/3 issues were fixed). At the moment we don't see obvious blockers.