[Docs] Document how caching can be controlled in the "Writing Custom Flyte Types" page

[fg91]

Description

The page Writing Custom Flyte Types explains how users can implement a type transformer for custom types.

This page should include a section that explains which ways exist to control the caching of instances of custom types.

Currently users need to have a lot of understanding of the code base to figure this out themselves.

In particular, the following options should be documented and explained, potentially with examples:

• Literal(hash=)
• LiteralType(schema=..., metadata=..., structure=...,
• Others?

Are you sure this issue hasn't been raised already?

• [X] Yes

Have you read the Code of Conduct?

• [X] Yes

[fg91]

class DataclassTransformer(TypeTransformer[object]):
    ...
    def get_literal_type(self, t: Type[T]) -> LiteralType:
        ...
        ts = TypeStructure(tag="", dataclass_type=literal_type)
        return _type_models.LiteralType(simple=_type_models.SimpleType.STRUCT, metadata=schema, structure=ts)

This snippet from the dataclass transformer for example suggests that adding metadata=schema to the LiteralType should handle cache misses in case of schema changes but it turns out that the metadata appears in fact to be not used to calculate the hash key here:

func generateTaskSignatureHash(ctx context.Context, taskInterface core.TypedInterface) (string, error) {

	…
    	if taskInterface.Outputs != nil && len(taskInterface.Outputs.Variables) != 0 {
		taskOutputs = taskInterface.Outputs
	}
	outputHash, err := pbhash.ComputeHash(ctx, taskOutputs)

Instead, it's the structure that does this. Things like this should be documented.

[luckyarthur]

#take

[fg91]

#take

Thank you, feel free to tag me as reviewer!

[github-actions[bot]]

"Hello 👋, this issue has been inactive for over 90 days. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! 🙏"

[github-actions[bot]]

Hello 👋, this issue has been inactive for over 90 days and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! 🙏