Clarification needed on pipelineRun.finished.outcome field

[IshaGupta18]

Hi CDEvents team 👋

While reviewing the spec for pipelineRun.finished, I noticed a couple of areas where additional clarity would be helpful:

1. Allowed values for outcome

• The spec text lists "success", "error", and "failure" as example values.
• However, the JSON schema appears to allow arbitrary strings, without enforcing a strict enum constraint.
• Could you clarify whether producers are required to use only "success", "error", or "failure", or if custom values are permitted?

2. Difference between "error" and "failure"

• The spec includes both outcome terms, but doesn’t clarify the intended distinction.
• For instance: is "error" intended for infrastructure or runtime issues (e.g. crashes, misconfigurations), whereas "failure" is used for legitimate pipeline execution failures (e.g. failed tests, build errors)?
• Additional guidance or example use cases would help ensure consistent interpretation and handling across toolchains.

Thanks a lot in advance and let me know should you need more context.

[davidB]

Hi @IshaGupta18 ,

Good question, the discussion about enum vs string is open (also discussed at #215 ), for version 0.4 & 0.5 (coming soon), it is a string and "success",... are provided as examples. So technically, any value is allowed. We plan to clarify this point and a few others after the release of 0.5. Your contribution and opinion are welcome.

About error vs failure, TBH I don't know, I guess it's like for test:

• error is when the "issue" is on the execution/running environment itself (no runner, misconfiguration, syntax error),
• failure is when the content/logic failed,

Sorry for the delay,

[xibz]

@IshaGupta18 - So before we get started I want to give a quick overview on the two types of enums. Generally when languages and people use the word enum, they mean a "closed" enum. What does that mean? It means any defined enum values can only exist in its set. Meaning if you have the enum definition, Blue, Red, Green, as the Color enum. You cannot use Color::Purple or cast (Color) "Purple" as an example.

Additionally there's another enum, which you may have guessed, called an "open" enum. An open enum let's you have enumerated types outside it's specified enum values. Meaning it's set is bigger than what is defined. An example in type script could force their closed enum to be a more open enum by defining it like this

type Color = "Blue" | "Red" | "Green" | string;

This opens the set to any string.

Here's a paper on a language design called Hylo that incorporates both closed and open enum in its language for more context: https://repository.tudelft.nl/record/uuid:653ad5bd-c418-4d8e-850b-b624d8c3f6ec

So with all that aside, when we say enum, what do we mean? Well, in this specific case it's an open enum. We need to be clear where we use open enum versus closed, because I know a lot of folks think enums as only being closed.

I think the long term solution if we wanted an open enum is just to use a string and generate types as constants. However, we will see. Still an ongoing discussion.

And as @davidB had stated, error is when something failed outside of the test.