Ingest FpML to CDM

[hugohills-regnosys]

Background

Rune DSL provides functionality to import xml schemas as a Rune model, while also providing functionality to (de)serialise xml documents to, and from, the imported model. The types and attributes of the imported FpML model can then be mapped to another model, such as CDM, using simple Rune syntax. The process of translating from an external data format, such as FpML xml, to another Rune model, such as CDM, is known as ingestion.

A similar approach has been successfully implemented on the ISO-20022 project, where the ISO-20022 xml schemas, from jurisdictions such as ESMA and JFSA, have been imported and mapping functions written to translate between the DRR model and the imported ISO-20022 models. The ISO-20022 type is then serialised into xml. The process of translating from Rune model types to an external data format, such as ISO-20022 xml, is known as projection.

Translate Open Sourcing.pdf

Proposal

Rune FpML model

The FpML Confirmation and Record-Keeping schemas (.xsd) to be imported into a Rune FpML model, where each schema type has a corresponding Rune data type.

• e.g. FpML dataDocument would be imported into the model as type DataDocument
• Create Rune FpML model by importing FpML schemas
• Add Rune DSL import features to support all FpML schema features
• Add Rune DSL features to support serialising xml to and from imported Rune FpML data types, e.g. FpML dataDocument xml message to be deserialised into the Rune DataDocument type, and then serialised back into an FpML dataDocument xml message
• Add Rune DSL syntax features to support both FpML Confirmation and Record-Keeping schemas with minimal duplication of mapping functions

Ingest functions

Ingest functions to be written to map the attributes from a Rune FpML type to a CDM type.

• e.g. ingest function FpmlDataDocumentToTradeState would be written to map the attributes from Rune FpML type DataDocument to CDM type TradeState
• Create Ingest PoC model with parent models CDM and Rune FpML that contains FpML to CDM ingestion functions
• Add functions to support ingestion from FpML message types to both CDM types TradeState and WorkflowStep
• Add Rune DSL features to allow reading and writing of meta data to support mapping of xml id/href and schemes

Diagnostics

Users should be able to view ingestion diagnostics, such as number of external data points and number of mapped data points.

• e.g. the number of data points on the Rune FpML type DataDocument, and the number of data points on the ingested CDM type TradeState

Rosetta support

Users should be able to access the Rune FpML and Ingest PoC models in Rosetta, write ingestion functions, run ingestion on all CDM FpML test pack samples, and view diagnostics.

Implementation and Scope

Phase 1 (2025 Q1 and Q2)

Rune FpML model:

• Create Rune FpML GitHub repository for the imported Rune FpML model
• Add Rune DSL import features to support all FpML schema features
• Add Rune DSL features to support serialising xml to and from imported Rune FpML data types

Ingest functions:

• Create Ingest PoC model with parent models CDM and Rune FpML
• Add functions to support ingestion from FpML message types to both CDM types TradeState and WorkflowStep
• Add Rune DSL features to allow reading and writing of meta data to support mapping of xml id/href and schemes
• Write functions to cover all samples in CDM FpML test packs

Diagnostics:

• Add Rune features to provide basic diagnostics / mapping stats - number of external data points, number of mapped data points

Rosetta support:

• Ingest support in Rosetta
• Access to Ingest PoC project in Rosetta

Phase 2 (2025 Q3 and Q4)

• FpML Record-Keeping imported as a Rune model
• DSL features to support multi-schema support (i.e., FpML Confirmation and Record-Keeping)
• DSL features to allow more concise and readable mappings
• Add Rune DSL syntax features to support both FpML Confirmation and Record-Keeping schemas with minimal duplication of mapping functions
• Support for private extension to the FpML schema. Although not covered by this issue, the same DSL features to support multiple FpML schemas efficiently will be reusable to support private FpML extensions (a common industry scenario)

[hugohills-regnosys]

GitHub repositories created

FpML as Rune GitHub repository for the imported FpML as Rune model https://github.com/rosetta-models/rune-fpml

Ingest PoC model with parent models CDM and FpML as Rune that contains FpML to CDM ingestion functions https://github.com/rosetta-models/fpml-cdm-ingest-poc

[hugohills-regnosys]

Ingest support in Rosetta

• Users can write ingestion functions, run ingestion on all CDM FpML test pack samples, and view diagnostics.
• Ingest PoC project is available in Rosetta (access granted on request)

[llynhiavu]

@lolabeis thanks for the presentation at the steerco. To follow up on my question about support for custom extension to FpML, it is good to hear that it is planned and will be supported in "target state - internal model" (ppt slide 6).

To that end, would it make sense to clarify the scope for Phase 2 (this page) to include something to the effect "support for private extension to the FpML schema" (i.e. firms extending the standard FpML schema with a custom xsd that includes extended types, etc. in a private namespace) ?

The current bullet that says "DSL features to support multi-schema support (i.e., FpML Confirmation and Record-Keeping)" is not explicitly mentioning private / "internal" schemas (and probably not the same topic altogether). i.e. there is perhaps (?) a discrepancy between the original scope of this issue, when it was written, and your latest thinking in the PPT. Thanks.

[lolabeis]

Thank you @llynhiavu for your feedback

To that end, would it make sense to clarify the scope for Phase 2 (this page) to include something to the effect "support for private extension to the FpML schema".

A bullet point has been added to that effect for Phase 2 (whilst making it clear that such support is not strictly speaking part of this issue).

[hugohills-regnosys]

Implementation Update – Q2 2025

At the end of Q2 2025, the Ingest functions for FpML Confirmation to CDM will be contributed to the CDM 7-dev version, where they will be available for beta testing by the CDM community. This milestone provides an opportunity to gather feedback, validate the implementation, and ensure alignment with community needs.

The initial contribution will include:

• Ingest functions for existing FpML test packs across Rates and other asset classes, excluding test packs for incomplete products and exotics. This ensures broad coverage of standard product types to support meaningful community testing.
• Migration of all custom Java implementations (used in the current synonym ingest process) to native Rune functions.
• Rune FpML – the FpML Confirmation schema imported as a Rune model – will be open-sourced and included in the CDM model.

Following the beta phase and incorporation of community feedback, the Ingest functions will be backported and contributed to the CDM 6 production version by the end of Q3 2025, enabling broader adoption in live environments.

[lolabeis]

Update presented at the CDM Steering Working Group - June 10th, 2025. Will keep this item in "Follow-Up" there, to provide periodic updates to the group based on the above plan.

[CDM-ReleaseManagement-LSG]

Discussed at the CDM Steering Working Group - July 8th, 2025 #3835.
The underlying PRs to deliver this issue will be tabled at the next CRWG.

[lolabeis]

Moving this one back to "Follow-Up". The issue that was previously discussed / approved was actually sub-issue #3836

[lolabeis]

Provided a progress update at the CDM Steering Working Group - September 23rd, 2025