What to do about the transformix outputpoints.txt file format?

[N-Dekker]

The outputpoints.txt file format (produced by transformix -def inputPoints.txt...) appears a bit too verbose, and old-fashioned. For example:

Point	0	; InputIndex = [ 0 0 ]	; InputPoint = [ 0.000000 0.000000 ]	; OutputIndexFixed = [ 1 -2 ]	; OutputPoint = [ 1.000000 -2.000000 ]	; Deformation = [ 1.000000 -2.000000 ]
Point	1	; InputIndex = [ 0 1 ]	; InputPoint = [ 0.000000 1.000000 ]	; OutputIndexFixed = [ 1 -1 ]	; OutputPoint = [ 1.000000 -1.000000 ]	; Deformation = [ 1.000000 -2.000000 ]
Point	2	; InputIndex = [ 1 0 ]	; InputPoint = [ 1.000000 0.000000 ]	; OutputIndexFixed = [ 2 -2 ]	; OutputPoint = [ 2.000000 -2.000000 ]	; Deformation = [ 1.000000 -2.000000 ]
Point	3	; InputIndex = [ 1 1 ]	; InputPoint = [ 1.000000 1.000000 ]	; OutputIndexFixed = [ 2 -1 ]	; OutputPoint = [ 2.000000 -1.000000 ]	; Deformation = [ 1.000000 -2.000000 ]

Is it still relevant nowadays? If so could it be made slightly more compact? For example more like, starting with a header, having the names of the columns tab separated:

Point	InputIndex	InputPoint	OutputIndexFixed	OutputIndexPoint	Deformation
0	[ 0 0 ]	[ 0.0 0.0 ]	[ 1 -2 ]	[ 1.0 -2.0 ]	[ 1.0 -2.0 ]
1	[ 0 1 ]	[ 0.0 1.0 ]	[ 1 -1 ]	[ 1.0 -1.0 ]	[ 1.0 -2.0 ]
2	[ 1 0 ]	[ 1.0 0.0 ]	[ 2 -2 ]	[ 2.0 -2.0 ]	[ 1.0 -2.0 ]
3	[ 1 1 ]	[ 1.0 1.0 ]	[ 2 -1 ]	[ 2.0 -1.0 ]	[ 1.0 -2.0 ]

It appears implemented inside elx::TransformBase::TransformPointsSomePoints: https://github.com/SuperElastix/elastix/blob/5.0.1/Core/ComponentBaseClasses/elxTransformBase.hxx#L1278-L1335

@mstaring @stefanklein Do you have any comment?

[stefanklein]

Indeed, it is a bit verbose. Easy to parse 😊. Instead of inventing a new suboptimal home-brew format, can we not just keep this one as option for backward compatibility, and add the possibility to save in a standard format, e.g. HDF5? Stefan

From: Niels Dekker @.> Sent: Friday, June 17, 2022 12:01 PM To: SuperElastix/elastix @.> Cc: Stefan Klein @.>; Mention @.> Subject: [SuperElastix/elastix] What to do about the transformix outputpoints.txt file format? (Issue #665)

Waarschuwing: Deze e-mail is afkomstig van buiten de organisatie. Klik niet op links en open geen bijlagen, tenzij u de afzender herkent en weet dat de inhoud veilig is. Caution: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

The outputpoints.txt file format (produced by transformix -def inputPoints.txt...) appears a bit too verbose, and old-fashioned. For example:

Point 0 ; InputIndex = [ 0 0 ] ; InputPoint = [ 0.000000 0.000000 ] ; OutputIndexFixed = [ 1 -2 ] ; OutputPoint = [ 1.000000 -2.000000 ] ; Deformation = [ 1.000000 -2.000000 ]

Point 1 ; InputIndex = [ 0 1 ] ; InputPoint = [ 0.000000 1.000000 ] ; OutputIndexFixed = [ 1 -1 ] ; OutputPoint = [ 1.000000 -1.000000 ] ; Deformation = [ 1.000000 -2.000000 ]

Point 2 ; InputIndex = [ 1 0 ] ; InputPoint = [ 1.000000 0.000000 ] ; OutputIndexFixed = [ 2 -2 ] ; OutputPoint = [ 2.000000 -2.000000 ] ; Deformation = [ 1.000000 -2.000000 ]

Point 3 ; InputIndex = [ 1 1 ] ; InputPoint = [ 1.000000 1.000000 ] ; OutputIndexFixed = [ 2 -1 ] ; OutputPoint = [ 2.000000 -1.000000 ] ; Deformation = [ 1.000000 -2.000000 ]

Is it still relevant nowadays? If so could it be made slightly more compact? For example more like, starting with a header, having the names of the columns tab separated:

Point InputIndex InputPoint OutputIndexFixed OutputIndexPoint Deformation

0 [ 0 0 ] [ 0.0 0.0 ] [ 1 -2 ] [ 1.0 -2.0 ] [ 1.0 -2.0 ]

1 [ 0 1 ] [ 0.0 1.0 ] [ 1 -1 ] [ 1.0 -1.0 ] [ 1.0 -2.0 ]

2 [ 1 0 ] [ 1.0 0.0 ] [ 2 -2 ] [ 2.0 -2.0 ] [ 1.0 -2.0 ]

3 [ 1 1 ] [ 1.0 1.0 ] [ 2 -1 ] [ 2.0 -1.0 ] [ 1.0 -2.0 ]

It appears implemented inside elx::TransformBase::TransformPointsSomePoints: https://github.com/SuperElastix/elastix/blob/5.0.1/Core/ComponentBaseClasses/elxTransformBase.hxx#L1278-L1335https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSuperElastix%2Felastix%2Fblob%2F5.0.1%2FCore%2FComponentBaseClasses%2FelxTransformBase.hxx%23L1278-L1335&data=05%7C01%7Cs.klein%40erasmusmc.nl%7C86085e7ab4c948ec402308da5048515e%7C526638ba6af34b0fa532a1a511f4ac80%7C0%7C0%7C637910568777936395%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Qv5KBkiob6vE685U%2FRhHxVv9QazDLdLrTH0Oe67EeLQ%3D&reserved=0

@mstaringhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmstaring&data=05%7C01%7Cs.klein%40erasmusmc.nl%7C86085e7ab4c948ec402308da5048515e%7C526638ba6af34b0fa532a1a511f4ac80%7C0%7C0%7C637910568777936395%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aASPRb4Gaoq90cpNEUCz%2BWEj4Ebh7ol4ZYDDWNZ7aak%3D&reserved=0 @stefankleinhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fstefanklein&data=05%7C01%7Cs.klein%40erasmusmc.nl%7C86085e7ab4c948ec402308da5048515e%7C526638ba6af34b0fa532a1a511f4ac80%7C0%7C0%7C637910568777936395%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=D3B4o0BEDzKxRExTBzr1jky9Ec7TSUvYfPNWWP24KYk%3D&reserved=0 Do you have any comment?

— Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSuperElastix%2Felastix%2Fissues%2F665&data=05%7C01%7Cs.klein%40erasmusmc.nl%7C86085e7ab4c948ec402308da5048515e%7C526638ba6af34b0fa532a1a511f4ac80%7C0%7C0%7C637910568777936395%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=H5psH2ncmnGlipflHaoqKdcrsk2O%2BEqdAd9JiLqjXnk%3D&reserved=0, or unsubscribehttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAF2LNNJOSQ7EP2I4ZQABVTVPREGVANCNFSM5Y7EANQQ&data=05%7C01%7Cs.klein%40erasmusmc.nl%7C86085e7ab4c948ec402308da5048515e%7C526638ba6af34b0fa532a1a511f4ac80%7C0%7C0%7C637910568777936395%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=wZ4vvS%2BZg7Ip1NTxS0bYPCKOQiMFTf5DBzhKQVfbIkk%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.@.>>

[N-Dekker]

Thanks for your comment @stefanklein Honestly I'm not opposed to a text format, but then I think a tab separated format would still be easier to parse than the old elastix outputpoints.txt format. A binary format (HDF5?) could be of interest if it would significantly reduce the file size, or if it would ease data exchange with other programs. To be continued... 😃

[crousseau]

@N-Dekker @stefanklein @mstaring We are extremely interested in having support for binary format for points in elastix (as an option) because we use it on datasets with millions of points and text is not workable. We actually need to ship a customised version of elastix in our own software for this very reason (https://github.com/ChristophKirst/ClearMap2/tree/master/ClearMap/External/elastix). However, it would make much more sense if we could have these changes upstream and be able to benefit from both updates (our changes are on 4.9) and existing python bindings. If you would be interested in merging our versions, I would be happy to discuss a way forward. Thanks

[N-Dekker]

@crousseau Very nice to hear that elastix is of help to you. We're certainly interested to see the customizations you did to elastix. We don't maintain elastix 4.9 anymore, any changes are usually applied on top of the main branch of https://github.com/SuperElastix/elastix In the last few years we achieved a significant run-time performance improvement, and we very much extended the library interface. If it is feasible for you to start using the library interface, you could use the new itk::TransformixFilter member functions SetInputMesh and GetOutputMesh, which allow setting and retrieving the points in memory, so that they can be read and written in any way that your application supports. Would such an approach be feasible to you?

If you still need binary file support integrated with the executable of elastix/transformix, please feel free to submit a pull request. However, we currently have limited time, so if it involves a lot of code changes, we might postpone your pull request until after the summer. Hope you understand!