Not loading ASCII files containing a footer

[hortica]

Dr Anna Sokolova ANSTO, BILBY instrument

Expected behavior

I would assume that data files with a footer are legit. For example, some commonly used software for protein scattering data analysis (ATSAS suit) creating files with footers.

Actual behavior

When trying to load a file with a footer, the following error comes up: "Error in execution of algorithm Load: Failed to recognize this file as an ASCII file, cannot continue."

Steps to reproduce the behavior

test_footer.txt Please find file attached. It is BILBY data with artificially added footer. With the footer is doesn't work, without it does.

Platforms affected

I've checked it only on Windows 7 Enterprise, SP1. Mantid release 1 March 2019 3.13.20190301.1751 (nightly build)

[Matthew-Andrew]

@DavidFair I can't remember whether this was ever resolved David?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If you feel this is incorrect please comment to keep it alive, with a reason why.

To prevent closure, e.g. for long-term planning issues, add the "Never Stale" label.

[hortica]

With this, I am asking to keep the issue open. I've tried the release 5.1.20210120.2323 and the ASCII files with footers still cannot be loaded. I would prefer do not to specify a format for the footer; it can vary from file to file. It would be good to have an algorithm that can read ASCII files and ignore the text footer, if its format is differ from the format of data lines.

The error message I am getting is the following: "this file: x.dat, error description: bad lexical cast: source type value could not be interpreted as target Error in execution of algorithm LoadAscii: Failed to recognize this file as an ASCII file, cannot continue. Error in execution of algorithm Load: Failed to recognize this file as an ASCII file, cannot continue."

[hortica]

Good day! May I please ask you to advise if this issue will be checked and resolve? Happy to provide more information if needed.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in 6 months. It will be closed in 7 days if no further activity occurs. Allowing issues to close as stale helps us filter out issues which can wait for future development time. All issues closed by stale bot act like normal issues; they can be searched for, commented on or reopened at any point. If you'd like a closed stale issue to be considered, feel free to either re-open the issue directly or contact a developer. To extend the lifetime of an issue please comment below, it helps us see that this is still affecting you and you want it fixed in the near-future. Extending the lifetime of an issue may cause the development team to prioritise it over other issues, which may be closed as stale instead.

[hortica]

The issue is still unresolved, and it would be great to have some comment/decision on it. Thank you in advance.

[darominski]

@hortica If the lines of your footer started with a comment mark, by default #, then the error does not appear. Is it possible for you to produce footers that are not free text but contain a special character that would denote that it is a comment? It can be any character that you would then need to provide to the CommentIndicator property of the loader, unless you choose the default, then it's already taken care of.

The footer in this case is different than a header, where the comment indicator is not requested, but reading of the lines of the header are completely skipped when the data is loaded. It is harder to guess that a footer has started in the loader that is processing the file line by line, as it just appears to the code as an incorrect line: it is neither a comment, nor does it contain the numbers.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in 6 months. It will be closed in 7 days if no further activity occurs. Allowing issues to close as stale helps us filter out issues which can wait for future development time. All issues closed by stale bot act like normal issues; they can be searched for, commented on or reopened at any point. If you'd like a closed stale issue to be considered, feel free to either re-open the issue directly or contact a developer. To extend the lifetime of an issue please comment below, it helps us see that this is still affecting you and you want it fixed in the near-future. Extending the lifetime of an issue may cause the development team to prioritise it over other issues, which may be closed as stale instead.

[stale[bot]]

This issue has been closed automatically. If this still affects you please re-open this issue with a comment or contact us so we can look into resolving it.