proposal to trace the includes files

[hugbe8]

A proposal to track files included in error messages or during common use. The detour to get there is not ideal but it meets the need to know the position of a line in the original file.

Sample use:

• for validate a file:

mapService, traceFile = mappyfile.open(strPathMapFile, output_trace=True, include_position=True) 
errors = mappyfile.validate(mapService, traceFile, version=7.6)
for e in errors:
    print(e)

• for commun use:

mapService, traceFile = mappyfile.open(strPathMapFile, output_trace=True, include_position=True)
for lay in layers:
    # trace original include mapFile of Layer
    pd = lay["__position__"]
    trace = traceFile[pd.get("line")]
    originFile = traceFile[0][trace]

Sample message output (the message indicates the location in the include file):

• on parsing error (mappyfile.open):

wms_datarul_format" "text/html"

ERROR] - Parsing of \\server001\systemes\mapserver\folder\theme\mapfiles\sample.include.map unsuccessful (Line: 14 Column: 28)
Traceback (most recent call last):
    ...
lark.exceptions.UnexpectedCharacters: No terminal matches '/' in the current parser context, at line 365 col 28
                wms_dataurl_format" "text/html"
                                         ^
Expected one of:
        * ESCAPED_STRING
        * SINGLE_QUOTED_STRING
        * DOUBLE_QUOTED_STRING
        * UNQUOTED_STRING
Previous tokens: Token('UNQUOTED_STRING', 'text')

• on validate (mappyfile.validate):

OUTLINECOLOR 252 211 19X

{'error': "'0', '9x' do not match any of the regexes: '^__[a-z]+__$'", 'message': "ERROR: Invalid value in STYLE, parent path: ['layer name: image_layer_projet', 'class name: Index Projet', 'style']", 'line': 31, 'column': 3, 'file': '\\\\server001\\systemes\\mapserver\\folder\\theme\\mapfiles\\sample.include.map'}

CONNECTIONTYPE PSTGIS

{'error': "'pstgis' is not one of ['contour', 'kerneldensity', 'idw', 'local', 'ogr', 'oraclespatial', 'plugin', 'postgis', 'sde', 'union', 'uvraster', 'wfs', 'wms', 'mygis']", 'message': "ERROR: Invalid value in CONNECTIONTYPE, parent path: ['layer name: image_layer_projet']", 'line': 7, 'column': 2, 'file': '\\\\server001\\systemes\\mapserver\\folder\\theme\\theme\\sample.include.map'}

[geographika]

Hi @hugbe8 - I appreciate the interest shown in mappyfile! I presume the reasoning for the update is to show the correct line numbers within INCLUDE files, rather than the fully "compiled" Mapfile?

There are some interesting parts to the pull request. The check for when Lark reports the error at the end of the file is a good addition. I'm not convinced on the approach of passing around the trace_o_incl - this has an impact on the API and many places in the code base.

Something like adding a structure of mappings of line numbers to files when adding the includes (here could be cleaner.

E.g.

{
  "main.map" [(1,205), (260, 300)]
  "include1.inc" [(206, 259)]
}

Any parsing or validation errors could then check their line number against this structure to get the child file if needed. This approach might still require modifications to error handling.

[hugbe8]

Hi @geographika - This is effectively to show the correct line numbers within INCLUDE files, rather than the fully "compiled" Mapfile

Before thinking of another structure of the line numbers. We must review the way to access the trace. Mappyfile.open does not make a property available to the function 'validate' as an object instance would. So I circulate the object to make it accessible in the code. I didn't dare to modify mappyfile to use it as an instance, it would have drastically changed the code and its usage. And I also did not dare to integrate the object trace in the dictionary returned by the 'open' function.

I am interested if you have a preference in the approach or if you have another idea.

[pelord]

@geographika Any update about this pr?

[geographika]

@pelord - thoughts are still the same as https://github.com/geographika/mappyfile/pull/130#issuecomment-945748510 It is a nice addition/feature, but the trace_o_incl is too intrusive, and it would be better to pass the structure around in the dictionary in an __includes__ property. The current pull request has 171 failing tests (and no new tests). I've left it open until now, as others may find it of interest.

I may look at this again in the future, but it is not a priority for me. I'm open to adding this through funded development - get in touch via https://geographika.net/ if you're interested.