Numerical value as exponent on 10s

[kermitt2]

The quantity CRF model recognizes numerical expressions with exponents on 10 (in particular distorted one due to PDF text extraction):

However we are not currently parsing it (in their "noisy" form) to actual BigDecimal values.

[lfoppiano]

Regarding this subject, I've started implementing a CRF model for parsing that.

The idea is to classify the value, like for example 5 x 10-5 as:

<val>5</val>
<operation>x</operation> --> althought this could be just <other>? 
<base>10</base>
<pow>-5</pow>

Regarding the resulting value, we need to accept that it will be not precise, as we have to approximate (we are talking about small values).

I would say we should save both form, the "structured" parsed value the schema above, and the parse value as BigDecimal to have an approximate value.

Does it make sense?

[kermitt2]

Hello!

The value parser should be generic enough to cover several cases described in #13.

The tagset has to be relevant for the different cases:

• <val> is misleading (this is the value parser), should be much better <number> for instance
• <operation> is useless, keep it non annotated
• <alpha> for alphabetic expression of numbers
• <exp> for the value of the exponential function (E/e kept non annotated)
• <time> for date and time expressions, a time parser more sophisticated than the existing date model of grobid will be needed

[kermitt2]

Regarding the approximation, the idea to introduce BigDecimal was to offer the possibility to set the precision, scaling and rounding, so that we can avoid the usual issue where 1 becomes 1.000000001.

[lfoppiano]

I've be been remarked that 2.3E5 correspond to https://en.wikipedia.org/wiki/Scientific_notation#E-notation meaning that it's a special case of the 10 power.

Was this the initial though you had, @kermitt2 ?

What do you mean with exponential function? I think I have misunderstood that part...

[kermitt2]

I would make the hypothesis that this notation is not relevant to scientific papers where the e is indeed always the exponential function - why would someone use the calculator/old program notation in a typeset scientific paper?

<exp> for the value of the exponential function (E/e kept non annotated) -> in this context we just annotate the value of the exponential, not the exponential function e, e.g. 10e5 -> <number>10</number>e<exp>5</exp> (note that we could maybe find something better than <number> as tag name)

[lfoppiano]

So this is the exponential function. Right? But then what about other functions like, for example log ? Since we don't have much data, maybe we could just support the exp "in future"?

[kermitt2]

I think this is the exponential function yes.

I think we don't express a value with a log (we don't write 3log(5)), if we have a log normally it's in an equation with variables, not as value.

mmm I dont see why would we wait for the future? The exponent e is quite frequent as part of a value and it is not complicated.