Unitful.jl icon indicating copy to clipboard operation
Unitful.jl copied to clipboard

How to get a parseable unit string

Open MartinOtter opened this issue 4 years ago • 13 comments

I have the following problem:

using Unitful
# Variable with unit defined
v = 2.0u"m/s"    

# Extract unit as string
v_unit = string(unit(v))   # = "m s^-1"

# This string cannot be parsed
uparse(v_unit)   # gives an error

# Therefore code generation with the v_unit string does not work
code = :( @u_str($v_unit) )      # = :( u"m s^-1" )
eval(code)   # Gives an error

The reason is that string(unit(v)) returns a string that cannot be parsed. In Julia this seems to be an unusual behavior because string(something) typically returns a string representation of something that can be again parsed by Julia.

I searched extensively in Unitful documentation and the Issues but did not find a solution.

I currently use a bad hack (that most likely does not work in all situations), by replacing " " by "*":

v_unit = replace(  string(unit(v)), " " => "*")    # = "m*s^-1"
uparse(v_unit)   # fine
code = :( @u_str($v_unit) )      # = :( u"m*s^-1" )
eval(code)   # fine

Help is appreciated

MartinOtter avatar Jan 01 '21 09:01 MartinOtter

I put a lenient parser here. This uses an unregistered fork of Unitful, but should be easily adaptable. It's lenient with regards to space or no space between number and unit.

hustf avatar Jan 12 '21 08:01 hustf

...also lenient with regards to brackets. It's used for parsing output from proprietary applications with various formats.

julia> # When parsing text file, spaces as multipliers and brackets are allowed. Just specify the numeric type:
julia> lin = "2 [s]\t11364.56982421875 [N]\t-44553.50244140625 [N]\t-26.586366176605225 [N]\t0.0[N mm]\t0.0[N mm]\t0.0[N mm]\t1561.00350618362 [mm]\t-6072.3729133606 [mm]\t2825.15907287598 [mm]"
"2 [s]\t11364.56982421875 [N]\t-44553.50244140625 [N]\t-26.586366176605225 [N]\t0.0[N mm]\t0.0[N mm]\t0.0[N mm]\t1561.00350618362 [mm]\t-6072.3729133606 [mm]\t2825.15907287598 [mm]"

julia> time, Fx, Fy, Fz, Mx, My, Mz, px, py, pz = parse.(Quantity{Float64}, split(lin, '\t'))
10-element Array{Quantity{Float64,D,U} where U where D,1}:
                 2.0s
   11364.56982421875N
  -44553.50244140625N
 -26.586366176605225N
    0.0mm∙N
    0.0mm∙N
    0.0mm∙N
   1561.00350618362mm
   -6072.3729133606mm
   2825.15907287598mm

hustf avatar Jan 12 '21 08:01 hustf

Same question here! Unitful often prints units that itself cannot parse:

julia> u"angstrom^3"
ų

julia> uparse("ų")
ERROR: ArgumentError: Symbol ų could not be found in unit modules Module[Unitful]
Stacktrace:
 [1] lookup_units(unitmods::Vector{Module}, sym::Symbol)
   @ Unitful ~/.julia/packages/Unitful/1t88N/src/user.jl:581
 [2] lookup_units
   @ ~/.julia/packages/Unitful/1t88N/src/user.jl:593 [inlined]
 [3] uparse(str::String; unit_context::Module)
   @ Unitful ~/.julia/packages/Unitful/1t88N/src/user.jl:536
 [4] uparse(str::String)
   @ Unitful ~/.julia/packages/Unitful/1t88N/src/user.jl:535
 [5] top-level scope
   @ REPL[5]:1

This is not helpful when I want to serialize a quantity with units into a JSON or YAML file, then reconstruct it from that file later.

singularitti avatar Feb 01 '21 07:02 singularitti

Bump. Is there any update on this?

I don't necessarily need a unit string that can be parsed by Julia. But I would like a unit string that can at least be parsed by Unitful.uparse.

E.g. currently none of these work:

julia> a = Unitful.u"mg/dL/dL"
mg dL⁻²

julia> string(a)
"mg dL⁻²"

julia> Unitful.uparse(string(a))
ERROR: Base.Meta.ParseError("extra token \"dL⁻²\" after end of expression")
Stacktrace:
 [1] #parse#3
   @ ./meta.jl:227 [inlined]
 [2] parse(str::String; raise::Bool, depwarn::Bool)
   @ Base.Meta ./meta.jl:258
 [3] parse
   @ ./meta.jl:258 [inlined]
 [4] uparse(str::String; unit_context::Module)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [5] uparse(str::String)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [6] top-level scope
   @ REPL[6]:1

julia> b = Unitful.mg / Unitful.dL / Unitful.dL
mg dL⁻²

julia> a === b
true

julia> string(b)
"mg dL⁻²"

julia> Unitful.uparse(string(b))
ERROR: Base.Meta.ParseError("extra token \"dL⁻²\" after end of expression")
Stacktrace:
 [1] #parse#3
   @ ./meta.jl:227 [inlined]
 [2] parse(str::String; raise::Bool, depwarn::Bool)
   @ Base.Meta ./meta.jl:258
 [3] parse
   @ ./meta.jl:258 [inlined]
 [4] uparse(str::String; unit_context::Module)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [5] uparse(str::String)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [6] top-level scope
   @ REPL[10]:1

julia> repr(a)
"mg dL⁻²"

julia> repr(b)
"mg dL⁻²"

julia> Unitful.uparse(repr(a))
ERROR: Base.Meta.ParseError("extra token \"dL⁻²\" after end of expression")
Stacktrace:
 [1] #parse#3
   @ ./meta.jl:227 [inlined]
 [2] parse(str::String; raise::Bool, depwarn::Bool)
   @ Base.Meta ./meta.jl:258
 [3] parse
   @ ./meta.jl:258 [inlined]
 [4] uparse(str::String; unit_context::Module)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [5] uparse(str::String)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [6] top-level scope
   @ REPL[13]:1

julia> Unitful.uparse(repr(b))
ERROR: Base.Meta.ParseError("extra token \"dL⁻²\" after end of expression")
Stacktrace:
 [1] #parse#3
   @ ./meta.jl:227 [inlined]
 [2] parse(str::String; raise::Bool, depwarn::Bool)
   @ Base.Meta ./meta.jl:258
 [3] parse
   @ ./meta.jl:258 [inlined]
 [4] uparse(str::String; unit_context::Module)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [5] uparse(str::String)
   @ Unitful ~/.julia/packages/Unitful/PcVKX/src/user.jl:535
 [6] top-level scope
   @ REPL[14]:1

DilumAluthge avatar May 19 '21 19:05 DilumAluthge

I don’t think this is easily possible right now, since there is no way to recover the symbol that a unit is bound to just from its value.

For example, in UnitfulAtomic.jl there is a unit UnitfulAtomic.ħ_au that is printed as ħ. However, uparse("ħ") evaluates to Unitful.ħ (which is a quantity in J*s, not a unit), not UnitfulAtomic.ħ_au.

sostock avatar May 20 '21 05:05 sostock

One way would be to provide a utility function that converts a unit to a parseable unit string, something like "parseableString(vunit)"

MartinOtter avatar May 20 '21 06:05 MartinOtter

Any news here? Note, whenever a unit needs to be stored on file (say JSON file), it needs to be converted to a string (and when reading from file, the string needs to be converted from the string representation to the Unitful representation). Any advice how to do this with the current Unitful release?

To repeat the most important information: string(unit(v)) returns a string that cannot be parsed. In Julia this seems to be an unusual behavior because string(something) typically returns a string representation of something that can be again parsed by Julia. Alternatively, a function should be provided to do this, e.g. uparse( parseableString(2.0u"m/s") ) == u"m/s".

My current bad implementation is

parseableString(var_with_unit) = replace(repr(unit(var_with_unit), context = Pair(:fancy_exponent,false)), " " => "*")

MartinOtter avatar Mar 02 '22 11:03 MartinOtter

One way would be to return a string of the actual representation of the unit:

julia> str = parseable(u"km^2") # this function does not exist yet
"FreeUnits{(Unit{:Meter, Dimensions{(Dimension{:Length}(1),)}()}(3, 2),), Dimensions{(Dimension{:Length}(2),)}(), nothing}()"

julia> Unitful.eval(Meta.parse(str))
km^2

julia> ans === u"km^2"
true

Right now, parsing such a string does work with Unitful.eval ∘ Meta.parse, but uparse would have to be adapted to accept it:

julia> uparse(str)
ERROR: ArgumentError: FreeUnits{(Unit{:Meter, Dimensions{(Dimension{:Length}(1),)}()}(3, 2),), Dimensions{(Dimension{:Length}(2),)}(), nothing} is not a valid function call when parsing a unit.
Only the following functions are allowed: [:*, :/, :^, :sqrt, :√, :+, :-, ://]

sostock avatar Mar 05 '22 09:03 sostock

It seems that only show(x::Quantity, etc.) are defined on Unitful.jl, and string(x::Quantity) are not defined. Implicitly, string calls print and print calls show, so string and show are currently identical. (The show determines output to the REPL, etc.).

I am currently trying to create a package called UnitfulParsabelString.jl, which will allow me to add a parsable string without breaking Unitful by defining Unitful.string(x::Quantity).

julia> using Unitful, UnitfulParsableString # <- my package

julia> a = u"mg/dL/dL"
mg dL⁻²

julia> string(a)
"mg*dL^-2"

julia> string(a) |> uparse
mg dL⁻²

julia> b = u"angstrom^3"
ų

julia> string(b)
"Å^3"

julia> string(b) |> uparse
ų

julia> c = 1.0u"m*kg/s^2"
1.0 kg m s⁻²

julia> string(c)
"1.0(kg*m*s^-2)"

julia> string(c) |> uparse
1.0 kg m s⁻²

Is there a demand for this package? And if so, what characteristics should it have? (Currently, there are some bugs (it does not parse well when the exponent part is a fraction, i.e., u"m^(1/2)"), so it is not available to the public.)

michikawa07 avatar Sep 07 '22 07:09 michikawa07

For example, in UnitfulAtomic.jl there is a unit UnitfulAtomic.ħ_au that is printed as ħ. However, uparse("ħ") evaluates to Unitful.ħ (which is a quantity in J*s, not a unit), not UnitfulAtomic.ħ_au.

This problem may be solved by specifying the unit_context argument to uparse(str; unit_context).

function uparse(str; unit_context=Unitful)
    ex = Meta.parse(str)
    eval(lookup_units(unit_context, ex))
end

michikawa07 avatar Sep 07 '22 08:09 michikawa07

It seems that only show(x::Quantity, etc.). are defined on Unitful.jl, and string(x::Quantity). are not defined. Implicitly, string calls print and print calls show, so string and show are currently identical. (The show determines output to the REPL, etc.). ... julia> string(c) "1.0(kgms^-2)"

julia> string(c) |> uparse 1.0 kg m s⁻²

Is there a demand for this package? And if so, what characteristics should it have? (Currently, there are some bugs (it does not parse well when the exponent part is a fraction, i.e., u"m^(1/2)"), so it is not available to the public.)

Yes, there is a demand for such a package (e.g. whenever something needs to be stored on file in a standardized format such as JSON). So, I highly appreciate if you can provide such a package (it would be even better, if it would be part of Unitful.jl).

All your examples looks good. I am just a bit surprised for:

julia> string(c) "1.0(kgms^-2)"

that this can be parsed with uparse. I checked, and indeed this works with Unitful.jl.

MartinOtter avatar Sep 07 '22 08:09 MartinOtter

@MartinOtter You have encouraged me to complete the package to the point where I am satisfied with it. This is in the process of registering it as an official package, but you are welcome to use it if you like. https://github.com/michikawa07/UnitfulParsableString.jl

(@v1.8) pkg > add https://github.com/michikawa07/UnitfulParsableString.jl

michikawa07 avatar Sep 11 '22 13:09 michikawa07