Feature request: merging YAML lists

[Nnwww]

For example, I encountered the following situation:

_exe-ghc-options: &exe-ghc-options
  - -O2
  - -threaded
  - -rtsopts
  - -with-rtsopts=-N

executables:
  mypackage:
    ghc-options: *exe-ghc-options
    ghc-prof-options:
      *exe-ghc-options
      - -with-rtsopts="-N -p"

Since the exe-ghc-options approach produces a nested list, this cause a parse error at ghc-prof-options. I guess that hpack have been using YAML for removing redundancies. But It's seems that the YAML specification do not allow us to merge lists. (https://stackoverflow.com/questions/24090177/how-to-merge-yaml-arrays)

The finer grained the project is, the more helpful it is. So could we implement merging lists in some way?

Thank you for your time and effort!

[liskin]

Alternatively, we could allow every list to be a hash instead. Related: https://github.com/sol/hpack/issues/186

[tfausak]

Seriously though, supporting hashes for GHC options might be nice. It could help with problems like #182 too.

ghc-options:
  -with-rtsopts: -N

[mizunashi-mana]

https://github.com/sol/hpack/issues/194#issuecomment-329560341 seems good. But, we should think about some cases and make the translating semantics more clearly.

In a easy case, it translates from

ghc-options: &base-ghc-options
  -Wall: true
  -rtsopts: true
  -threaded: true
  -with-rtsopts: -N

ghc-prof-options:
  <<: *base-ghc-options
  -Wall: false
  -with-rtsopts: -N -p

to

ghc-options: "-Wall" "-rtsopts" "-threaded" "-with-rtsopts=-N"
ghc-prof-options: "-rtsopts" "-threaded" "-with-rtsopts=-N -p"

This semantics is simple:

• <option>: null => same as <option>: true
• <option>: true => "<option>"
• <option>: false =>
• <option>: <parameter> => "<option>=<parameter>"

However, if we use -O2 and -O0 options, how to behave for merging?

ghc-options: &base-ghc-options
  -O2: true

ghc-prof-options:
  <<: *base-ghc-options
  -O0: true
  # Should we write `-O2: false` ?

[mizunashi-mana]

P.S. if there's no idea for the several cases, I will write a PR to support hash options with above simple translation.

[sol]

Alternatively, we could allow every list to be a hash instead.

Before considering this I want to see if #211 will help the situation.

[liskin]

Before considering this I want to see if #211 will help the situation.

I think that's an entirely different usecase. Mine was using yaml anchors/aliases as macros for things like test suites, groups of dependencies, etc. Unfortunately I no longer use Haskell at work so I haven't had a chance to use these macros for long enough to have any sort of insight into how it ought to work. (And my pet projects don't have 20 packages in one git repo.)

[mizunashi-mana]

I think that's an entirely different usecase.

I think so, too.

And, hash styles have affinity with YAML. Because my custom options tend to be complex, I agree with hash supports for option fields. (About me, that's enough.)

[quasicomputational]

I've also felt this pain and it's annoying that we can't solve it at the YAML syntax level. Stupid suggestion: auto-flatten lists? So, e.g., this:

dependencies:
  - base
  - [text]
  - - [containers]
    - bytestring

would be equivalent to this:

dependencies: [base, text, containers, bytestring]

Then you can use YAML references to build lists-of-lists and hpack will DTRT.

[sol]

@quasicomputational That is an interesting idea. We could even expand on this, to accept e.g.:

dependencies:
  - deepseq
  - QuickCheck: 2.11.2

I still want to gather experience with defaults first, before doing anything here.

[quasicomputational]

Another idea in the design space: I think we could write something like this, too:

dependencies: !flatten
  - [base]
  - [text]
  - !flatten
    - [containers, bytestring]

This would need support from the yaml library for a new !flatten tag, similar to !include. I don't have an opinion on whether it should only do a single level of flattening (and hence require all of its children to be lists) or should automatically flatten sublists as well (and hence could permit atoms as direct children). This has the advantage of needing few, if any, hpack changes, and being in a sense orthogonal.

[vlatkoB]

I just started using hpack, so I'm not sure if my question is related to this issue. I'd like to merge include, global and subsection field (dependencies in particular). Something like this:

• include.yaml

- &def-deps
  dependencies:
    - bytestring

• package.yaml

_default-opts: !include "../include.yaml"

<<: *def-deps
dependencies:
   - text
...
library:
  dependencies:
    - containers

So the library stanza in generated app.cabal would look like this:

library:
  dependencies:
    - bytestring
    - containers
    - text

Is it even possible to do something like this now (with some special syntax), or not and related to this issue?

[sol]

@vlatkoB This can be achieved with defaults, see https://github.com/sol/hpack#defaults.

Example:

# include.yaml
dependencies:
- bytestring

# package.yaml
name: foo

defaults:
  local: include.yaml

dependencies:
- text

library:
  dependencies:
  - containers

[vlatkoB]

@sol Thanks, that worked. Obviously, I wasn't reading carefully enough because I somehow understood defaults is some kind of new include. :-(

Lots of word "defaults" in reference. However, now that I read it again, all seems clear.

[calebmer]

The original feature request would be helpful for sharing warning configurations between library code, executable code, and test code. For example, I currently have this in my library configuration, but I would like to share it with my tests:

  ghc-options:
    # Enable all warnings with -Weverything, then disable the ones we don’t care about
    - -Weverything
    - -Wno-missing-exported-signatures # missing-exported-signatures turns off the more strict -Wmissing-signatures. See https://ghc.haskell.org/trac/ghc/ticket/14794#ticket
    - -Wno-missing-import-lists # Requires explicit imports of _every_ function (e.g. ‘$’); too strict
    - -Wno-missed-specialisations # When GHC can’t specialize a polymorphic function. No big deal and requires fixing underlying libraries to solve.
    - -Wno-all-missed-specialisations # See missed-specialisations
    - -Wno-unsafe # Don’t use Safe Haskell warnings
    - -Wno-safe # Don’t use Safe Haskell warnings
    - -Wno-missing-local-signatures # Warning for polymorphic local bindings; nothing wrong with those.
    - -Wno-monomorphism-restriction # Don’t warn if the monomorphism restriction is used
    - -Wno-implicit-prelude # Of course we want to implicitlly import the prelude

[sol]

The original feature request can be achieved today, without any modifications to hpack:

_exe-ghc-options: &exe-ghc-options
  - -O2
  - -threaded
  - -rtsopts

executables:
  mypackage:
    main: Main.hs
    ghc-options: *exe-ghc-options
    ghc-prof-options: *exe-ghc-options
    when:
      condition: true
      ghc-options: -with-rtsopts=-N
      ghc-prof-options: -with-rtsopts="-N -p"

This will produce:

executable mypackage
  main-is: Main.hs
  other-modules:
      Paths_foo
  ghc-options: -O2 -threaded -rtsopts
  ghc-prof-options: -O2 -threaded -rtsopts
  if true
    ghc-options: -with-rtsopts=-N
    ghc-prof-options: -with-rtsopts="-N -p"
  default-language: Haskell2010

As a side note, it's best not to assume any ordering guarantees until we figured out how to deal with https://github.com/sol/hpack/issues/255. This is why I moved -with-rtsopts=-N into the conditional.