accelerate [feature request] Cut new release and/or clarify release roadmap

Is your feature request related to a problem? Please describe.

I've been trying to get a nix flake working for Simon Marlow's book, and had a little bit of (now resolved) friction with accelerate.

It appears that accelerate is still seeing active development and compiles up to GHC 9.10 (see cce3400ab64c4c6e0af7a28f4281a5f27b03a896), but the latest commit (49a39ea6e3d2d13cbfa8605dcb57a29ef13db1f9) to the stable branch/1.3.0.0 tag is 5 years old. This is reflected on hackage as well, giving constraints on dependencies that are probably too restrictive -- not to mention what ever fixes and features are in the 300+ commits from 2020 til now!

Describe the solution you'd like

I'm new to the library, so I'm unfamiliar with the development status. There's plenty of valid reason why work might not be considered "stable", but:

if all or some the work is stable, getting a new major-or-minor version bump onto hackage would be great.
if none of the work is release-ready, indicating in the README:
- what the release process is
- indicating in the README what should be done by library users who would like a more recent version (working from master?)

Describe alternatives you've considered

None.

Additional context

I've submitted a PR to nixpkgs pulling directly from source.

Apr 25 '25 05:04 dragospe

Accelerate is a project developed primarily by researchers, some in paid time, some in their free time. Incentives and time allotment don't always align with getting the best user experience for users, unfortunately — and this includes cutting releases. (Also, funding is temporary and people move on, so activity at any particular time does not necessarily mean activity at a different time.)

Having said all that: apologies for the lack of releases over the past couple of years; this is mostly due to an in-progress full native & ptx backend rewrite (that I'm not personally working on, by the way) that sucks up ~all the resources. An Accelerate version with this new rewritten set of backends may be released this calendar year, but we can give no guarantees on that.

Personally I intend to cut a release with roughly what's on master, plus some work to remove the library dependency on LLVM (and call the clang executable instead), somewhere this spring. But again, I can give no guarantees there.

Thank you for tending to Nix! None of us use Nix, so we do not feel comfortable maintaining Nix metadata ourselves.

Apr 25 '25 07:04 tomsmeding

Makes sense. Thanks for the info!

So, for someone who's diving into Simon Marlowe's Parallel and Concurrent Haskell for the first time: do you recommend using the stable version, or is master likely to be a better experience?

The main reason I ask is because I have a nvidia RTX 40-series GPU, which was released in 2022. I'm wondering whether "maybe getting the older version to support newer hardware" or "dealing with some possibly-unstable changes on master" is a more advisable.

In either case, I'll try to keep y'all updated with my experience as I'm learning the library.

Apr 25 '25 07:04 dragospe

Definitely master, and in particular:

master of AccelerateHS/accelerate and AccelerateHS/accelerate-llvm
The llvm-15 branch of llvm-hs/llvm-hs; this means that you will need LLVM version 15 (latest on hackage is LLVM 9)
The cuda-12 branch of tomsmeding/cuda; this corresponds to this PR on upstream tmcdonell/cuda that will likely be merged soon (I'm in contact with tmcdonell).

In fact, if you're okay with non-master versions, I can recommend going more bleeding-edge with the following cabal.project (or the equivalent Nix configuration, if you're not using cabal directly):

source-repository-package
  type: git
  location: https://github.com/AccelerateHS/accelerate.git
  -- current master (2025-04-25)
  tag: 40a7e2ac8823304e9a3317a31ef39c91e05821a5

source-repository-package
  type: git
  location: https://github.com/tomsmeding/accelerate-llvm
  -- branch 'no-link-llvm-ptx' (2025-04-25)
  tag: d4219967ef2b5f8614497ee91a54ab59d296ed33
  subdir:
    accelerate-llvm
    accelerate-llvm-native
    accelerate-llvm-ptx

source-repository-package
  type: git
  location: https://github.com/tomsmeding/llvm-pretty
  -- branch 'ptx' (2025-04-25)
  tag: 16fb617f282bde97a090b8b08cbd2dcf5e259ad3

This configuration only requires you to install clang (any recent version), not LLVM 15 in a specific build configuration.

The llvm-pretty fork will likely be vendored into an Accelerate release this spring, if that does happen. We do plan to upstream the additional llvm-pretty functionality in the long term, but the extra functionality is breaking (it adds various fields to existing data constructors), and just using a fork is lower overhead for us currently than working with upstream to merge various breaking changes.

Apr 25 '25 07:04 tomsmeding

Excellent! This is the exact sort of thing I like having nix for. Once I get things working for myself with a nix flake, I'll at least send a ping with a working setup for as many (haskell and non-haskell) dependencies as I can get. Can't promise I can maintain it, but it will at least be a start for any other newcomers!

Apr 25 '25 07:04 dragospe

@dragospe I have a working flake with GHC 9.10 here

May 20 '25 01:05 cassandracomar

I'm trying the more bleeding edge versions, but running into this error:

Cloning into '/home/jaro/haskell/par/dist-newstyle/src/accelerate-bc364d47be689e0b0d70b0c14c246a316df1f41e69fb113a9a819eb608637e3c'...
remote: Enumerating objects: 32635, done.
remote: Counting objects: 100% (2329/2329), done.
remote: Compressing objects: 100% (236/236), done.
remote: Total 32635 (delta 2189), reused 2099 (delta 2093), pack-reused 30306 (from 3)
Receiving objects: 100% (32635/32635), 16.08 MiB | 17.15 MiB/s, done.
Resolving deltas: 100% (17488/17488), done.
HEAD is now at 40a7e2ac Better type inference for V<n> pattern synonyms
Submodule 'cbits/tracy' (https://github.com/wolfpld/tracy.git) registered for path 'cbits/tracy'
Cloning into '/home/jaro/haskell/par/dist-newstyle/src/accelerate-bc364d47be689e0b0d70b0c14c246a316df1f41e69fb113a9a819eb608637e3c/cbits/tracy'...
Submodule path 'cbits/tracy': checked out '5d542dc09f3d9378d005092a4ad446bd405f819a'
Entering 'cbits/tracy'
Cloning into '/home/jaro/haskell/par/dist-newstyle/src/accelerat_-3eb59045cd99e2e4e0c09ea7668e34e06a7eb46314a19918fccbd159f4e10662'...
remote: Enumerating objects: 25274, done.
remote: Counting objects: 100% (2590/2590), done.
remote: Compressing objects: 100% (168/168), done.
remote: Total 25274 (delta 2481), reused 2422 (delta 2422), pack-reused 22684 (from 2)
Receiving objects: 100% (25274/25274), 4.21 MiB | 17.05 MiB/s, done.
Resolving deltas: 100% (13122/13122), done.
fatal: Could not parse object 'd4219967ef2b5f8614497ee91a54ab59d296ed33'.

Sep 12 '25 09:09 noughtmare

@noughtmare A bunch of stuff has been merged to master. To get these more bleeding-edge versions, you now need:

AccelerateHS/accelerate master (81a2e722bef86393c077ba22642206c2a96b4d6c)
AccelerateHS/accelerate-llvm master (d9c556d18e5e4588b3785ef4e175cd73c57610c7)
~~tomsmeding/llvm-pretty ptx (a253a7fc1c62f4825ffce6b2507eebc5dadff32c)~~ This should be arranged automatically with a git submodule

The cuda package is now on Hackage again, at least for CUDA 12 (working on CUDA 13).

Sep 12 '25 09:09 tomsmeding

With those newer versions, I'm still getting the same git failure with the cbits/tracy submodule.

Sep 12 '25 09:09 noughtmare

Everything seems in order for me here; what commits are you on precisely, and what is the error that you get?

11:53:09 /tmp$ git clone [email protected]:AccelerateHS/accelerate
Cloning into 'accelerate'...
remote: Enumerating objects: 32635, done.
remote: Counting objects: 100% (2328/2328), done.
remote: Compressing objects: 100% (230/230), done.
remote: Total 32635 (delta 2188), reused 2104 (delta 2098), pack-reused 30307 (from 3)
Receiving objects: 100% (32635/32635), 16.08 MiB | 12.48 MiB/s, done.
Resolving deltas: 100% (17488/17488), done.

11:53:20 /tmp$ git clone [email protected]:AccelerateHS/accelerate-llvm
Cloning into 'accelerate-llvm'...
remote: Enumerating objects: 25255, done.
remote: Counting objects: 100% (2583/2583), done.
remote: Compressing objects: 100% (168/168), done.
remote: Total 25255 (delta 2474), reused 2415 (delta 2415), pack-reused 22672 (from 2)
Receiving objects: 100% (25255/25255), 4.21 MiB | 6.33 MiB/s, done.
Resolving deltas: 100% (13121/13121), done.

11:53:23 /tmp$ cd accelerate

11:53:25 /t/accelerate$ git submodule update --init --recursive
Submodule 'cbits/tracy' (https://github.com/wolfpld/tracy.git) registered for path 'cbits/tracy'
Cloning into '/tmp/accelerate/cbits/tracy'...
Submodule path 'cbits/tracy': checked out '5d542dc09f3d9378d005092a4ad446bd405f819a'

11:53:30 /t/accelerate$ cd ../accelerate-llvm/

11:53:34 /t/accelerate-llvm$ git submodule update --init --recursive
Submodule 'accelerate-llvm/llvm-pretty' (https://github.com/tomsmeding/llvm-pretty) registered for path 'accelerate-llvm/llvm-pretty'
Cloning into '/tmp/accelerate-llvm/accelerate-llvm/llvm-pretty'...
Submodule path 'accelerate-llvm/llvm-pretty': checked out 'a253a7fc1c62f4825ffce6b2507eebc5dadff32c'

11:53:42 /t/accelerate-llvm$

Sep 12 '25 09:09 tomsmeding

When I create a fresh cabal project (cabal init -mnq) and add this cabal.project:

packages: .

source-repository-package
  type: git
  location: https://github.com/AccelerateHS/accelerate.git
  -- master
  tag: 81a2e722bef86393c077ba22642206c2a96b4d6c

source-repository-package
  type: git
  location: https://github.com/tomsmeding/accelerate-llvm
  -- master
  tag: d9c556d18e5e4588b3785ef4e175cd73c57610c7
  subdir:
    accelerate-llvm
    accelerate-llvm-native
    accelerate-llvm-ptx

If I then run cabal build I get this output:


Cloning into '/tmp/acc/dist-newstyle/src/accelerate-bc364d47be689e0b0d70b0c14c246a316df1f41e69fb113a9a819eb608637e3c'...
remote: Enumerating objects: 32635, done.
remote: Counting objects: 100% (2329/2329), done.
remote: Compressing objects: 100% (235/235), done.
remote: Total 32635 (delta 2189), reused 2100 (delta 2094), pack-reused 30306 (from 3)
Receiving objects: 100% (32635/32635), 16.08 MiB | 9.34 MiB/s, done.
Resolving deltas: 100% (17488/17488), done.
HEAD is now at 81a2e722 Fix a warning
Submodule 'cbits/tracy' (https://github.com/wolfpld/tracy.git) registered for path 'cbits/tracy'
Cloning into '/tmp/acc/dist-newstyle/src/accelerate-bc364d47be689e0b0d70b0c14c246a316df1f41e69fb113a9a819eb608637e3c/cbits/tracy'...
Submodule path 'cbits/tracy': checked out '5d542dc09f3d9378d005092a4ad446bd405f819a'
Entering 'cbits/tracy'
Cloning into '/tmp/acc/dist-newstyle/src/accelerat_-3eb59045cd99e2e4e0c09ea7668e34e06a7eb46314a19918fccbd159f4e10662'...
remote: Enumerating objects: 25274, done.
remote: Counting objects: 100% (2590/2590), done.
remote: Compressing objects: 100% (168/168), done.
remote: Total 25274 (delta 2481), reused 2422 (delta 2422), pack-reused 22684 (from 2)
Receiving objects: 100% (25274/25274), 4.21 MiB | 9.94 MiB/s, done.
Resolving deltas: 100% (13122/13122), done.
fatal: Could not parse object 'd9c556d18e5e4588b3785ef4e175cd73c57610c7'.

Sep 12 '25 09:09 noughtmare

Ah, I see I should change https://github.com/tomsmeding/accelerate-llvm to https://github.com/AccelerateHS/accelerate-llvm.

Sep 12 '25 10:09 noughtmare

The errors that you get from git (via cabal) here are very uninformative. You first have to realise they're errors from git in the first place, and then you have to manually check where exactly the unknown hash is specified to be able to start thinking about what the solution is. This hash was indeed not cbits/tracy, it was the hash in your cabal.project. :P

Let us know if there is anything else you run into.

Sep 12 '25 10:09 tomsmeding