Programming-Language-Benchmarks icon indicating copy to clipboard operation
Programming-Language-Benchmarks copied to clipboard

Lisp benchmark is not run

Open bpecsek opened this issue 2 years ago • 7 comments

Could you please let us know why the lisp benchmarks are not run?

bpecsek avatar May 10 '22 17:05 bpecsek

It looks like Lisp (and Fortran and Haskell) was disabled in this commit and in that commit the README file was changed to say that the Programming Language and compiler Benchmarks "focuses more on new programming languages, classic programming languages that are covered by CLBG receive limited or no maintainence, based on their popularity." Lisp certainly isn't new and a pretty good argument can be made that it isn't very popular either.

I think many people would agree that for brevity, resource usage, and maintainability reasons it might not be a good idea to include too many languages. People will of course disagree about what criteria (like level of popularity) should be used for deciding whether a language should be included or not. It looks like you'll need to convince this project's maintainers to change its criteria in order to get Lisp included. If they don't decide to change this project's criteria to something you like, then you might want to consider using/creating an alternate benchmarking project instead. With that being said, in addition to including languages that are popular or quickly rising in popularity, I think it would be nice for this project to try to also include some languages which are reasonably interesting/unique/influential which I think Lisp could qualify as.

JZerf avatar May 15 '22 23:05 JZerf

I just wanted to preface this by saying thank you to the owner/contributors for working on this project for public use, both for being able to see the benchmarks and for the CI setup reference. It's the second result on Google for me when searching for "programming language benchmarks".

I just wanted to add that I was hoping to see Lisp benchmarks on the site as well.

If the owner doesn't want to spend their time writing benchmarks for a language they're not interested in, that makes total sense. However, seeing as the Lisp benchmarks already exist, or in a future scenario where someone wants to contribute for a lesser known language, why would having more data be a bad thing? With that said, if a language's presence is generating more maintenance time, increases complexity, or massively increases build time, I could see why it would be excluded. But from my limited understanding it seems like once a language is incorporated into the build system, it wouldn't require more work, since it's not like the mandelbrot code for that version of a language is suddenly going to become outdated or something. Of course, I could be very wrong, in which case please ignore my comment.

Thanks again for the benchmarks guys!

familyfriendlymikey avatar Jun 03 '22 17:06 familyfriendlymikey

I've updated all the CL codes for the SBCL and SB-SIMD so it shouldn't be a problem including SBCL at all.

bpecsek avatar Jun 06 '22 16:06 bpecsek

I would very much like to see CL performance. Lisps are very much alive in Clojure on JVM and Common Lisp.

oboff avatar Jun 15 '22 00:06 oboff

SBCL-2-2-6 including the sb-simd by default as contrib module is out. I have updated all the sbcl lisp codes to be compatible with the latest sbcl in my PR. It would be great to see SBCL Common Lisp among the bench-marked languages again as it was in the beginning until its been removed.

bpecsek avatar Jun 30 '22 12:06 bpecsek

Clojure at least deserves to be covered, because the Debian benchmarks don't cover Clojure.

Clojure is also technically a new language; it's overshadowed by Haskell in the functional programming space, but it's more production-oriented and produces unicorns at a rate Haskell doesn't match (Haskell only got its first unicorn a few years back with Hasura). It is also relatively new; it's a 2005 / 2007 language, whereas Python and Haskell are roughly 1993.

Haskell, while I wish it would be included, is already covered by Debian benchmarks and cross-correlations are viable.

liamzee avatar Dec 22 '22 12:12 liamzee

That's sad. For reference, here's a screenshot (where Lisp is at the top!): https://preview.redd.it/vn5juu36v2681.png?width=715&format=png&auto=webp&v=enabled&s=26ea1277170c63b4f17127785367f3b11f9076f1

vindarel avatar May 19 '23 22:05 vindarel