Please change screamer's maintainership

[swapneils]

Per this comment @nikodemus is no longer working on Screamer.

I've added a few constructs, debugged some issues, and integrated some of the languishing PRs over the last year, so would like to direct Quicklisp to the https://github.com/swapneils/screamer repository instead (changing this file).

Backwards-incompatible changes: There have been API changes, mainly new symbols. From the particular symbols changed, I don't expect any conflicts that won't be automatically resolved by CL's function/macro redefinition semantics, however.

The systems have also been reorganized to allow only importing portions of the functionality, but users can continue importing the :screamer system to get a "default experience".

Screamer now depends on :alexandria, :serapeum, and :iterate; if users are getting the library from Quicklisp this shouldn't be an issue.

~~EDIT 2024-12-23: There is one incompatible change to the API of ith-value. It's not meaningfully different from the changes between 3.2 and 4, allows for significantly more flexibility in usage and maintenance of this form, and ith-value itself isn't as useful compared to all-values and the new n-values, so don't think this is a blocking risk to library users.~~

• Reverted in 5.2 in favor of a new form.

[quicklisp]

I can't clone the htmlize.sh submodule any more - it uses the deprecated git interface to github, rather than https. Can it be updated or (better) removed?

On Sun, Dec 1, 2024 at 4:19 PM Swapneil Singh @.***> wrote:

Per this comment https://github.com/nikodemus/screamer/issues/31#issuecomment-1777717695 @nikodemus https://github.com/nikodemus is no longer working on Screamer.

I've added a few constructs, debugged some issues, and integrated some of the languishing PRs over the last year, so would like to direct Quicklisp to the https://github.com/swapneils/screamer repository instead.

Backwards-incompatible changes: There have been API changes, mainly new symbols. From the particular symbols changed, I don't expect any conflicts that won't be automatically resolved by CL's function/macro redefinition semantics, however.

The systems have also been reorganized to allow only importing portions of the functionality, but users can continue importing the :screamer system to get a "default experience".

Screamer now depends on :alexandria, :serapeum, and :iterate; if users are getting the library from Quicklisp this shouldn't be an issue.

— Reply to this email directly, view it on GitHub https://github.com/quicklisp/quicklisp-projects/issues/2455, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACPNLKFUZJQDRT7ZVS4W532DN4ONAVCNFSM6AAAAABS2CXRVWVHI2DSMVQWIX3LMV43ASLTON2WKOZSG4YDSOBWGE4DMMA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[swapneils]

Updated to use https. Keeping it around in case I get bandwidth to replicate Nikodemus's intro website to Screamer.

[nikodemus]

Just confirming that I'm fine with the maintainer change. I have very little bandwidth atm, but if @swapneils submits a PR with appropriate pointers to his current repo, I'll be happy to merge it.

[fstamour]

Seems like a good case for https://github.com/sharplispers

[swapneils]

Seems like a good case for https://github.com/sharplispers

Considered using Screamer as my cover letter for that group 😉, but I think there's potential here beyond the existing functionality, and the sharplispers charter of "maintaining" may mislead users into thinking the new features are stable-by-default (the README and docstrings would tell them otherwise, but people generally don't RTFM). Some examples:

• Nondeterminism and parallelism are two sides of the same coin, and I already have some constructs in :screamer/experimental which parallelize Screamer code by just switching a function name (built on lparallel).
• Probabilistic programming is just an extension of nondeterminism, and a very useful building-block for making real-world applications that need to model a scenario or find an optimal decision. Also prototyped already (in :screamer/prob).
• The variable system can be extended into a framework for automatically-dependently-typed programs -- warnings if you violate code assumptions, optimizing or (given user permission) entirely rewriting forms based on compile-time type knowledge (which could be arbitrarily broader than the default CL type system, and potentially extensible by other libraries), the works.
  • Still working out the semantics and UX here. We don't want to limit users to the library-defined types, so the existing implementation needs to be reworked into an easily-understood, extensible-yet-normalizable type system first, and then integrated into the various "-v" functions.
• On a smaller scale, some iteration code is just more convenient to express via nondeterminism than via any of the popular iteration libraries, and the performance impact isn't high enough to matter for most cases given Screamer minimizes the amount of code rewriting (some optimizations I have in mind could reduce it even further, but that's dependent on the dependent typing implementation).

More selfishly, in the future Screamer is a useful test-bed for some libraries I'm locally prototyping to make similar "framework"/"compiler" projects easier to write in CL (e.g. one lib encapsulates the function-modeling framework, and ideally would integrate SBCL's call graphs to build it automatically when available).

[fstamour]

Sounds like you might want to fork screamer instead. One big reason being compatibility: people won't use Screamer if their existing code breaks.

The variable system can be extended

I thought about that last year, it would be very interesting. There are many ways the variables could be extended

[swapneils]

Sounds like you might want to fork screamer instead. One big reason being compatibility: people won't use Screamer if their existing code breaks.

One major issue I see in the Common Lisp ecosystem is constant NIH syndrome, and I'm far from the first to notice.

Whenever possible, I think it's more pro-social to extend an existing library with new functionalities instead of forking it or making a 'competitor'. Doing so:

• Reduces the amount of consideration new users need to select libraries
• Assures mutual compatibility and/or clearly-noted caveats (since library authors generally know how their own ASDF systems interact) for those wanting to mix the old and new features
• Provides a central target for additional contributions (and/or to port over features of similar libraries)
• Allows existing users of earlier versions to incorporate new features without difficulty

Compatibility and existing-user migration would admittedly also be served by a well-designed extension library (such as Screamer+), but the benefits to new users and potential contributors are unique to a "one problem, one library" approach.

The problem of different use-cases / prioritizations can generally be solved with specialized sub-modules within the same project, and centralizing these sub-systems doesn't pose significant maintainability issues even at larger scales. For a real example of this approach, see e.g. Doom Emacs' module system. ASDF tracks dependencies between sub-modules if you break them into systems, significantly reducing the complexity of handling large numbers of submodules within the same project.

For Screamer in particular:

So far I've managed to avoid backwards incompatible changes to Screamer, and I don't expect to be forced to make them in the near future:

• The optimizations and bugfixes that stood out so far have been accomplished without changing APIs. The new README specifies backwards-incompatible changes, so far just new symbols and a change to the boolean *nondeterministic-context* which doesn't modify its truthyness or violate any explicit type-declarations in the original code.
  • If something explicitly violates Screamer's contract, e.g. instances of incorrect reasoning in multiple functions, I don't think it's reasonable to consider fixing those a backwards-incompatible change, as users should not have been seeing that behavior in the first place and it's likely undesirable for their use-case. Any maintainer would do the same, even without their own vision for the project.
• Adding new exported symbols is not a compatibility issue (in CL users can trivially redefine them without breakage)
• Library internals and the specific output of code-transformations were never intended to be depended on, which frees us up to modify e.g. the CPS setup (though this will likely be in a new system regardless to allow inlining and CLTL2).
• Future changes to variables specifically can share type info with the current slot-based paradigm, preventing impact here.
  • Fortunately classic Screamer never assumed by default that variables will have one of its types, so we can add more constraints without breaking the existing ones.
• Future changes to solution specifically will most likely be backwards-compatible with custom ordering force functions using the existing format.
• Importing new libraries (as mentioned above) shouldn't impact anyone using Quicklisp for installation, though I do aim to minimize new dependencies for the :screamer system. Those manually managing their packages won't get the upgrade unless they do it manually, and loading :screamer should make any new package requirements clear long before they move to production.

If any new feature changes the current Screamer behavior (e.g. using static-dispatch or polymorphic-functions to not reinvent the type-based optimization wheel), I'll add new not-loaded-by-default sub-modules (e.g. :screamer/experimental is already used for this).

If the above still doesn't work --- e.g. some fundamental limitation to the above extensions disallowing Screamer's use of the condition system or severely limiting its rewriting abilities --- then yeah, I'll probably move to a new package to better organize things, though with the "optional systems" paradigm this technically wouldn't pose a compatibility issue to existing users, just bloat their install with things they can't use.

[swapneils]

I see this isn't marked as canbuild. @quicklisp Could you try re-building https://github.com/swapneils/screamer? ASDF/ql:quickload work without issues, as do make test and make clean.