Would it make sense to merge your canges back into the repo you forked from?

[MHumm]

I do guess that you forked from this one: https://github.com/Winkelsdorf/DelphiEncryptionCompendium

Which got quite some changes over the last few years to make it cross platform compatible, fix bugs, transform the test program into proper DUnit/DUnitX unit tests etc.

Would it make sense to check what you changed from the original to make it better FPC compatible and join forces in the one above? (I'm asking because I'm currently the main contributor to the one above) If you like you can find my e-mail address in the notices.txt over there (the one marked as main one) and contact me via that one.

[decfpc]

I didn't fork any existing repository. Unfortunately, much of the development of DEC in the past decade happened in private, at least it seemed to me that way back in 2017. At that time, there was no public git repository anywhere to be found. I used v5.2 as the base for my work which was released in late 2008.

In 2017 I was working on a project for my set-top-box which required cryptography. I was very familiar with DEC, thus I decided to port it to FreePascal and share my efforts with other people who might find it useful. The box is an embedded ARM-based system with only 32 MB of flash memory, so size is a concern. I do think that overall the older library is a better fit for it.

There are many things that I dislike about newer Delphi versions, especially their misguided next-gen compiler. Just look at the mess you were forced to make of DECFormat.pas. XMLDoc makes for pretty tooltips in the IDE, but it's also an incredibly bloated form of documentation that makes class interfaces hard to navigate in the code (or maybe modern Delphi has some auto-folding for it that I'm not aware of). DUnit/DUnitX is certainly fancy and more in line with today's software engineering practices, but when I compare them to the "old arcane test program" (which I reworked a bit to be less arcane), I can't help but notice that they too are very verbose compared to the declarative nature of the old test vector file.

I think the visions our two projects follow are a bit different. I'm focused on legacy & cross-platform, you're focused on modern software engineering, new algorithms, etc. In my opinion, that's fine - we can co-exist. I'm not planning on making any big additions here, I'm just going to maintain the state the library is in right now. I've already alerted you about some cipher bugs and cherry-picked a couple fixes from your repository. If you find and fix an issue that's likely been there for decades, I'd appreciate it if you'd drop a note here.

I'll add a note at the top of my README that there's a newer version people might be interested in, depending on their needs. Other than that I'll soon file a pull request in your repo with some improvements I've come up with. When I find the motivation I'll also help with adding additional unit tests with confirmed/official vectors (where available). I hope we won't find more esoteric ciphers that have always been bugged in this library, but given what I've seen so far I wouldn't rule it out 😛

[MHumm]

I can understand your reasoning and given your hardware specs I see your points. Yes, since 5.2 the development mostly was private by the former maintainer (which was not Haagen), but (something I only noticed later) with knowledge of the German Delphi community as he sometimes had asked questions there and got some help from there with some details.

About that Nexgen compiler: I took the development of the library over because I had need for this on mobile as I had used this for a desktop software already and wanted to reuse the encrypted file format. Luckily the former maintainer had created all those Pascal ports of the algorithms already, but there was still enough work left to do. A few years later, Embarcadero changed their mind a bit about Nextgen, especially about memory management (which didn't bother me with DEC because ARC was mostly compatible with the old way if you did things like you had done for the old way anyway) and that 0-based strings problem is only forced in one situation on developers without being able to turn it off: when using the helper classes like MyString.Remove(3, 5). But since they were introduced after D2009 and DEC currently specifies D2009 as minimum version we cannot use it yet anyway. That might change in the future when we increase minimum supported version.

About XMLDOC: there is a request to be able to put that into the implementation section instead of the interface one, but that has not high priority. One could remedy the situation by putting {$region} {$endregion} around them so one can colapse/fold them. I didn't bother yet, but this might be some idea.

I once tried to understand that test program, but I've given up, given my spare time is limited as well. The pro side of using DUnit is that there is tooling around it, like Stefan Glienke's TestInsight. That's a Delphi IDE plugin which when enabled can run the unit tests and show the results in an IDE panel. But the best of it is: you can configure it in such a way, that it runs the tests automatically on each save or after a defined time and these runs are in the background so you can keep working and see immediately if your last change broke some test. And yes: I'm focussed on modern software engineering and adding more algorithms, especially more modern ones. So we can definitively coexist and your name is already in DEC's notice.txt file due to your contribution fixing that SCOP failure. Where are your test vectors for that one from? Did you translate that C program given in their paper? And of course I try to remember informing about issues in older algorithms found so you may port it over. I'd be happy about further contributions as well and I guess I should start to check the existing test vectors systematically.

We're currently implementing SHA3 if you didn't already notice, but we still have some crash on Android. But this time we implemented all official test vectors specified by NIST. I wrote a generator which takes their data files and produces DEC unit test compatible output I can just copy and paste. The downside of this is, that the unit with these tests (I've extracted the SHA3 tests into their own unit because of the size) is about 10 MB in size alone ;-)

I simply want a good future for this library! Oh and if you're an Android phone you can search for DEC cipher demo and DEC hash demo on Google Play. I published these two FMX based demo programs there for free. ;-) (yes, still with the broken SCOP cipher as I didn't find time to update these yet.

[MHumm]

Another topic: did you notice that I tried to start implementing unit tests for the individual cipher block chaining algorithms? I got stuck on that as I'm no expert in those algorithms and as I was unsure how to implement tests for some in such a way that the data used really tests the differences in the methods. Do you have any ideas about that area or any interest to cooperate there? (you could add such tests, if useful to your use, into that test program reading the vectors from a text file).

[MHumm]

Two additional notes:

1. I just found out that you can collapse all these XMLDOC comments in the Delphi IDE via view menu. They get 1-liners each then. Only issue is that the default coloring (at least in dark theme) sticks out way too much. One can partly change that but not get rid of the still sticking out frame around these collapsed areas.
2. If you look at my recent commits I started to revisit the test vectors used in order to use the "official" ones where I can find them. I started with the hash algorithms and found out that some already used the official ones. For MD2 I added the official ones but I'm not half done with that yet. After the hashes the ciphers shall follow but both is a daunting task and might be done in parallel with other outstanding tasks. I also add comments pointing to the sources for those vectors in the unit tests so if anybody is interested he/she can check the source. (use the source Luke!)

[MHumm]

Just to let you know that I released a V6.3 yesterday. Main new features are support for Shake128 and Shake256 hash algorithms and removing some comment which deactivated use of optimized 32 bit asm implementations of MD2, SHA384, Tiger and Snefru128/256 hash algorithms. Details of the other changes see VersionHistory.pdf in docs subfolder.

[MHumm]

Just to let you know that a GCM block chaining mode implementation for 128 bit block sized ciphers is pretty far implemented in my version already. It passes the official unit tests but there's one open question I'm discussing with the donator of the original code and I should add a few unit tests for some new methods which raise exceptions when called on ciphers with an active mode other than GCM. A bugfix in my CalcStream implementation for hashes (only affecting hashes where the last byte can be incomplete, currently the SHA3 family only in DEC) when called with size < 0 has been implemented as well.