Problem with encryption of self-security database

[pavel-zotov]

Ticket has been created after talk with Alex.

Copy from %FB_HOME%\examples\prebuilt\plugins\ to %FB_HOME%\plugins\ files: fbSampleDbCrypt.* and fbSampleDbCrypt.*

Add following lines in %FB_HOME%\plugins.conf

Plugin = KH2 {
    Module = $(dir_plugins)/fbSampleKeyHolder
    RegisterName = fbSampleKeyHolder
    Config = KH2
}

Config = KH2 {
    Auto = false
}

Put in firebird.conf following:

AuthServer = Srp, Win_Sspi
BugCheckAbort = 1
ClearGTTAtRetaining = 0
ClientBatchBuffer = 131072
ConnectionIdleTimeout = 0
DefaultDBCachePages = 10K
ExtConnPoolLifeTime = 10
ExtConnPoolSize = 10
ExternalFileAccess = Full
InlineSortThreshold = 1000
IpcName = xnet_fb6x_ss
KeyHolderPlugin = fbSampleKeyHolder
MaxIdentifierByteLength = 252
MaxIdentifierCharLength = 63
MaxParallelWorkers = 8
MaxUnflushedWrites = -1
MaxUnflushedWriteTime = -1
OptimizeForFirstRows = true
ParallelWorkers = 1
ReadConsistency = 0
RemoteServicePort = 3600
ServerMode = Super
SnapshotsMemSize = 64K
StatementTimeout = 300
TempCacheLimit = 1G
TipCacheBlockSize = 4M
UseFileSystemCache = true
UserManager = Srp
WireCrypt = Enabled
WireCryptPlugin = ChaCha, Arc4

Add following alias in databases.conf:

tmp_selfsec_alias = r:\temp\tmp_selfsec.fdb
{
    KeyHolderPlugin = KH2
    SecurityDatabase = tmp_selfsec_alias
    AllowEncryptedSecurityDatabase = true
}

(adjust file name of test database to your env.; this file must not exist)

Restart FB service.

Then do: Scenario-1. (when "AllowEncryptedSecurityDatabase = true", see above databases.conf)

C:\FB\60SS>isql -q
SQL> create database 'tmp_selfsec_alias' user sysdba password 'alterkey';
SQL> create user sysdba password 'alterkey' using plugin Srp;
SQL> commit;
SQL> connect 'localhost:tmp_selfsec_alias' user sysdba password 'alterkey';
Database: 'localhost:tmp_selfsec_alias', User: SYSDBA
SQL> set list on;
SQL> select mon$sec_database from mon$database;
MON$SEC_DATABASE                Self

SQL> select rdb$config_value from rdb$config where rdb$config_name = 'AllowEncryptedSecurityDatabase';
RDB$CONFIG_VALUE                true

SQL> commit;
SQL> alter database encrypt with "fbSampleDbCrypt";
Statement failed, SQLSTATE = 08004
unsuccessful metadata update
-ALTER DATABASE failed
-Missing database encryption key for your attachment
-Plugin fbSampleDbCrypt:
-Crypt key not set
SQL> quit;

Scenario-2. Comment out "AllowEncryptedSecurityDatabase = true" in databases.conf and repeat:

C:\FB\60SS>isql -q
SQL> create database 'tmp_selfsec_alias' user sysdba password 'alterkey';
SQL> create user sysdba password 'alterkey' using plugin Srp;
SQL> commit;
SQL> connect 'localhost:tmp_selfsec_alias' user sysdba password 'alterkey';
Database: 'localhost:tmp_selfsec_alias', User: SYSDBA
SQL> set list on;
SQL> select mon$sec_database from mon$database;
MON$SEC_DATABASE                Self

SQL> select rdb$config_value from rdb$config where rdb$config_name = 'AllowEncryptedSecurityDatabase';
RDB$CONFIG_VALUE                false

SQL> alter database encrypt with "fbSampleDbCrypt";
SQL> commit;
SQL> select current_time from rdb$database;
CURRENT_TIME                    22:17:11.0000 Europe/Moscow
-- ... let us wait until crypt thread complete its work ...
SQL> select current_time from rdb$database;
CURRENT_TIME                    22:20:26.0000 Europe/Moscow

SQL> commit;
SQL> show database;
Database: localhost:tmp_selfsec_alias
        Owner: SYSDBA
. . .
ODS = 14.0
Database encrypted, crypt thread not complete <<<<<<<<<<<<<<<< NOT COMPLETE ?..
Wire crypt plugin: ChaCha
Creation date: Aug 24, 2024 22:16:35
...
SQL> quit;
-- here ISQL hangs: control does not return to OS.

Database after final "quit;" (in 2nd scenario) will really look like 'partially encrypted', see screen:

Output of gstat -h:

Database "r:\temp\tmp_selfsec.fdb"
...
	Implementation		HW=AMD/Intel/x64 little-endian OS=Windows CC=MSVC
	...
	Creation date		Aug 24, 2024 19:16:35
	Attributes		force write, encrypted, crypt process, plugin fbSampleDbCrypt

    Variable header data:
	Crypt checksum:	jMkZlIcVw7OHdX1q/Rvd+BcdgF8=
	Key hash:	BHliPMkWmvaIdustnz8mBtgUEMY=
	Database GUID:	{17B758EF-81AD-4C99-9B2E-4CD23686A8FC}
...

FB service can not be stopped using normal way (only taskkill /f /t /im firebird.exe helps) Dump, stack-trace and FB snapshot (WI-T6.0.0.442 Firebird 6.0 4a68fe2) are here: https://drive.google.com/drive/folders/1LaB7fcbZs-fnRuZzzQPVLEAjoiSvs808?usp=sharing