firebird icon indicating copy to clipboard operation
firebird copied to clipboard

add to gfix -progress switch to show progress of validation operation

Open livius2 opened this issue 1 year ago • 16 comments

Please add to gfix -progress switch to show progression of operation validation of the database if possible.

It can be log into firebird.log by eg. every 10% to not spam too much the log.

Now when we must verify database we do not know where we are in the process...

livius2 avatar Dec 18 '23 19:12 livius2

The progress of what operation?

mrotteveel avatar Dec 19 '23 09:12 mrotteveel

I mean progress of e.g. -validate -full.

livius2 avatar Dec 19 '23 10:12 livius2

I don't like the suggestion that it is output to firebird.log. If you specify an option like that in a commandline tool, I would expect progress to be output to the standard out (or error stream), not to some log file somewhere else.

mrotteveel avatar Dec 19 '23 10:12 mrotteveel

But today gfix write log about errors/warnings to firebird.log. Maybe another switch or change in current behavior that gfix always write to std out. Now when we start validation of multiple databases in the same time, log is mixed, so i like your proposition a lot :-)

livius2 avatar Dec 19 '23 10:12 livius2

Did you considered online validation ?

hvlad avatar Dec 19 '23 10:12 hvlad

Did you considered online validation ?

What about?

livius2 avatar Dec 19 '23 10:12 livius2

But today gfix write log about errors/warnings to firebird.log.

That (IMHO) design mistakes where made in the past, does not mean we should perpetuate them. If a user ask for progress, I expect them to want to see it. Otherwise we could just log the progress in firebird.log always, without having to explicitly ask for it.

mrotteveel avatar Dec 19 '23 10:12 mrotteveel

@mrotteveel I have modified my comment probably when you were writing ;-) Look above "so i like your proposition a lot :-)"

livius2 avatar Dec 19 '23 10:12 livius2

Did you considered online validation ?

What about?

https://firebirdsql.org/file/documentation/release_notes/html/en/2_5/rnfb25-apiods-api.html#rnfb25-apiods-api-onlinevalidation

hvlad avatar Dec 19 '23 10:12 hvlad

@hvlad but what about it? It report progress? Even if so it cannot validate all situation like exclusive access with gfix do? So still gfix progress will help a lot. Yesterday I had to check a very large database and the only thing I could estimate was to take the size of the database and divide it by looking at readings per second from the disk ;-)

livius2 avatar Dec 19 '23 10:12 livius2

@hvlad but what about it? It report progress?

Try and see. I consider - yes, it is reported progress. But you could have another opinion.

Even if so it cannot validate all situation like exclusive access with gfix do?

It validates most cases. The approach could be extended to the offline validation, if required. Or we can add some events for the trace, like it is done for sweep.

So still gfix progress will help a lot. Yesterday I had to check a very large database and the only thing I could estimate was to take the size of the database and divide it by looking at readings per second from the disk ;-)

gfix way to run validation physically can't report any kind of progress. All gfix does is just attach and detach. All validation activity happens inside attach, by the engine. Online vailidation was implemented as a service that could start task and get results (progress).

hvlad avatar Dec 19 '23 11:12 hvlad

Try and see. I consider - yes, it is reported progress. But you could have another opinion.

Do not get my question as some criticism i simply did not catch the context when you wrote about online validation.

It validates most cases. The approach could be extended to the offline validation, if required. Or we can add some events for the trace, like it is done for sweep.

Is there some document about comparision what is checked by online validation vs gfix -validate - full? It will be very helpfull information.

All gfix does is just attach and detach

but it report how many errors, warnings was is it from detach process only?

livius2 avatar Dec 19 '23 12:12 livius2

On 12/19/23 14:55, Vlad Khorsun wrote:

gfix way to run validation physically can't report any kind of progress. All gfix does is just attach and detach. All validation activity happens inside attach, by the engine. Online vailidation was implemented as a service that could start task and get results (progress).

As the simplest (but restricted and bad in all aspects except simplicity) solution we can add progress report at least when gfix is invoked as service. That's true easy. If utility is much wanted to also report progress it's possible via set of callbacks and special OP in remote protocol - approximately same actions like when dbcrypt key is delivered from client during attach. Or change approach of offline validation - run it not on attach but using separate API call with existing attachment.

AlexPeshkoff avatar Dec 19 '23 12:12 AlexPeshkoff

If utility is much wanted to also report progress it's possible via set of callbacks and special OP in remote protocol

Can't gfix utility simply use the service instead?

aafemt avatar Dec 19 '23 12:12 aafemt

On 12/19/23 15:26, Dimitry Sibiryakov wrote:

If utility is much wanted to also report progress it's possible
via set of callbacks and special OP in remote protocol

Can't gfix simply use the service instead?

When working with fresh enough server - why not?

AlexPeshkoff avatar Dec 19 '23 12:12 AlexPeshkoff

Utilities usually are supposed to work with servers of the same version. Ability to work with servers of different versions is just a nice extra feature.

aafemt avatar Dec 19 '23 12:12 aafemt