cannot clear metadata when having source related inconsistency

[scriptonist]

Version Information

Server Version: v2.8.0 CLI Version (for CLI related issue): v2.8.0

Environment

OSS

What is the expected behaviour?

When using hasura metadata clear or Clicking "Reset" metadata on console should clear metadata

Keywords

metadata, clear_metadata

What is the current behaviour?

As a user when I have a source-related inconsistency in my metadata, doing a hasura metadata clear or using the "Reset" metadata button on the console throws an error

How to reproduce the issue?

To replicate this bug:

1. Apply some metadata with a bad source (say a wrong db connection string)
2. Now try to clear metadata (via CLI or console).
3. This clearing or reset will fail
4. Go to the console and then via the inconsistent metadata page, drop inconsistent objects. This will go through.

Screenshots or Screencast

Please provide any traces or logs that could help here.

N/A

Any possible solutions?

Both CLI and Console uses clear_metadata metadata API to implement this functionality. So, probably looking into the implementation details of this API might uncover details of this behaviour.

[tirumaraiselvan]

Two cases :

1. Source is inconsistent in schema cache but available
2. Source is inconsistent in schema cache and not available

Few design questions here:

1. Should we clear metadata irrespective of whether source is available or not?
2. What's the relationship between clear_metadata behaviour and pg_drop_source behaviour?

[rakeshkky]

From initial investigation,

• It is quite clear that even if source is not available, the runPostDropSourceHook, that we do before clearing metadata, does not throw any exception. It just logs the exception, if raised any.
• It is askSourceInfo that is causing the exception mentioned in this issue. We can not afford to call askSourceInfo if the said source is inconsistent.

[rakeshkky]

@tirumaraiselvan

1. Should we clear metadata irrespective of whether source is available or not?

The reason for this issue existence, in the first place, is the user should be able to clear/reset the metadata irrespective of existing inconsistencies. As discussed in the above comment, we aren't raising any exception when the source isn't available.

2. What's the relationship between clear_metadata behaviour and pg_drop_source behaviour?

The pg_drop_source behavior is similar to that of clear_metadata except it is targeting a single source. It (also) drops source even if it is inconsistent unless cascade is not set.

[rakeshkky]

Potential fix:

Before clearing metadata, call runPostDropSourceHook only for consistent sources, i.e., sources tracked in SchemaCache. Leave inconsistent sources.

[codingkarthik]

I think the objective of clear_metadata is for Hasura to provide a way to provide a clean slate and ideally, there should be no hindrance for the user to do this. In our case, to deal with inconsistent sources, we could provide a way to "force" clear a source where if a source is inconsistent we don't do any source post-drop cleanups.