Missing namespace config results in not helpful error

Describe the bug

I'm trying to run the service that I installed from brew on Mac OS I've got an error: unable to initialize config provider: bad file descriptor.

Reproducing the bug

Start Keto without namespace config. Defaults to ./keto_namespaces, iirc, which then gives you the error.

Server logs

unable to initialize config provider: bad file descriptor

Server configuration

dsn: memory

serve:
  read:
    host: localhost
    port: 9922
  write:
    host: localhost
    port: 9922

Expected behavior

At least a helpful error.

Additional context

Suspected culprit: https://github.com/ory/keto/blob/f8af777f0b2040aa00fe65a33ad0269595232124/internal/driver/config/namespace_watcher.go#L60

cc @rustamgaifullin

Hello folks, Let me give a little bit of a context first. I'm trying to run keto that was installed from brew on Mac OS(System Version: macOS 11.3.1 (20E241) Kernel Version: Darwin 20.4.0) I created keto.yml file in my home directory ~/keto.yml and fill it with the example from https://github.com/ory/keto/blob/master/contrib/cat-videos-example/keto.yml

When I run it using keto serve or keto serve -c ~/keto.yml (including /Users/user.name/keto.yml) I've got: unable to initialize config provider: bad file descriptor However, if I use different directory e.g. keto server -c ~/some_directory/keto.yml it works fine.

Let me know if you need some additional information.

Hi @zepatrik! 👋 thanks for the detailed description. I have been following Keto project for a while. Love it! I am interested in contributing to the project and this issue looks like a good one to get started. Just wondering if anyone has already worked on this and if it sounds good to let me take it.

Sure, go ahead. I also think that this is a good one to start :+1: I don't know of anyone working on this.

Hi @zepatrik I have been studying the Zanzibar document and this made me more interested in keto, I don't know if this issue has been resolved, I would like to start learning more and contributing to this project.

Probably, I can pick this if it hasn't been done?

Yes, just try it and open a draft PR to ask for help if you get stuck.

@zepatrik This is what I discovered while trying to recreate the bug on Mac OS (BigSur), the fsnotify module tries to Add directories watcher.Add(dir) and files in the $HOME of the user

if watchDir {
	if err := w.watchDirectoryFiles(name); err != nil {
		return "", err
	}
}

, most likely there are some files it can't handle which resulted in bad file descriptor error, it kinds of depends on the $HOME environment of the user.

I created a directory in my $HOME and used it as the path to the keto.yml file -c ~/<created-directory>/keto.yaml, everything worked fine because there was no other file to add to the watcher in the directory.

My suggestions: is it possible to suggest to the user to possibly just have a directory where maybe the keto.yaml exists if the file is in the $HOME directory or further, suggest to them in the error message to try to ensure the condition is met?

Another approach that worked, instead of watching the directory as it was done in the watcherx package in the ory/x module, we can watch the file.

//dir := filepath.Dir(file)
if err := watcher.Add(file); err != nil {
    close(c)
    return nil, errors.WithStack(err)
}

What do you think?

We watch the directory of the file because we need a kubernetes configmap proof filewatcher: https://github.com/ory/x/tree/master/watcherx/integrationtest, so just watching a file is not an option. Can you specify what errors you see exactly when watching $HOME? I don't think it should give you a bad file descriptor, as we don't open the files, but only the directory and config file. Would be interesting to see the exact error and permissions on your $HOME. The original error is I think related to the default fallback of $HOME/namespaces/ not being available, so the fix would be to look up that path and then print a meaningful error in that case.

Also, fsnotify says it uses kqueue to watch files on OSX, maybe you can try to use that directly to watch $HOME and see if you get more details?

Hello contributors!

I am marking this issue as stale as it has not received any engagement from the community or maintainers for a year. That does not imply that the issue has no merit! If you feel strongly about this issue

• open a PR referencing and resolving the issue;
• leave a comment on it and discuss ideas on how you could contribute towards resolving it;
• leave a comment and describe in detail why this issue is critical for your use case;
• open a new issue with updated details and a plan for resolving the issue.

Throughout its lifetime, Ory has received over 10.000 issues and PRs. To sustain that growth, we need to prioritize and focus on issues that are important to the community. A good indication of importance, and thus priority, is activity on a topic.

Unfortunately, burnout has become a topic of concern amongst open-source projects.

It can lead to severe personal and health issues as well as opening catastrophic attack vectors.

The motivation for this automation is to help prioritize issues in the backlog and not ignore, reject, or belittle anyone.

If this issue was marked as stale erroneously you can exempt it by adding the backlog label, assigning someone, or setting a milestone for it.

Thank you for your understanding and to anyone who participated in the conversation! And as written above, please do participate in the conversation if this topic is important to you!

Thank you 🙏✌️

Hi, I researched this issue, and here are my findings.

I reproduced the problem with macOS Monterey 12.5.1 / Homebrew 3.6.16, but I was also able to get the bad file descriptor when using keto serve -c /Users/keto.yml where keto.yml did not have read permissions for the user.

When installing keto from the source codes; Users home directory works as expected and the file without read permission gives a good error unable to initialize config provider: open /Users/keto.yml: permission denied

I am not familiar with how homebrew works and if there could be some configuration to make $HOME work. Another way could be to go to the source that gives this error ory x configx/provider.go to check before if provider.(*KoanfFile) is readable and through that could give a proper error for the user when running in homebrew.

I have a similar problem right now:

unable to initialize service registry: stat ./keto_namespaces: no such file or directory
Error: unable to initialize service registry: stat ./keto_namespaces: no such file or directory
unable to initialize service registry: stat ./keto_namespaces: no such file or directory

I try to run the sample from the repository on Mac OS but the service does not start

P.S. add namespace to config - fix

namespaces:
  - name: my_app
    id: 1

Hello contributors!

I am marking this issue as stale as it has not received any engagement from the community or maintainers for a year. That does not imply that the issue has no merit! If you feel strongly about this issue

• open a PR referencing and resolving the issue;
• leave a comment on it and discuss ideas on how you could contribute towards resolving it;
• leave a comment and describe in detail why this issue is critical for your use case;
• open a new issue with updated details and a plan for resolving the issue.

Throughout its lifetime, Ory has received over 10.000 issues and PRs. To sustain that growth, we need to prioritize and focus on issues that are important to the community. A good indication of importance, and thus priority, is activity on a topic.

Unfortunately, burnout has become a topic of concern amongst open-source projects.

It can lead to severe personal and health issues as well as opening catastrophic attack vectors.

The motivation for this automation is to help prioritize issues in the backlog and not ignore, reject, or belittle anyone.

If this issue was marked as stale erroneously you can exempt it by adding the backlog label, assigning someone, or setting a milestone for it.

Thank you for your understanding and to anyone who participated in the conversation! And as written above, please do participate in the conversation if this topic is important to you!

Thank you 🙏✌️