Proposal: Add Authentication to API Layer

[KevinMellott91]

We would like to add an authentication layer to the Marquez API, so that we can natively secure calls to the API endpoints using techniques like JWT, oAuth, LDAP, etc.

The Apache Shiro framework is the proposed solution, since it's a highly configurable Java component that contains a large number of integrations. This framework's components includes Authentication, Authorization, Session Management, and Cryptography.

Future work could utilize the Authentication component to allow for "read" roles (explore the metadata) vs "write" roles (publish OpenLineage events). However, for the sake of this Issue I am only proposing basic authentication functionality.

Related functionality was created as part of Issue #817, this proposal differs in that it aims to implement strict authentication behavior that can be enabled or disabled at the time of deployment. To keep onboarding simple, this would be disabled by default.

[julienledem]

Is this still up to date or are you thinking of adding authorization instead?

[wslulciuc]

My vote would be for using Shiro given it's configurability (as highlighted by @KevinMellott91)

[gage-russell]

@wslulciuc @harels resuming auth-related slack thread here --

I'll preface with: Shamefully, I am not a Java guy and might not be all that helpful in driving this issue .. but will at the very least contribute in reviews and commentary as I think this could be a road-blocking feature for many potential users.

Maybe we can make this issue a bit more palatable by adding some scope. I'll naively list out what I am picture as the scope of work required to resolve this issue:

• Add optionally enabled authentication layer to Marquez API
  • Shiro seems to be implementation choice by popular vote
• Add optionally enabled login page to Marquez UI
  • does "optionally enabled" make sense here. Not being a web-developer, having a hard time picture this UI component as optional based on its enablement in the backend
• Update Marquez helm chart to enable/disable authentication layer

Apologies if this isn't helpful commentary. Mostly wanting to just paint a picture of what all this issue entails.

[sarlife360]

+1

[davidsharp7]

• 1