feat: appwrite adapter

[sourabpramanik]

Covered most of the cases and methods. Follow the to-do to know about the upcoming changes:

• [x] Docker setup for testing
• [x] Shell script to automate initialization
• [x] Fix Verification token methods
• [x] Add detailed steps for using the adapter, and process to set up the database, collections, and attributes.
• [x] Clean up

☕️ Reasoning

This PR is for adding the Appwrite adapter for auth.js and next-auth.js

🧢 Checklist

• [x] Documentation
• [x] Tests
• [x] Ready to be merged

🎫 Affected issues

Feat #7709

📌 Resources

• Security guidelines
• Contributing guidelines
• Code of conduct
• Contributing to Open Source

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
auth-docs	❌ Failed (Inspect)			Aug 2, 2024 3:43pm

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
next-auth-docs	⬜️ Ignored (Inspect)	Visit Preview		Aug 2, 2024 3:43pm

[vercel[bot]]

@sourabpramanik is attempting to deploy a commit to the authjs Team on Vercel.

A member of the Team first needs to authorize it.

[ndom91]

Hey @sourabpramanik thanks for the contribution! Without having taken a closer look at the code, it seems like you've got a lot of the basics covered. As you mentioned in your todo list above, a testing setup is required.

I'd encourage you to first take a look at our upcoming "creating a database adapter guidelines" and double check you've got everythign covered :pray:

Thanks!

[sourabpramanik]

Hey @sourabpramanik thanks for the contribution! Without having taken a closer look at the code, it seems like you've got a lot of the basics covered. As you mentioned in your todo list above, a testing setup is required.

I'd encourage you to first take a look at our upcoming "creating a database adapter guidelines" and double check you've got everythign covered 🙏

Thanks!

Yes, only thing that is left is to add the documentation to explain the structure of the database and the test script. The test works fine when you have the following:

• Appwrite cloud account
• Existing project in appwrite So a fully automated script that can create a project is not possible without having an account. What should I do?

I have tested this locally all the test cases were passing and created a NextJS project with NextAuth to use this Adapter with the GitHub provider and Email provider, there too this adapter works well. The logic is working fine just stuck at writing the test script for it to automate.

[ndom91]

Is there a local dev package for the appwrite maybe? Like the "firebase local emulator" in the case if firestore 🤔

Having a set of tests for which we need a cloud account is not something we can accept unfortunately

[sourabpramanik]

No there's no other option I could find. But I am talking to the Appwrite team about this. Let's see.

[sourabpramanik]

@ndom91 the Upstash Redis adapter has a test dependency on Redis URL and Key similar so if that can work then the Appwrite adapter can also work right. For appwrite we will need client Id, endpoint, and API secret.

[ndom91]

@ndom91 the Upstash Redis adapter has a test dependency on Redis URL and Key similar so if that can work then the Appwrite adapter can also work right. For appwrite we will need client Id, endpoint, and API secret.

Hmm odd, i don't remember ever setting an upstash api key anywhere locally. They even reference a local-only alternative for testing / development in their docs (https://upstash.com/docs/oss/sdks/ts/redis/developing).

Let us know if you find a solution 🙏 I'll talk to the other folks on the team about what they think in the mean time

[sourabpramanik]

Sure I am working on it. Thankyou

[ndom91]

So we really want to avoid another cloud provider requirement for tests. The upstash one has been nothing but a pain in the butt, in fact i just refactored it out after i found the serverless-redis-http recommendation from them while researching this haha. See https://github.com/nextauthjs/next-auth/pull/10033

Anyway, we're totally happy to accept new adapters like this, but please no cloud requirements for testing 🙏

[sourabpramanik]

Yes it does make sense to automate such an extra process so that we can only focus on the parts we care about.

I will raise a request about it in today's office hours and let's see if there's a way we can avoid using the cloud. If not then all the hard work will be for nothing at all 🥲

[DH-555]

So we really want to avoid another cloud provider requirement for tests. The upstash one has been nothing but a pain in the butt, in fact i just refactored it out after i found the serverless-redis-http recommendation from them while researching this haha. See #10033

Anyway, we're totally happy to accept new adapters like this, but please no cloud requirements for testing 🙏

@ndom91 Appwrite contributor here. Appwrite is open source so it can be Self-hosted very easily in your own machine or in your own small VPS with a single command after installing docker: https://appwrite.io/docs/advanced/self-hosting

Is that enough for this case?

Thanks!

[sourabpramanik]

Hello @ndom91 so I have come to a conclusion that at this moment there is no other way to create a project in Appwrite(cloud/console) without having an account, meaning we need an account with a project in Appwrite to test this adapter. Let me know what should we do.

[ndom91]

Hmm yeah so a lot of adapters spin up a container of something to test against.

Would that be an option? You mentioned something about programmatic account creation might be difficult, but is there a flag to set a root acct pw on the container? Maybe via env var?

[sourabpramanik]

Yes I tried using a container as well but still we need to create an account to create a project, and generate an API key. We cannot add root user creds in the env as of now

[socket-security[bot]]

👍 Dependency issues cleared. Learn more about Socket for GitHub ↗︎

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

View full report↗︎

[sourabpramanik]

@ndom91 I have finally figured out the way to spin up the docker container and test the adapter. If you can please check everything then I can start working on the documentation.

[sourabpramanik]

@ndom91 @ThangHuuVu can you please review this PR?

[socket-security[bot]]

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher

View full report↗︎

[ndom91]

Hope you don't mind, I pushed a small commit adding the adapter to the turbo.json turborepo config and updated the pnpm-lock.yaml with the appwrite-adapter package and dependencies.

Anyway, jesus this requires a lot of containers :sweat_smile:

Unfortunately it didn't pass for me. It seemed to fail creating the account during setup.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 39.48%. Comparing base (e87a0b6) to head (13c4b69).

:exclamation: Current head 13c4b69 differs from pull request most recent head d89f830

Please upload reports for the commit d89f830 to get more accurate results.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10000      +/-   ##
==========================================
- Coverage   39.92%   39.48%   -0.45%     
==========================================
  Files         179      171       -8     
  Lines       28671    27320    -1351     
  Branches     1243     1168      -75     
==========================================
- Hits        11446    10786     -660     
+ Misses      17225    16534     -691     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[sourabpramanik]

Hope you don't mind, I pushed a small commit adding the adapter to the turbo.json turborepo config and updated the pnpm-lock.yaml with the appwrite-adapter package and dependencies.

Anyway, jesus this requires a lot of containers :sweat_smile:

Unfortunately it didn't pass for me. It seemed to fail creating the account during setup.

Not a problem

I will check maybe some issues with turbo. I used bun so that's why it worked out for me. Anyways thanks for the update I will fix it

[PineappleIOnic]

Hey There 👋 Appwrite Engineer here, Is there any way I can be of service to help push this forwards?

With regards to the amount of containers Appwrite has, since we're only testing Auth here we can probably get away with just the appwrite, traefik, appwrite-worker-databases and the mariadb containers for the context of these tests. I can fork and test this locally real quick

[sourabpramanik]

Hey There 👋 Appwrite Engineer here, Is there any way I can be of service to help push this forwards?

With regards to the amount of containers Appwrite has, since we're only testing Auth here we can probably get away with just the appwrite, traefik, appwrite-worker-databases and the mariadb containers for the context of these tests. I can fork and test this locally real quick

Oh yes, I was very close but had some other priority tasks at hand. You can definitely work on this and maybe I can join back next weekend

[PineappleIOnic]

Thank you for all the work you've already done! I'm currently working on getting the tests to work in Turbo, I've also removed most of the containers we do not need. The main reason tests were not working in Turbo were due to environment variables that Appwrite needs not being set so I've just hard coded them in the compose file. I will have to check this is the right way to do it with one of the maintainers though.

[sourabpramanik]

Thank you for all the work you've already done! I'm currently working on getting the tests to work in Turbo, I've also removed most of the containers we do not need. The main reason tests were not working in Turbo were due to environment variables that Appwrite needs not being set so I've just hard coded them in the compose file. I will have to check this is the right way to do it with one of the maintainers though.

You are amazing! Thanks man

[PineappleIOnic]

Hey @sourabpramanik I got the tests working for the most part on my machine except for some reason this line:

 if [ -f .env ]; then
    sed -i "s/API_KEY_SECRET=.*/API_KEY_SECRET=$secret/" .env || echo "API_KEY_SECRET=$secret" >>.env
    exit 0
 fi

Doesn't seem to work on my machine correctly, giving me this error:

appwrite-adapter:test: sed: 1: ".env": invalid command code .

What OS/Distro are you using for your tests?

One more question as well, this adapter is using only appwrite database. How come it's not built for Appwrite Auth?

[sourabpramanik]

Hey @sourabpramanik I got the tests working for the most part on my machine except for some reason this line:

 if [ -f .env ]; then
    sed -i "s/API_KEY_SECRET=.*/API_KEY_SECRET=$secret/" .env || echo "API_KEY_SECRET=$secret" >>.env
    exit 0
 fi

Doesn't seem to work on my machine correctly, giving me this error:

appwrite-adapter:test: sed: 1: ".env": invalid command code .

What OS/Distro are you using for your tests?

One more question as well, this adapter is using only appwrite database. How come it's not built for Appwrite Auth?

I am using the Arch Linux machine with a bash terminal. I will look into it.

When I started this project, Appwrite auth was not released at that time. But since it is out now, do you have any plans to use that? I am up for it

[PineappleIOnic]

Hey @sourabpramanik I got the tests working for the most part on my machine except for some reason this line:

 if [ -f .env ]; then
    sed -i "s/API_KEY_SECRET=.*/API_KEY_SECRET=$secret/" .env || echo "API_KEY_SECRET=$secret" >>.env
    exit 0
 fi

Doesn't seem to work on my machine correctly, giving me this error:

appwrite-adapter:test: sed: 1: ".env": invalid command code .

What OS/Distro are you using for your tests? One more question as well, this adapter is using only appwrite database. How come it's not built for Appwrite Auth?

I am using the Arch Linux machine with a bash terminal. I will look into it.

When I started this project, Appwrite auth was not released at that time. But since it is out now, do you have any plans to use that? I am up for it

Ah I see, I checked the other adapters and they all use the database so we'll stick with that for now. I managed to solve the sed problem by using this code instead as GNU Sed and MacOS's Sed is different:

echo "API Key Secret created successfully, writing .env file"
if [ -f .env ]; then
    # Detect the operating system
    if [[ "$OSTYPE" == "darwin"* ]]; then
        # macOS (BSD sed)
        sed -i '' "s/API_KEY_SECRET=.*/API_KEY_SECRET=$secret/" .env || echo "API_KEY_SECRET=$secret" >>.env
    else
        # Linux (GNU sed)
        sed -i "s/API_KEY_SECRET=.*/API_KEY_SECRET=$secret/" .env || echo "API_KEY_SECRET=$secret" >>.env
    fi
    exit 0
fi

[sourabpramanik]

Hey @sourabpramanik I got the tests working for the most part on my machine except for some reason this line:

 if [ -f .env ]; then
    sed -i "s/API_KEY_SECRET=.*/API_KEY_SECRET=$secret/" .env || echo "API_KEY_SECRET=$secret" >>.env
    exit 0
 fi

Doesn't seem to work on my machine correctly, giving me this error:

appwrite-adapter:test: sed: 1: ".env": invalid command code .

What OS/Distro are you using for your tests? One more question as well, this adapter is using only appwrite database. How come it's not built for Appwrite Auth?

I am using the Arch Linux machine with a bash terminal. I will look into it.

When I started this project, Appwrite auth was not released at that time. But since it is out now, do you have any plans to use that? I am up for it

Ah I see, I checked the other adapters and they all use the database so we'll stick with that for now. I managed to solve the sed problem by using this code instead as GNU Sed and MacOS's Sed is different:

echo "API Key Secret created successfully, writing .env file"
if [ -f .env ]; then
    # Detect the operating system
    if [[ "$OSTYPE" == "darwin"* ]]; then
        # macOS (BSD sed)
        sed -i '' "s/API_KEY_SECRET=.*/API_KEY_SECRET=$secret/" .env || echo "API_KEY_SECRET=$secret" >>.env
    else
        # Linux (GNU sed)
        sed -i "s/API_KEY_SECRET=.*/API_KEY_SECRET=$secret/" .env || echo "API_KEY_SECRET=$secret" >>.env
    fi
    exit 0
fi

That's great 👍