o365-moodle
o365-moodle copied to clipboard
microsoft
User matching problem with Azure OIDC
Hi here,
We have an issue where Moodle users are not being matched with their M365 accounts causing them to receive the following error:
"Invalid login: User not found in Moodle. If this site has the "authpreventaccountcreation" setting enabled, this may mean you need an administrator to create an account for you first"
We are able to manually match users by uploading a csv file in the user matching tool in the plugin's sync settings but can't work out why the user matching is failing in the first place.
Any help greatly appreciated.
For additional context we are using Moodle 3.9 and are on version 3.9.8 of the plugin.
In the logs of the \local_o365\task\usersync tasks here is an example of an affected user:
76791d08c633:20213 ......... Syncing user (M365 username) 76791d08c633:20213 ......... User doesn't exist in Moodle 76791d08c633:20213 ......... Not creating a Moodle user because that sync option is disabled.
Hi @ZadkielSantospirito,
When logging in or running the usersync task, Moodle will try to find a Moodle user whose username matches the UPN of the Microsoft 365 user. Moodle gets the UPN from either Graph API call (when running the usersync task) or from ID token on successful user login (when logging in using auth_oidc plugin).
Could you check the UPN of the user who tries to login, and the username of the Moodle account that is expected to be for the same user, and confirm if they match please.
The error messages in both cases indicate that a Moodle user with username matching the UPN is not found, and while Moodle tries to create a new account with the username, the Moodle option "authpreventaccountcreation" is enabled, so new account cannot be created according to this setting.
Regards, Lai
Hi @weilai-irl,
Thanks for getting back to me.
The issue is that the UPN does exist in Moodle in these cases, the UPN does actually exist in Moodle, it's simply not being picked up during the usersync task. We use the Moodle email address as the field that contains the UPN rather than the Moodle username and those do always match in these cases.
We use a in-house middleware system to create the accounts in Moodle that passes the username and UPN to Moodle using a web service so we do have authpreventaccountcreation enabled (since as far as we understand, that would end up creating duplicate accounts - just to check, is that correct?).
Do you have any suggestions about what might be preventing the UPNs that are in Moodle from being found?
All the best,
Zad
Hi @ZadkielSantospirito,
Since you are matching Microsoft 365 account UPN with Moodle account email instead of username, do you have the "Match Azure usernames to moodle emails instead of moodle usernames during the sync" option checked in the user sync settings? By default matching is only checked between Microsoft UPN and Moodle account username.
Regards, Lai
Hi @weilai-irl ,
Thanks for getting back to me.
Yes, we do have that setting enabled in the sync settings.
Regards, Zad
Hi @ZadkielSantospirito,
I was reading your original reporting of the issue, and it looks that you were able to match Microsoft 365 accounts with Moodle accounts using the "User Matching" advanced tool. Then this is expected.
The "Match Azure usernames to moodle emails instead of moodle usernames during the sync" option in the "User sync" section of the "Sync Settings" tab is only applied when running the user sync scheduled task, and is not considered on user login.
When a user tries to login using their Microsoft 365 account, auth_oidc will check (1A) if the user's UPN is already matched to a Moodle user (in auth_oidc_tokens table), and if not, (1B) if the user's UPN is the same as the username of any existing Moodle user; then if either (1A) and (1B) are matched, it will check (2) if the user uses auth_oidc; if (2) is true, then it will try to login the user. It will not try to match Microsoft 365 account UPN with Moodle account email in this process.
So what you experienced is how the system is designed to work.
Regards, Lai
Hi @weilai-irl,
Thanks for the information. Does this mean that there is no way to run scheduled tasks to match Azure usernames to Moodle emails? If I'm understanding you correctly this means there's only a manual sync possible for the option to match Azure usernames with Moodle emails which seems strange.
We aren't trying to get the users matched on login - we are using the \local_o365\task\usersync scheduled task to match the Moodle usernames and Moodle emails - is that not something that's possible to do?
All the best, Zad
Hi @ZadkielSantospirito,
If what you want to achieve is just to map a Moodle account with a Microsoft 365 account, then either the "User matching" tool or the usersync scheduled task can do it.
When using the usersync scheduled task, assuming you have "Match Azure usernames to moodle emails instead of moodle usernames during the sync" option enabled, consider the following situation:
- You have an existing Moodle user with username "testuser1" and email "[email protected]", using manual authentication.
- You have a Microsoft user with UPN "[email protected]".
- When running the usersync task, the users will be matched, you will see something like the following in the log.
......... Syncing user [email protected]
......... Found a user in Azure AD that seems to match a user in Moodle
......... moodle username: testuser1, aad upn: [email protected]
......... Matched user, but did not switch them to OIDC.
......... Adding o365 object record for user.
......... User is now synced.
This will not change the authentication method of the user, so the user still needs to login using their Moodle username and password. Upon login, in the Microsoft block, the user will see a text "You are almost connected to Microsoft 365". The user will need to click the link in the block and login in the popup window to finish the connection, but even without it, the users are mapped.
Regards, Lai
Hi @weilai-irl,
The issue is that we are using OIDC for authentication so we need the users to be matched before before they log in.
Not sure if it helps but here's an example of the log output showing both a user that is matched and a user where matching has failed (I've replaced the actual upns as they are actual email addresses for data protection purposes).
ff71b7d5f0d5:9634 ......... Syncing user
ff71b7d5f0d5:9634 ......... Syncing user
While the logs say that the second user doesn't exist in Moodle, they can be found when searching under the user search if the email field is used for filtering and their UPN is put in for the email address.
Hoping this can help find what's not set up right.
All the best,
Zad
Hi @weilai-irl,
To try to help you understand the problem, I'm attaching a screengrab of our settings:
Hope this helps!
All the best,
Zad
Hi @weilai-irl,
Hope you're well.
Do you have any advice that can help us with our user matching issue? Any help will be very welcome indeed.
All the best,
Zad
Hi @ZadkielSantospirito,
Sorry for not getting back to you. I was a bit busy last week.
I ran a test in your scenario and it generates different results. Here are my details:
- In Moodle, I updated the user sync settings as set in your screenshot, and temporarily disabled the user sync task.
- In Moodle, I created an account using auth_oidc, with username "user1" and email address"[email protected]".
- In Microsoft, I created an account with UPN "[email protected]".
- I ran the user sync task in Moodle, and it shows the user is matched.
......... Syncing user [email protected]
......... The user is already using OIDC for authentication.
......... Adding o365 object record for user.
......... User is now synced.
......... Updating Moodle user data from Azure AD user data.
......... User is now updated.
Note compared to other already matched users, it contains a line "Adding o365 object record for user".
- I then confirmed the user can log in.
If this still doesn't work for you, could you confirm the Microsoft 365 user UPN and Moodle user name are example match, even for letter cases please.
Regards, Lai
Hi,
The mentioned test case was a lucky path due to the username still being equal to the first part of the UPN. Further down the method existing users are only identified by comparing all username variants (UPN, lower case UPN, UPN without @ and domain part, etc.) against the array's keys which are only usernames, no matter the setting to compare UPNs against Moodle mail addresses instead. The fix seems simple: just make the mail address the first selected column, so the returned array is indexed by Moodle user's mail addresses.
The only TODO remaining after my PRs concerns installations with (Moodle) setting allowaccountssameemail set and multiple users sharing the same mail address which can lead to problems for unique matching of users.
Regards, Philipp
@phager-at
Hi Philipp,
Thank you for making the pull request. What you described makes perfect sense. I'll include the PR in the next release after review and test.
Regards, Lai
Hi @weilai-irl and @phager-at,
Thanks for the attention on this issue. Is this believed to be able to fi our issue? Our UPNs are all uniform, are all lowercase, all contain @ and there are no instances that have been found in the unlinked accounts where duplicate emails have been the problem.
All the best,
Zad
Hi @ZadkielSantospirito,
Please test out the latest release on your site (ideally in a test environment first, of course) to see if the changes helps. Please report back if it works.
@phager-at Thank you again for your contribution in this one.
Regards, Lai
Hi @ZadkielSantospirito,
Please test out the latest release on your site (ideally in a test environment first, of course) to see if the changes helps. Please report back if it works.
@phager-at Thank you again for your contribution in this one.
Regards, Lai
Glad I could be helpful!
Hi @weilai-irl,
We've asked our Moodle hosts to upgrade our staging site to the latest version of the plugin however version 4.0.1 requires Moodle 4.0 and we are on 3.9.
There was also a 3.9.10 version released at the same time - can you confirm that this version also contains the relevant fix?
Many thanks,
Zad
Hi @ZadkielSantospirito,
I confirm the latest release 3.9.10 for Moodle 3.9 contains the same fix.
Regards, Lai
Hi @weilai-irl,
We've had our Moodle hosts apply the 3.9.10 fix on our staging site however it has not fixed our issue.
What might be worth mentioning is that we discovered during our testing that students were being properly synched and that it was only staff that are not synching correctly.
The reason this might be significant is that staff accounts don't have their Moodle username as the first part of their UPN as opposed to students who do. Because of this we were very hopeful that the update would address the problem but for some still unknown reason it has not.
It seems like our issue does have some relation to the problem identified by @phager-at however there is still some issue remaining preventing the \local_o365\task\usersync task from properly matching staff accounts.
Our staff UPNs are in the format [initial].[surname]@bbk.ac.uk (so for instance my UPN is [email protected]) and our student UPNs are in the format [username]@student.bbk.ac.uk.
Hoping this information will help with solving the problem.
Thanks in advance,
Zad
Hi @weilai-irl,
Just following up on my previous message. We've now released the updated plugin to our production environment and it still hasn't resolved the issues we're having. Is there any advice you can offer? More than happy to provide any additional information you need.
All the best,
Zad
Hi @ZadkielSantospirito,
Could you provide some more information to help me performing more investigation on this please. The information I need are:
- The Microsoft 365 account UPN and email for a user that has this issue.
- For the Moodle account that's supposed to be matched to the Microsoft account in question, I would need its username and email address. Take a note at the user ID too - I won't need it, but you will for the other information.
- Find if there is a matching record in the
mdl_local_o365_objectstable, by runningSELECT * FROM mdl_local_o365_object WHERE type='user' AND o365name LIKE "[MICROSOFT ACCOUNT UPN]";in your Moodle database, where [MICROSOFT ACCOUNT UPN] is the UPN of your Microsoft account. This should return exactly one record, with themoodleidfield matching the Moodle user ID. Please confirm. - Find if there is a matching record in the
mdl_auth_oidc_tokentab, by runningSELECT * FROM mdl_auth_oidc_token WHERE oidcusername LIKE "[MICROSOFT ACCOUNT UPN]";in your Moodle database, where [MICROSOFT ACCOUNT UPN] is the UPN of your Microsoft account. This query may return one or zero record. Please post back the result.
Regards, Lai
Hi @weilai-irl
Thanks for getting back to me.
The user's UPN (and username) is [email protected]. I've run the queries on the database and neither table returns a result for the user. I've run control tests with a successfully matched user and the queries return the result expected.
Hopefully this helps identify what's going on.
All the best,
Zad
Hi @ZadkielSantospirito,
Could you provide the details of the Moodle user that's expected to be matched with the named Microsoft 365 user ([email protected]) please. I'll need her:
- username, and
- email address
If this is still doesn't work, I'd like to arrange a quick call with you, as it might be easier to figure the issue that way.
Regards, Lai
Hi @weilai-irl,
No problem, the user's username is ublgre006 and their email address is the same as the UPN (if you remember we're using the option "Match Azure usernames to Moodle emails instead of Moodle usernames during the sync").
If that doesn't work a quick call would be great. Let me know what you'd need from me to set that up.
All the best,
Zad
Hi @ZadkielSantospirito,
I ran a test in my dev environment based on your answer, but it still gave different results as yours. I think a call might be easier.
Please send me an email to arrange a call so that we can walk through this together. My email can be found in my user profile.
Regards, Lai
After a call, we found out that this is caused by the user sync task falls back to use username, rather than email address as configured, as the key of the existing user cache, because it finds duplicate active Moodle accounts with duplicate email addresses. The logic is at https://github.com/microsoft/o365-moodle/blob/5786739652cb444752e3944c3382827b318420b8/local/o365/classes/feature/usersync/main.php#L982-L987.
The current logic is if such duplicate accounts exist, then the complete existing user cache falls back to use username. This is too broad and need to be updated to reduce the impact of the fallback.
Lai
Hi @ZadkielSantospirito,
I have provided a fix to the issue that we found. Please apply the fix to your test environment and give it a try.
Regards, Lai
Hi @weilai-irl,
Thanks for this - we really appreciate your help.
Could I just clarify whether the intention is for this fix to go into the main plugin at some point or would be be relying on staying on a forked version moving forwards?
All the best,
Zad