Allowed memory size issue with update_groups_cache

[wo14580]

Hi,

We're encountering a minor issue with the \local_o365\task\coursesync task. Our task_scheduled_max_runtime is configured for 30 minutes, but the script execution time ranges from 20 minutes to over 30 minutes. As a result, Moodle often terminates the script prematurely.

Here are the versions we're currently using:

• Moodle: 4.3.10
• moodle-local_o365: 4.3.6
• Php 8.1

After investigating, we've found that the delay occurs when the script is fetching groups from the Graph API client:

• local/o365/classes/utils.php:583

The while loop in this method is taking too long to complete. This is the exact line of code where the script is looping and ending prematurely: local/o365/classes/rest/unified.php:199

[wo14580]

After further investigation, we determined that the main issue was not as initially suspected. Instead, we encountered a memory size limit issue when updating the group cache. To resolve this, we implemented a method to increase the memory limit within the update_groups_cache function.

update_groups_cache

    public static function update_groups_cache(unified $graphclient, int $baselevel = 0) : bool {
        global $DB;

        static::mtrace("Update groups cache.", $baselevel);

        // This is necessary because the task may process a large number of groups.
        raise_memory_limit(MEMORY_HUGE);
        ...

For us, this is just a workaround, and I'm not sure if it's the best solution. I noticed you’re already using the method to raise the memory limit in the user sync task.

Do you have plans to optimize the script to improve both memory usage and execution time?

[wo14580]

With the recent improvements we made (as explained in my previous comment) the script is working flawlessly but generating a large amount of logs.

Would it be possible to implement log levels in the code (eg: debug, info, error) ?

Because of that, we had to increase temporary our disk space.

We have hundreds of thousands (maybe millions) of lines like this, which are not really relevant. ...... Group ID f7d58448-bc14-4f8b-a773-e628531626ba in cache is up to date.

Thanks in advance for your feedback.

[patmr7]

Hi @wo14580

Thank you for your report and analysis. I have added a task to the queue to analyze deeper issues with the update_groups_cache function.

Best Regards, Patryk