New command: Get all the Microsoft Teams messages for the given user

[rabwill]

Usage

m365 teams user message list [options]

Description

Get all Microsoft Teams messages for the given user

Options

Option	Description
--userName<userName>	The UPN of the user whose messages are to be retrieved
--fromDateTime[fromDateTime]	The start of time range to query
--toDateTime[toDateTime]	The end of time range to query
-p, --pageSize[pageSize]	Number of objects to return
-n, --pageNumber[pageNumber]	Page number to return if pageSize is specified (first page is indexed as value of 0)

Additional Information

https://docs.microsoft.com/en-us/microsoftteams/export-teams-content#how-to-access-teams-export-apis

Prerequisites to access this API

https://docs.microsoft.com/en-us/microsoftteams/export-teams-content#prerequisites-to-access-teams-export-apis

[rabwill]

@pnp/cli-for-microsoft-365-maintainers here is a new command for review of spec, please let me know if this is okay and then we can put it in the backlog and label it appropriately. Thanks 😊

[waldekmastykarz]

Thanks for the suggestion.

1. I wouldn't call the command export. As it retrieves all items, we should name it Get all Microsoft Teams messages for the given user
2. To make the command easier to use, I'd suggest that we allow specifying user's UPN rather than ID, which users would need to retrieve separately first
3. Are filter, pageSize and pageNumber meant to be required? If not, let's denote them as optional using [] rather than <>
4. What can be specified in the filter?
5. What is the default behavior if pageSize and pageNumber are not specified?

[rabwill]

Good catch on point # 3! thanks @waldekmastykarz. In the filter they can specify the date range or messageType etc. See sample response here

{
"id": "string (identifier)",
"replyToId": "string (identifier)",
"from": {"@odata.type": "microsoft.graph.identitySet"},
"etag": "string",
"messageType": "string",
"createdDateTime": "string (timestamp)",
"lastModifiedDateTime": "string (timestamp)",
"deletedDateTime": "string (timestamp)",
"subject": "string",
"from": {
                "application": null,
                "device": null,
                "conversation": null,
                "user": {
                    "id": "string (identifier)",
                    "displayName": "string",
                    "userIdentityType": "aadUser"                }
            },
"body": {"@odata.type": "microsoft.graph.itemBody"},
"summary": "string",
"chatId": "string (identifier)"
"attachments": \[{"@odata.type": "microsoft.graph.chatMessageAttachment"}\],
"mentions": \[{"@odata.type": "microsoft.graph.chatMessageMention"}\],
"importance": "string",
"locale": "string",
}

The API gives back @odata.nextlink in case there are multiple values similar to other commands or this sort.

[plamber]

Not sure if we will be able to use this command at a large scale. I fell into this already a couple of times with some other interesting endpoints.

Microsoft Teams APIs in Microsoft Graph that access sensitive data are considered protected APIs. Export APIs require that you have additional validation, beyond permissions and consent, before you can use them. To request access to these protected APIs, complete the request form.

The command will throw a 401 exception until you got an approval from Microsoft and your AppId has been enabled. @waldekmastykarz I do not believe that MS will enable our PnP CLI app with these permissions due to security reasons. I tried to get the approval already a couple of times but failed in the past.

Is this something we want to clarify before? Maybe have some discussions with the Microsoft responsible parties to see how we can proceed here?

[waldekmastykarz]

@rabwill rather than having a catch-all filter property, I'd suggest we introduce specific options for the specific values so that we can apply validation to it.

@plamber I wouldn't expect getting clearance for our app, especially as it doesn't support app-only auth. Instead, users would get clearance for their own app and use it in combination with CLI. In the past, I requested and got approval so the process is working given proper justification. I agree that the required clearance will likely limit the usage of this command. Perhaps if we can describe a number of usage scenarios and the justification, it could help users get approvals and use this command.

[rabwill]

@waldekmastykarz so just the date range to start off now, and we can add any as we please?

[waldekmastykarz]

@rabwill is date the most common filter?

[rabwill]

Since it is already mentioned as a filter example in the doco, perhaps it's a good start? ☺️

[waldekmastykarz]

Good point 👍

[rabwill]

Shall we move this to backlog then? Do we need more discussion @plamber @waldekmastykarz ?

[waldekmastykarz]

I think we're good to go 👍

[dips365]

I would like to take this

[garrytrinder]

Opening up due to lack of activity

[martinlingstuyl]

We do have a command for listing messages nowadays

https://pnp.github.io/cli-microsoft365/cmd/teams/chat/chat-message-list/

How would this relate? Should we maybe review the specs?

[waldekmastykarz]

Following this spec, this command would use the Teams export APIs which are different than the Chat message list APIs. Chat message list APIs are not meant to be used at scale and could easily lead to throttling, especially on active tenants.

[martinlingstuyl]

I understand, but in that case I think we should review the command name and description. Otherwise it's like you have two commands for what seems the same func.

[martinlingstuyl]

For example teams chat message export?

[martinlingstuyl]

Oh wait, this is across all chats for the user. Never mind. Great spec!

[martinlingstuyl]

On...the...other...hand 🤗

The teams export api exposes two endpoints according to the docs, one for user chat messages and the other for channel messages.

If we where to add both as commands, we'd run into our existing channel message list.

To align it with our existing commands we could have two new commands:

teams chat message export (current issue) teams channel message export (to be created as issue)

This way they would align better and it would be clearer what the commands do.

I'd also vote for an additional option:

Option	Description
--licenseModel [licenseModel]	The license model to use when executing the command. Accepted values: A/B/Evaluation. Default value is Evaluation. Read more about licensing for export api's in the docs

[waldekmastykarz]

Great suggestion @martinlingstuyl! What if we'd combine them both under teams message export? We already have a message namespace in the CLI. Would that make sense or are the chat message and channel message APIs so different that it wouldn't make sense to combine them under message?

[martinlingstuyl]

I think the only thing that's different is the URL. So it would be possible to combine them.

Now you're saying it, we do not have teams channel message list, I was speaking too quickly. We have teams message list. All the teams message <verb> commands refer to channel messages.

You might debate whether it is logical to put a command in there that can export both channel and chat messages. On the other hand, the teams chat message namespace is all about the current user. While teams message is not really about the current user.

I think I like the idea. We do need different options:

Option	Description
--userId [userId]	The Id of the user whose messages are to be exported. Specify either userId or channelId.
--channelId [channelId]	The Id of the channel of which the messages are to be exported. Specify either userId or channelId.

I'd say we drop userName from the specs.

[Adam-it]

@martinlingstuyl I agree. Since we already have the m365 teams message list with --channelId option I would leave it and include a new option which is --userId. So I totally agree and feel it the same way as you described 👍. @pnp/cli-for-microsoft-365-maintainers any other comments? maybe we could open this up?... and maaaybe even as "good first issue" ?

[martinlingstuyl]

As spoken about above, We should update the spec a bit first:

teams message export [options]

[martinlingstuyl]

I updated it, what do you guys think?

[Adam-it]

@martinlingstuyl weren't we suppose to drop the userName option for now?

Besides this single comment all other seem ok and I think we may open this up. do you think this might be a "good first issue"?

[martinlingstuyl]

We have these double options userId and userName in other places as well. Hence I added them both. It's effectively the same thing when calling the graph api, but it's clearer to the user what he can use.

But before we open it up: we need some additional research here:

https://devblogs.microsoft.com/microsoft365dev/announcing-general-availability-of-microsoft-graph-export-api-for-microsoft-teams-messages/

We might need teamId instead or next to channelId.

And we certainly need a licenseModel option as well. We discussed that earlier and it is applicable on this api.

[martinlingstuyl]

Also: what about retrieving chat-attachments:

Message Attachments: Export APIs include the links to the attachments that are sent as part of messages. Using Export APIs you can retrieve the files attached in the messages.

We could add something of a switch to trigger behavior that downloads the attachments.

[martinlingstuyl]

Ok, I updated the specs a bit, adding the licenseModel option and teamId.

What do you guys think of auto exporting the attachments as well, (using --includeAttachments, and --filePath options) or should we leave that to the user, as the attachment urls are provided?

[martinlingstuyl]

I also did a check: it seems the endpoint getAllMessages can only be called on team level, for all channels. It seems to be unavailable on channel level.

I need to do a better test, but it seems we can drop channelId

[Adam-it]

good research @martinlingstuyl 👍. I think the --includeAttachments option would be ok and not exporting them by default.