Add support for ExportItems and UploadItems

[richard-einfinity]

[azurecla]

Hi @richard-einfinity, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution! In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.azure.com.

TTYL, AZPRBOT;

[azurecla]

@richard-einfinity, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.
Thanks, AZPRBOT;

[flugaoveltem]

Hi eInfinity, how are you? I don't know if that this is the place to ask for help or make requests. I'm trying to implement the ExportItem implementation, but I can maket work. Could you upload a code sample of ExportItem and a UploadItem?

Best Regards

[richard-einfinity]

Hi @flugaoveltem,

What are you looking for the code for the implementation or a sample of the implementation being used?

Kind regards

Richard

[flugaoveltem]

Hi @richard-einfinity,

I'm looking for the code for implementation,

Thanks

[richard-einfinity]

Hi @flugaoveItem,

The code for the implementation is in our fork. Compare the branch to the main project branch for the implementation code.

Cheers

Richard

Sent from my iPhone

On 29 May 2017, at 13:50, flugaoveltem <[email protected]mailto:[email protected]> wrote:

Hi @richard-einfinityhttps://github.com/richard-einfinity,

I'm looking for the code for implementation,

Thanks

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/OfficeDev/ews-managed-api/pull/109#issuecomment-304653946, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AI891a9uq3DB-CBbvMKzYtlw3ZBChT-7ks5r-r8pgaJpZM4MRXK8.

[flugaoveltem]

Sorry for the inconvenience @richard-einfinity. I got confused on the english.

Im looking for a sample of the implementation being used

thanks.

[richard-einfinity]

Hi @flugaoveltem,

See below

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Exchange.WebServices.Data;

namespace ExchangeMigration.Sample
{
    class Program
    {
        private const string EWS_URI = "https://outlook.office365.com/ews/exchange.asmx";
        static void Main(string[] args)
        {

            ExchangeService sourceService = new ExchangeService(ExchangeVersion.Exchange2013_SP1);             
            sourceService.Credentials = new WebCredentials("<SOURCE_EMAIL>", "<SOURCE_PASSWORD>");
            sourceService.Url = new Uri(EWS_URI);
                        
            ExchangeService destinationService = new ExchangeService(ExchangeVersion.Exchange2013_SP1);
            destinationService.Credentials = new WebCredentials("<DESTINATION_EMAIL>", "<DESTINATION_PASSWORD>");
            destinationService.Url = new Uri(EWS_URI);

            ItemView itemView = new ItemView(25) { PropertySet = PropertySet.IdOnly, Offset = 0 };

            while (true)
            {
                FindItemsResults<Item> result = sourceService.FindItems(WellKnownFolderName.Inbox, itemView);
                List<ItemId> idsToExport = result.Items.Select(i => i.Id).ToList();            

                if (idsToExport.Count > 0)
                {
                    ServiceResponseCollection<ExportItemsResponse> exportResponses = sourceService.ExportItems(idsToExport);

                    foreach (ExportItemsResponse exportResponse in exportResponses)
                    {

                        if (exportResponse.Result == ServiceResult.Success)
                        {

                            UploadItem itemToUpload = new UploadItem(destinationService)
                            {
                                ParentFolderId = WellKnownFolderName.Inbox,
                                Data = exportResponse.Data,
                                CreateAction = CreateAction.CreateNew
                            };
                        
                            UploadItemsResponse uploadItemResult = destinationService.UploadItem(itemToUpload);
                        

                            Console.WriteLine("Uploaded Item\r\n{0}\r\n{1}", exportResponse.ItemId.UniqueId, uploadItemResult.Id.UniqueId);
                        }
                        else
                        {
                            Console.WriteLine("Failed to export Item\r\n{0}\r\n{1}\r\n{2}\r\n", exportResponse.ErrorCode, exportResponse.ErrorMessage, exportResponse.ErrorDetails);
                        }
                    }
                    idsToExport.Clear();
                }

                if(result.MoreAvailable)
                {
                    itemView.Offset += itemView.PageSize;
                }
                else
                {
                    break;
                }
            }
        }
    }
}

[flugaoveltem]

Hi @richard-einfinity,

Thanks. You are awsome.

[martinlaukkanen]

Hi Richard,

Thanks for the great commit!

I've been testing out your implementation in my own code here, and I think I have found a memory leak. At least I seem to have eliminated my code as the cause of this leak, what I see essentially is that when using ExportItems all items remain in memory for that thread until the thread ends.

I haven't tested with your same code above, but the main difference in my implementation is that after calling ExportItems I write the byte[] array data to disk with a simple File.WriteAllBytes(tempFile, data) call.

At that point a reference to the data bytes is left open somewhere and eventually if the folder being exported contains more items than memory permits eventually an out of memory exception occurs.

Specifically if I do NOT write the bytes to disk after calling ExportItems then the memory is released normally.

Can you confirm if this occurs for you?

Quick follow up: I've adapted my code to use the Byte[] array in memory just as you do without any writing / reading from disk and I have the same issue, so I assume your code example above also will.

Thanks, Martin

[richard-einfinity]

Hi @martinlaukkanen,

I haven't seen this issue previously.

Can you give me a sample of the code?

Cheers

Richard

[martinlaukkanen]

Hi Richard,

Thanks for following up. I took a look at your sample above to see if I can replicate my issue, and initially your sample code worked without the issue.

However to match my use case I modified it as below and was able to reproduce the issue.

Better yet I was also able to solve it! Basically the difference was that I am creating a Folder and referencing it by Folder.Id property rather than using a WellKnownFolderName, this is how I work with Item.Bind and such without similar issues.

The issue was in the Folder.Id reference on the UploadItem construction, as you can see in my comments in the code referencing the parent folder by: ParentFolderId = new FolderId(folder.Id.UniqueId) is the only way to avoid the memory leak.

`using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using Microsoft.Exchange.WebServices.Data;

namespace TestEwsExportItems { class Program { private const string EWS_URI = "https://outlook.office365.com/ews/exchange.asmx"; static void Main(string[] args) {

        ExchangeService sourceService = new ExchangeService(ExchangeVersion.Exchange2013_SP1);
        sourceService.Credentials = new WebCredentials("", "");
        sourceService.Url = new Uri(EWS_URI);

        ExchangeService destinationService = new ExchangeService(ExchangeVersion.Exchange2013_SP1);
        destinationService.Credentials = new WebCredentials("", "");
        destinationService.Url = new Uri(EWS_URI);

        ItemView itemView = new ItemView(25) { PropertySet = PropertySet.IdOnly, Offset = 0 };

        // NEW SAMPLE CODE: Target folder is created
        var folder = new Folder(destinationService)
        {
            DisplayName = "Copy of Inbox",
            FolderClass = "IPF.Note"
        };
        folder.Save(WellKnownFolderName.Root);

        while (true)
        {
            FindItemsResults<Item> result = sourceService.FindItems(WellKnownFolderName.Inbox, itemView);
            List<ItemId> idsToExport = result.Items.Select(i => i.Id).ToList();

            if (idsToExport.Count > 0)
            {
                ServiceResponseCollection<ExportItemsResponse> exportResponses = sourceService.ExportItems(idsToExport);

                foreach (ExportItemsResponse exportResponse in exportResponses)
                {

                    if (exportResponse.Result == ServiceResult.Success)
                    {

                        UploadItem itemToUpload = new UploadItem(destinationService)
                        {
                            //ParentFolderId = WellKnownFolderName.Inbox, // Original sample code
                            //ParentFolderId = folder.Id, // Memory leak - all items remain referenced
                            ParentFolderId = new FolderId(folder.Id.UniqueId), // Working version
                            Data = exportResponse.Data,
                            CreateAction = CreateAction.CreateNew
                        };

                        UploadItemsResponse uploadItemResult = destinationService.UploadItem(itemToUpload);


                        Console.WriteLine("Uploaded Item\r\n{0}\r\n{1}", exportResponse.ItemId.UniqueId, uploadItemResult.Id.UniqueId);
                    }
                    else
                    {
                        Console.WriteLine("Failed to export Item\r\n{0}\r\n{1}\r\n{2}\r\n", exportResponse.ErrorCode, exportResponse.ErrorMessage, exportResponse.ErrorDetails);
                    }
                }
                idsToExport.Clear();
            }

            if (result.MoreAvailable)
            {
                itemView.Offset += itemView.PageSize;
            }
            else
            {
                break;
            }
        }
    }
}

} `

[msftclas]

All CLA requirements met.

[MichelZ]

@davster ready for merge!