Implement cursor for search results

[Lakritzator]

Atlassian posted an update to their API, it is now using a cursor to go to the next and previous results. More information can be found here: https://developer.atlassian.com/cloud/confluence/change-notice-moderize-search-rest-apis/

This is NOT yet implemented in the current version of Dapplo.Confluence

[stevelknievel76]

Does this mean that the previous paging solution in Confluence doesn't work? So you can't do paging at all with this sdk?

I have the following:

                searchResult = await confluenceClient.Content.SearchAsync(Where.And(Where.Space.Is("CDO"), Where.Label.Is("incident-reports"), Where.Created.AfterOrOn.DateTime(DateTime.Parse("2023-01-07T00:00:00Z"))), pagingInformation: new PagingInformation
                {
                    Limit = 25,
                    Start = start
                });

I know that there are 40 matching results. On the first call, start is 0 and I add 25 to it while searchResult.HasNext is true to get the whole set of results in batches of 25. I see that the first 2 REST calls generated by the sdk are:

https://XXXXXXXXX.atlassian.net/wiki/rest/api/content/search?cql=%28space%20%3D%20%22CDO%22%20and%20label%20%3D%20%22incident-reports%22%20and%20created%20%3E%3D%20%222023-01-07%22%29&limit=25&start=0&expand=version%2Cspace%2Cspace.icon%2Cspace.description%2Cspace.homepage

https://XXXXXXXXX.atlassian.net/wiki/rest/api/content/search?cql=%28space%20%3D%20%22CDO%22%20and%20label%20%3D%20%22incident-reports%22%20and%20created%20%3E%3D%20%222023-01-07%22%29&limit=25&start=25&expand=version%2Cspace%2Cspace.icon%2Cspace.description%2Cspace.homepage

... which looks right, but both REST calls yield the same payload (the first 25 records), and the second call has searchResult.HasNext = true. I was expecting the second call to get the remaining 15 records and have searchResult.HasNext = false.

If I use the cursor url returned in _links instead:

https://XXXXXXXXX.atlassian.net/wiki/rest/api/content/search?next=true&cursor=_sa_WyJcdDEyMzEwOTM3NyBOZGNYdGdQQUlZXSpWLko8bUx1KyBjcCJd&expand=version%2Cspace%2Cspace.icon%2Cspace.description%2Cspace.homepage&limit=25&start=25&cql=%28space%20%3D%20%22CDO%22%20and%20label%20%3D%20%22incident-reports%22%20and%20created%20%3E%3D%20%222023-01-07%22%29

Now it does correctly return the remaining 15 rows.

[stevelknievel76]

Hi @Lakritzator. I've coded a solution for paging using the next and prev links. Keen to get a pull request to you. However, not having access to your integration test Confluence wiki I can't run the tests. Some of the code I've changed is more or less a guess as it's regarding functionality I'm not interested in, but I've extensively tested that the changes in ContentExtensions.SearchAsync work with my code. What is the best approach? Do I create a pull request or do you give access to your wiki for testing?

[Lakritzator]

Oh cool 👍

About the integration tests, if you tested most on your side I will see if it works on my side by cloning locally and have them run before merging. I unfortunately cannot add credentials to my repository, there are too many people abusing stuff.

I have paging in my Jira library, so I'd love to see that solution be aligned with this.

I have entities for passing the page information: https://github.com/dapplo/Dapplo.Jira/blob/master/src/Dapplo.Jira/Entities/Page.cs

To get back the information from the server: https://github.com/dapplo/Dapplo.Jira/blob/master/src/Dapplo.Jira/Entities/PageableResult.cs

And to figure out the next page: https://github.com/dapplo/Dapplo.Jira/blob/master/src/Dapplo.Jira/Entities/SearchIssuesResult.cs#L49

I was hoping to bring both Confluence and Jira to one solution, so one repo, where I can share such code...

[Lakritzator]

So far I remember, do remind that I do not work on this repository on a regular base, I was half-way with confluence.

It's possible to pass the paging information, but I didn't test everything and there was no easy way to get the next page. https://github.com/dapplo/Dapplo.Confluence/blob/a62580c026e6ef088e5dd02f435e85fb9bd018b3/src/Dapplo.Confluence/Entities/Result.cs#L39

[MiguelTVMS]

Added the implementation to the cursor at the pull request https://github.com/dapplo/Dapplo.Confluence/pull/66

[stevelknievel76]

Hi @Lakritzator. Sorry about the delay - I've been moved on to other projects. I saw that @MiguelTVMS has created #66 in the meantime. I've committed my changes on my fork anyhow, and could create an alternative pull request, but I'm a bit of a n00b so you probably wouldn't like my approach much!

In the UK we have a saying: You wait ages for a bus and then two come along at once!