Integration with JabRef online

[Siedlerchr]

This PR keeps track of the whole process to integrate JabRef with JabRef online. New PRs should be targeted to this PR. After all PRs have gone into this one AND this one is ready, it will be merged.

Refs #7796 #7798 #7582

To run the new tests: gradle test --tests "*online*"

TODO:

• [x] Automatically download graphql schema during build
• [x] Remove test queries and their classes
• [ ] Add authentication using a custom cookieJar that stores the cookie across sessions (might be helpful: https://github.com/franmontiel/PersistentCookieJar or https://www.programmersought.com/article/21021005004/ )
• [x] Pull changes
• [ ] Push changes
• [x] Find solution for https://github.com/origin-energy/java-snapshot-testing/issues/108

---

• [] Change in CHANGELOG.md described in a way that is understandable for the average user (if applicable)
• [] Tests created for changes (if applicable)
• [] Manually tested changed features in running JabRef (always required)
• [] Screenshots added in PR description (for UI changes)
• [] Checked documentation: Is the information available and up to date? If not created an issue at https://github.com/JabRef/user-documentation/issues or, even better, submitted a pull request to the documentation repository.

[Siedlerchr]

Probably needs to be included in the jlink as well

[calixtus]

Should we merge this PR or let some other PR pick up the branch?

[Siedlerchr]

I would vote to merge this PR, so we have a basis for further development and avoid any conflicts with the gradle stuff

[Ali96kz]

@Siedlerchr will we merge that PR?

[tobiasdiez]

@Ali96kz The idea was to keep this PR open as a central place to coordinate the integration with JabRef online. Further PRs should build upon this PR by targeting the jabrefonline branch.

So if you want to work on the JabRef online integration, which you are more then welcome to do, then just create a new branch off from jabrefonline and add your work there. Once you are satisfied, open a new PR (with target branch jabrefonline). Since this a very complex feature, it would be helpful to have many smaller PRs that implement part of the big picture instead of one very huge one. So please keep this in mind.

[tobiasdiez]

@Siedlerchr @Ali96kz Did you already found time working on this? Since it is required for the word integration which is making huge steps, the integration with jabref online is semi-urgent. The api interface is now also relatively stable. Maybe this is something for the jabcon?

[tobiasdiez]

Update from my side: I pretty much completed now the work on the graphql api for adding/updating and getting documents. At least for journal articles, proceedings and thesis; other entry types are still on my todo list (https://github.com/JabRef/JabRefOnline/issues/332). The API should be relatively stable, but it's probable that I missed a few uses cases. If that's the case, please let me know and I will do the necessary changes. There are also still issues with the database, so for example when inserting an article most of the fields are not yet persisted.

Also note that the documentation of the fields contains the correct biblatex field, so that transforming/translating/transpiling/adapting (@k3KAW8Pnf7mkmdSMPHz27 😺 ) to the correct format should be relatively straightforward.

@Ali96kz: Did you already found the time to further work on this? @Siedlerchr Do you plan to work on this during JabCon? It would be nice if we could get the data to jabref online, so that people can play around with the word addin relatively soon.

[Siedlerchr]

I am trying to get it up with the latest changes. Is this right that I need to specify all kind of fields that I want to have back?

query getDocumentById($userDocumentId: ID!) {
	userDocument(id: $userDocumentId){
  id
  citationKeys
  lastModified
  added
  title
  subtitle
  titleAddon
  abstract
  authors {
    ... on Person {
      name
      id
    }
    ... on Organization {
      id
      name
    }
  }
  note
  languages
  publicationState
  keywords
  
}
}

[tobiasdiez]

Yes, that's right! See https://github.com/JabRef/JabRefOnline/issues/251 for the reason behind this design.

[Siedlerchr]

Okay, I adjusted the examples to the new schema. ./gradlew generateApolloSources now generates the new ones from the schema.

Download schema ./gradlew downloadApolloSchema --endpoint="https://jabref-dev.azurewebsites.net/api" --schema="src/main/java/org/jabref/jabrefonline/graphql/schema.json"

[tobiasdiez]

Thanks @Siedlerchr! The api for adding/updating and querying articles should work now (at least there are some working integration tests). So I would propose to start with the sync of articles, and once this works and is stable we add more entry types (which should be straightforward).

[tobiasdiez]

@Siedlerchr any progress on this? Did you encounter any problems or are things unclear?

[Siedlerchr]

I now added the schema download task and generating of the classes to the generateSource task

[tobiasdiez]

The apollo library now should work with the module system: https://github.com/apollographql/apollo-android/tree/dev-3.x https://github.com/apollographql/apollo-android/pull/3558

@Siedlerchr could you try this out?

[Siedlerchr]

Ah good to hear, that will hopefully make things easier. I will give it a try

[Siedlerchr]

Good news, the 3.x apollo dev version is now working without any custom jpsm hacks. There have been some syntax changes and it took me a while to understand this new Optional... (you need to instantiate it) and the nice builder pattern has changed somehow :(

[tobiasdiez]

Nice!

[Siedlerchr]

Fuu, we are hitting module limits again: I can't believe they haven't fixed this yet! https://bugs.openjdk.java.net/browse/JDK-8240567 I mean on the one hand they are encouraging use of JPMS, but then...

Error: jdk.internal.org.objectweb.asm.MethodTooLargeException: Method too large: jdk/internal/module/SystemModules$all.moduleDescriptors ()[Ljava/lang/module/ModuleDescriptor;

[tobiasdiez]

I remember we had this before...how do we fixed it back then?

EDIT: Okay, it was in https://github.com/JabRef/jabref/pull/7126 and we solved it by removing other dependencies....damn

[Siedlerchr]

I hope that the Tinylog conversion stuff maybe helps by removing the log4j stuff

[tobiasdiez]

Can you please have a look at the generated file and compare it with the one from the main branch. Maybe there is also something the apollo guys can/should improve in their module description.

[tobiasdiez]

Now that the tinylog PR is merged, can you have a look if that solves the issue @Siedlerchr. Thanks!

[Siedlerchr]

@tobiasdiez It works now again. But a more general question, I am trying to get the mutation running as well. The code is generated using the new Optional class from apollo and every parameter is wrapped in it. And from the code generation it seems like I really need to state this as parameter. This is ridiculous? Or is this just a bad kotlin integration? It sucks.

[tobiasdiez]

In general this is good news! Thanks for trying.

Not sure about the optional thing. Maybe ask in the apollo repo if this indeed the best way. Maybe they also provide a builder pattern to construct the input?

[tobiasdiez]

The new java (not kotlin) runtime seems to work well. Currently, further experimentation is blocked by https://github.com/apollographql/apollo-kotlin/issues/4448.

[github-actions[bot]]

Your code currently does not meet JabRef's code guidelines. We use OpenRewrite to ensure "modern" Java coding practices. The issues found can be automatically fixed. Please execute the gradle task rewriteRun, check the results, commit, and push.

You can check the detailed error output at the tab "Checks", section "Tests" (on the left), subsection "OpenRewrite".

[github-actions[bot]]

Your code currently does not meet JabRef's code guidelines. We use Gradle Modernizer Plugin to ensure "modern" Java coding practices. Please fix the detected errors, commit, and push.

You can check the detailed error output at the tab "Checks", section "Tests" (on the left), subsection "Modernizer".

[github-actions[bot]]

Your code currently does not meet JabRef's code guidelines. We use Checkstyle to identify issues. The tool reviewdog already placed comments on GitHub to indicate the places. See the tab "Files" in you PR. Please carefully follow the setup guide for the codestyle. Afterwards, please run checkstyle locally and fix the issues.

More information on code quality in JabRef is available at https://devdocs.jabref.org/getting-into-the-code/development-strategy.html.

[github-actions[bot]]

The build of this PR is available at https://builds.jabref.org/pull/7832/merge.