JSqlParser icon indicating copy to clipboard operation
JSqlParser copied to clipboard

Sphinx Website

Open manticore-projects opened this issue 3 years ago • 12 comments

Summary

Add a HTML5 Website based on Sphinx:

a) scales on devices (from mobile phone to very large screens) b) theme-able (shown is a modified theme based on Read The Docs, being cleaner and scaling on large screens much better, landing page added) c) CSS customizable (you can change the Accent Colours) d) integrated search function e) please adjust/change the icons according to your needs

Integrate the Sphinx Website into the Gradle Build:

d) derive the actual Version Number (Release and Snapshot) from the GIT Tags e) generate a clean Changelog from GIT (template can be customized) f) update all Version references (Maven Artifact, Downloads etc) automatically

Add some content: g) explain the Contribution process h) link to JSQLFormatter online for Statement Testing and Visualisation i) add the Railroad Diagrams explaining the syntax (but without Reserved Keywords yet)

Added Object Tree Visualization tools, which will generate a TreeNode based tree structure explaining the parsed java Object Tree.

select 1 from dual where a=b;
SQL Text
  └─Statements: net.sf.jsqlparser.statement.select.Select
     └─selectBody: net.sf.jsqlparser.statement.select.PlainSelect
        ├─selectItems -> Collection<SelectExpressionItem>
        │  └─selectItems: net.sf.jsqlparser.statement.select.SelectExpressionItem
        │     └─LongValue: 1
        ├─Table: dual
        └─where: net.sf.jsqlparser.expression.operators.relational.EqualsTo
           ├─Column: a
           └─Column: b

Build Instructions

  1. you will need Python Sphinx and GIT installed
  2. build the website via gradle sphinx
  3. optionally, place the Sphinx HTML theme into the _themes folder and/or provide a custom.css file

Preview

Live preview: http://manticore-projects.com/JSQLParser/index.html

Static preview image: image

Static preview package: sphinx.zip

Attention

It has been built upon the Keywords PR, mainly because of the build infrastructure and the keywords documentation. So the Keywords PR is included and need to be accepted first (or dropped altogether in favor of this Sphinx PR).

manticore-projects avatar Sep 02 '22 08:09 manticore-projects

Please help to exclude the folder `src/site/' from the Codacy configuration. Alternatively disable checks for Python, CSS and JavaScript. Thank you!

image

manticore-projects avatar Sep 02 '22 08:09 manticore-projects

Cool .. you are fighting at all fronts :)

wumpz avatar Sep 03 '22 19:09 wumpz

So #898 is still in place, meaning the sphinx generated page should go to the github pages?

wumpz avatar Sep 03 '22 19:09 wumpz

If I understand that right, you need to have a python installed to generate it? Not so cool. Is there an integration for maven?

wumpz avatar Sep 03 '22 19:09 wumpz

If I understand that right, you need to have a python installed to generate it? Not so cool. Is there an integration for maven?

Yeah, sorry about that. I really tried to avoid that and have tested a lot of different things. But only SPHINX did what I wanted and needed. JBaker was the only Java alternative I could find and it was not superior.

There is a Maven plugin but I found that more difficult to use than just calling Sphinx from Java. Although the Website Sources are RESTRUCTURED TEXT and can be used with any other framework.

I suggest to look at it first and try out the live demo. Then make a list of pros and cons and then decide.

manticore-projects avatar Sep 03 '22 19:09 manticore-projects

Btw, no feedback on that super cool Java Parsed Object Tree?

manticore-projects avatar Sep 03 '22 19:09 manticore-projects

If I understand that right, you need to have a python installed to generate it? Not so cool. Is there an integration for maven?

Your feedback and opinion are important to me -- so I did a re-assessment of the options and found only 3 serious contesters:

  • Sphinx based Python/Pygments with many good templates
  • Hugo based on Go/Chroma with many good templates
  • JBake based on Java, unclear about Syntax highlighting, almost no templates available

Everything else depends on JavaScript or NodeJS or NPM, so I won't even mention it.

Obviously we would love to use JBake, but I did not find any good Templates and I also did not find anything about Code Syntax Highlighting or Math support. It seems mostly being used for Blogs.

This leaves Hugo, but I honestly do not see the advantage of trading one non-Java binary against another non-Java binary. And Sphinx seems to be superior for Technical Documentation -- it is more Latex/Docbook alike than Hugo.

If you have any better idea, please let me know. I am all ears!

manticore-projects avatar Sep 04 '22 11:09 manticore-projects

The tree visualizer is quite useful. Cool. So as I see this, the sphinx site is generated as the java project site? Since I have no time writing it, I will go with your decision to use Sphinx. However, the Jekyll way would be GitHub internal, which would be cool to have. I even saw a RST Plugin for Jekyll.

wumpz avatar Sep 10 '22 21:09 wumpz

Is this branch one for its own? I saw some changes from the keyword stuff in here.

wumpz avatar Sep 10 '22 21:09 wumpz

However, the Jekyll way would be GitHub internal, which would be cool to have. I even saw a RST Plugin for Jekyll.

With "GitHib internal", you do mean the Readme.md GitHub frontpage?

GitHub reads/displays RST out of the box, although without the most interesting Directives (TOC and substitutions). Means, you can't easily update/change the GitHub README.rst or README.md with latest version, latest artifact and changelog an stuff. (Of course we could build that README.md in Groovy from a template.)

So my suggestion is: Define a minimal, static README.md with only provides basic, timeless summary of the project and links to the actual website + change-log + download instructions.

Is this branch one for its own? I saw some changes from the keyword stuff in here.

Yes, its built on top of they keywords PR -- because I need the Railroad Diagrams and Keywords for the documentation. Also the examples work only with proper Keyword whitelisting.

manticore-projects avatar Sep 11 '22 06:09 manticore-projects

With "GitHib internal", you do mean the Readme.md GitHub frontpage?

No I mean Github pages: https://pages.github.com/. Those pages are generated by a "Github internal" Jekyll process. So you only have to push and GitHub will generate the whole stuff again and again.

So here is one autogenerated example: https://java-diff-utils.github.io/java-diff-utils/ but you can build whole sites.

wumpz avatar Sep 20 '22 17:09 wumpz

No I mean Github pages: https://pages.github.com/. Those pages are generated by a "Github internal" Jekyll process. So you only have to push and GitHub will generate the whole stuff again and again.

I get it. And yes, Jekyll is a natural competitor. So if you want to go this route, we would need to a) translate RST into MD b) find a proper Jekyll theme (I have been unimpressed by what they offered) c) find a way to template the Version Numbers and Download Links properly

To me, b) and c) have been the decision criteria and the reason to use Sphinx. I use GitHub -- but I never would want to depend on it.

My idea was to build the static pages offline and then push/upload them to GitHub or any WeSserver. This is what we do with the company website.

manticore-projects avatar Sep 20 '22 17:09 manticore-projects

Btw, Linux Kernel uses Sphinx for documentation also. They have adopted 1 year ago or so and we can assume they have discussed the pros and cons intensively before.

Please help to deactivate the python/js checks on codacy.

manticore-projects avatar Sep 23 '22 05:09 manticore-projects

Btw, Linux Kernel uses Sphinx for documentation also. They have adopted 1 year ago or so and we can assume they have discussed the pros and cons intensively before.

I know, but sphinx simply runs on linux. Python and stuff.

Please help to deactivate the python/js checks on codacy.

I will look into it.

wumpz avatar Sep 28 '22 19:09 wumpz

So Codacy JS and Python checks should be disabled now.

wumpz avatar Sep 28 '22 19:09 wumpz

I think your impression is the themes are only those from the github pages auto generator. In fact there are countless themes: https://jekyllthemes.io/free. Responsive, with search, ...

wumpz avatar Sep 28 '22 20:09 wumpz

However, I do not prefer Jekyll over Sphinx, it was just a question since this discussion was a time ago.

wumpz avatar Sep 28 '22 21:09 wumpz

Please also disable CSS in Codacy. Thanks.

manticore-projects avatar Nov 07 '22 02:11 manticore-projects

Example of the keyword documentation: https://manticore-projects.com/JSQLParser/keywords.html It's generated by the updateKeyword task.

manticore-projects avatar Nov 09 '22 00:11 manticore-projects

I have added a Github Action, that builds the Website on Commit to Master and deploys the static Site to GH Pages.

Demo: https://manticore-projects.github.io/JSqlParser/

manticore-projects avatar Dec 11 '22 16:12 manticore-projects

On Thu, 2022-12-22 at 14:46 -0800, Tobias wrote:

Why all these implements Serializable stuff?

The tree viewer, i need to serialize the objects first into xml before applying xpath and deserialize.

manticore-projects avatar Dec 22 '22 22:12 manticore-projects

Please also disable CSS in Codacy. Thanks.

Kind reminder, since I am not allowed to do that by myself unfortunately.

manticore-projects avatar Dec 23 '22 08:12 manticore-projects

Disabled CSSLint, StyleLint in Codacy.

wumpz avatar Dec 27 '22 10:12 wumpz

Disabled CSSLint, StyleLint in Codacy.

Thanks. Will you accept the PR or do you expect me to push/update and rerun the Codacy CI-Tests before?

manticore-projects avatar Dec 27 '22 10:12 manticore-projects

The tree viewer, i need to serialize the objects first into xml before applying xpath and deserialize.

Where is this xpath parser implemented? I did not find it at first glance. You know this serialization stuff is somehow deprecated: https://www.securityinfowatch.com/cybersecurity/information-security/article/12420169/oracle-plans-to-end-java-serialization-but-thats-not-the-end-of-the-story. I would like to find another way.

To make things easier I could make node generation for every production a default. Since performance is not that important anymore. I measured a decrease of 20% for queries.

wumpz avatar Dec 27 '22 10:12 wumpz

Where is this xpath parser implemented?

Separately, in JSQLFormatter only.

I did not find it at first glance. You know this serialization stuff is somehow deprecated: https://www.securityinfowatch.com/cybersecurity/information-security/article/12420169/oracle-plans-to-end-java-serialization-but-thats-not-the-end-of-the-story. I would like to find another way.

There are other ways like Jackson, however that would introduce Third Party dependencies because they all rely on a kind of annotation and implementation of Serializers and De-Serializers.

In your own link its written, that "Oracle plans ..." since 2018 and nothing has happened since. Even if anything happens, it would depend on Java 19 plus while we still (want to) maintain Java 8 compliance for a good reason.

Companies still run JRE/JDK 8 and won't upgrade.

Last but not least, I think some Serializable Objects were there already, I just added a few more but did not introduce anything unheard of before.

image

manticore-projects avatar Dec 27 '22 10:12 manticore-projects

Fair point.

wumpz avatar Dec 27 '22 10:12 wumpz

Could you check this Codacy checks again (making a commit). Codacy somehow does not to allow me to reevaluate specific commits again.

wumpz avatar Dec 27 '22 10:12 wumpz

Pew, finally!

manticore-projects avatar Dec 27 '22 12:12 manticore-projects

Integrate the RR diagram into the website. Applied a neutral Sphinx theme.

Preview: https://manticore-projects.github.io/JSqlParser/syntax.html

Besser als das wird es nicht! Btw, die Aenderung der Versions Nummer in der Readme koennten wir uns sparen mit der WebSite, welche die Versionierung automatisch verfolgt.

manticore-projects avatar Jan 07 '23 12:01 manticore-projects