opengrok

opengrok copied to clipboard

dynamic history cache chunk size

[{"_id":"637e35105ae95c7a222d633e","body":"As visible on https:\/\/github.com\/oracle\/opengrok\/pull\/3647#issuecomment-867116127 it may be worthwhile to consider also non even chunk sizes. Lots of the repositories were converted from other Version Control Systems so w.r.t. history cache creation the initial changesets have bigger memory requirements. Starting with smaller chunks and increasing the chunk size (exponentially ?) might make sense in order to keep the heap usage low yet maintaining speed.","issue_id":1660204174410,"origin_id":867119985,"user_origin_id":2934284,"create_time":1624478509,"update_time":1624478509,"id":1669215504858,"updated_at":"2022-11-23T14:58:24.857000Z","created_at":"2022-11-23T14:58:24.857000Z"}]

As mentioned in PR #3647 the history cache chunk count could be made more dynamic, based on heap size, CPU count etc.

vladak

flatten history cache parallelism

[{"_id":"637e27e2bc25e83db007ced2","body":"Thinking about this I realized that the nested history cache parallelism is actually limited when creating history cache for multiple repositories. While the history of multiple repositories is created using `getHistoryExecutor()`, for individual files `getHistoryFileExecutor()` is used. By default the sizes of both thread pools are set based on the number of available CPUs in the system. This means all currently indexed repositories will share the common thread pool for individual files so the parallelism multiplication is not really there.","issue_id":1660204174412,"origin_id":867019702,"user_origin_id":2934284,"create_time":1624468783,"update_time":1624468783,"id":1669212130296,"updated_at":"2022-11-23T14:02:10.296000Z","created_at":"2022-11-23T14:02:10.296000Z"}]

As mentioned in #3647, the history cache parallelism is nested which leads to challenges with resource planning. It might be worthwhile to investigate the same scheme as is used in...

vladak

darkmode

[{"_id":"637e31624b97542c9a2dff78","body":"https:\/\/tombrow.com\/dark-mode-website-css","issue_id":1660204174414,"origin_id":539931554,"user_origin_id":2934284,"create_time":1570615228,"update_time":1570615228,"id":1669214562446,"updated_at":"2022-11-23T14:42:42.445000Z","created_at":"2022-11-23T14:42:42.445000Z"}]

The current (?) UI fashion seems to be the dark mode (ala IDEA). I wonder if there is a way how to make the switch without re-introducing distinct CSS style...

vladak

present tags as list in history API endpoint

[]

As mentioned in https://github.com/oracle/opengrok/pull/3636#discussion_r652917370 the tags can be stored as a list, not string. The history API endpoint would benefit.

vladak

Failed to scan [file:/var/tomcat8/webapps/source/WEB-INF/lib/annotations-19.0.0.jar]

[{"_id":"637e2e4c5ae95c7a222d5c16","body":"Interestingly `opengrok-indexer.pom` has the `annotations` artifact at 20.1.0 and the web app has also another version:\r\n```\r\n# find \/var\/tomcat8\/webapps\/source\/ -name 'annotations*.jar' -type f\r\n\/var\/tomcat8\/webapps\/source\/WEB-INF\/lib\/annotations-12.0.jar\r\n\/var\/tomcat8\/webapps\/source\/WEB-INF\/lib\/annotations-20.1.0.jar\r\n```\r\nLooks like one of the .jar files has a dependency on 19.0.0 however it is not `opengrok.jar`.\r\n\r\nI went through all the `META-INF\/MANIFEST.MF` files for all the .jar files shipped with OpenGrok in the `dist` directory and there is no mention of annotations-19.0.0.","issue_id":1660204174419,"origin_id":737536053,"user_origin_id":2934284,"create_time":1606948361,"update_time":1606950341,"id":1669213772843,"updated_at":"2022-11-23T14:29:32.843000Z","created_at":"2022-11-23T14:29:32.843000Z"},{"_id":"637e2e4c5ae95c7a222d5c17","body":"I can see this only when Tomcat is restarted. For the record, the only other applications are the `manager` and `host-manager`.","issue_id":1660204174419,"origin_id":737541418,"user_origin_id":2934284,"create_time":1606949123,"update_time":1606949510,"id":1669213772846,"updated_at":"2022-11-23T14:29:32.845000Z","created_at":"2022-11-23T14:29:32.845000Z"},{"_id":"637e2e4c5ae95c7a222d5c18","body":"Going through https:\/\/mvnrepository.com\/artifact\/org.jetbrains\/annotations\/19.0.0\/usages?p=1 I noticed chronicle-map related artifacts. Namely chronicle-bytes-2.20.80.jar depends on annotations-19.0.0.jar. Even the latest version of chronicle-bytes-2.20.105 depends on annotations-19.0.0.jar so upgrade will not fix that.","issue_id":1660204174419,"origin_id":737549258,"user_origin_id":2934284,"create_time":1606950316,"update_time":1606950316,"id":1669213772848,"updated_at":"2022-11-23T14:29:32.847000Z","created_at":"2022-11-23T14:29:32.847000Z"},{"_id":"637e2e4c5ae95c7a222d5c19","body":"can we add dependency to opengrok-indexer pom.xml and opengrok-web pom.xml ??? both of them version fix to 19.0.0\r\n```\r\n <dependency>\r\n <groupId>org.jetbrains<\/groupId>\r\n <artifactId>annotations<\/artifactId>\r\n <version>19.0.0<\/version>\r\n <\/dependency>\r\n\r\n```","issue_id":1660204174419,"origin_id":852782601,"user_origin_id":16930340,"create_time":1622616236,"update_time":1622616236,"id":1669213772876,"updated_at":"2022-11-23T14:29:32.875000Z","created_at":"2022-11-23T14:29:32.875000Z"},{"_id":"637e2e4c5ae95c7a222d5c1a","body":"No, we cannot. `19.0.0` is old and that would be also wrong as `annotations` is only compile time dependency so there is no need to actually include it in the release jar. For now the best \"fix\" is to disable jar scanning on the tomcat level until chronicle will fix their libraries.","issue_id":1660204174419,"origin_id":852938350,"user_origin_id":12401160,"create_time":1622632292,"update_time":1622632292,"id":1669213772885,"updated_at":"2022-11-23T14:29:32.885000Z","created_at":"2022-11-23T14:29:32.885000Z"},{"_id":"637e2e4c5ae95c7a222d5c1b","body":"Created an issue for jetbrains annotations dependency in chronicle-map (https:\/\/github.com\/OpenHFT\/Chronicle-Map\/issues\/289)\r\n\r\nI think there is still a couple of other errors with the same cause (if I recall correctly, it was `javax.el` from `cron-utils`).","issue_id":1660204174419,"origin_id":857963334,"user_origin_id":12401160,"create_time":1623263854,"update_time":1623263854,"id":1669213772888,"updated_at":"2022-11-23T14:29:32.888000Z","created_at":"2022-11-23T14:29:32.888000Z"}]

After #3387 there is one remaining jar that needs to be addressed: ``` 02-Dec-2020 23:23:35.081 WARNING [localhost-startStop-1] org.apache.tomcat.util.scan.StandardJarScanner.processURLs Failed to scan [file:/var/tomcat8/webapps/source/WEB-INF/lib/annotations-19.0.0.jar] from classloader hierarchy java.io.IOException: java.lang.reflect.InvocationTargetException at org.apache.tomcat.util.compat.Jre9Compat.jarFileNewInstance(Jre9Compat.java:205) at...

vladak

Rejuvenate log levels

[{"_id":"637e2ad14b97542c9a2df8e9","body":"Cool! Are there any articles (thesis) publicly available which look at the tool in detail? I'd like to read more on that :)","issue_id":1660204174423,"origin_id":853134814,"user_origin_id":12401160,"create_time":1622648446,"update_time":1622648446,"id":1669212881132,"updated_at":"2022-11-23T14:14:41.131000Z","created_at":"2022-11-23T14:14:41.131000Z"},{"_id":"637e2ad14b97542c9a2df8ea","body":"We have a research paper for this project: https:\/\/arxiv.org\/pdf\/2104.07736.pdf. Please feel free to contact us if you have any questions.","issue_id":1660204174423,"origin_id":853955454,"user_origin_id":10117031,"create_time":1622733903,"update_time":1622733903,"id":1669212881135,"updated_at":"2022-11-23T14:14:41.134000Z","created_at":"2022-11-23T14:14:41.134000Z"},{"_id":"637e2ad14b97542c9a2df8eb","body":"I think we should probably pick just some pieces, not the whole thing.","issue_id":1660204174423,"origin_id":854721013,"user_origin_id":2934284,"create_time":1622813000,"update_time":1622813000,"id":1669212881137,"updated_at":"2022-11-23T14:14:41.137000Z","created_at":"2022-11-23T14:14:41.137000Z"},{"_id":"637e2ad14b97542c9a2df8ec","body":"If comment on the lines, @saledouble can revert those specific lines. If you can comment on why a particular transformation is undesirable, that would be very helpful. Thank you again for your help!","issue_id":1660204174423,"origin_id":854963523,"user_origin_id":2048831,"create_time":1622836167,"update_time":1622839864,"id":1669212881140,"updated_at":"2022-11-23T14:14:41.140000Z","created_at":"2022-11-23T14:14:41.140000Z"}]

Here's a reissue of https://github.com/oracle/opengrok/pull/2978 which is closed due to inactivity and conflicts. We have run our [logging tool](https://github.com/ponder-lab/Rejuvenate-Logging-Levels) on the last version of the project and solved all conflicts....

yiming-tang-cs

There is no data in web after restart tomcat or docker whithout no code change and reindex

[{"_id":"637e37db70db72139b0ea021","body":"could you attach full logs from the container ? (`docker logs <container_name>`)","issue_id":1660204174426,"origin_id":854220182,"user_origin_id":2934284,"create_time":1622758415,"update_time":1622758415,"id":1669216219375,"updated_at":"2022-11-23T15:10:19.375000Z","created_at":"2022-11-23T15:10:19.375000Z"}]

There is no data in web after restart tomcat or docker, but data and config is no change, so can i just upload configuration.xml ? There was an error! The...

menggya

renamed file paths in History should not overlap with file paths in HistoryEntry

[{"_id":"637e27cebc25e83db007cebf","body":"Of course, respective repository code needs to change as well, e.g. https:\/\/github.com\/oracle\/opengrok\/blob\/1374d7eaf95a0381acb2a576c1d5590d9d2bf276\/opengrok-indexer\/src\/main\/java\/org\/opengrok\/indexer\/history\/GitRepository.java#L620-L628","issue_id":1660204174428,"origin_id":843036394,"user_origin_id":2934284,"create_time":1621332299,"update_time":1621332299,"id":1669212110393,"updated_at":"2022-11-23T14:01:50.392000Z","created_at":"2022-11-23T14:01:50.392000Z"}]

The renamed file handling in `FileHistoryCache` is needlessly complex. The way it currently works is that the repository in question will place the renamed files into both the `renamedFiles` set...

vladak

The file type selector could be capable to enable more than one value

[{"_id":"637e355670db72139b0e9d95","body":"Hello, \r\n\r\nWhat's the current status of this ticket?\r\nIt seems front-end implementation was merged on #1535\r\n\r\nI'd like to try it out, just need some I need some details about how to proceed.","issue_id":1660204174431,"origin_id":841192948,"user_origin_id":30418115,"create_time":1620992708,"update_time":1620992708,"id":1669215574284,"updated_at":"2022-11-23T14:59:34.284000Z","created_at":"2022-11-23T14:59:34.284000Z"},{"_id":"637e355670db72139b0e9d96","body":"This seems to be a duplicate of #2580 \r\n\r\nI've looked at this 2 weeks back and the JS library used (searchable option list) would need some work. By adding `multiple` to the select it actually starts to perform like the project selection - it starts to show bubbles for every item selected which is not wanted.","issue_id":1660204174431,"origin_id":841195814,"user_origin_id":12401160,"create_time":1620993111,"update_time":1620993111,"id":1669215574287,"updated_at":"2022-11-23T14:59:34.287000Z","created_at":"2022-11-23T14:59:34.287000Z"}]

The file type could be capable to enable more than one value 

tulinkry

Comment On/Off Feature

[{"_id":"637e327d70db72139b0e9ad3","body":"How do you perform the search exactly ? Is it full text \/ symbol \/ definition search ? Can you post an example of the code ?","issue_id":1660204174434,"origin_id":838168393,"user_origin_id":2934284,"create_time":1620726078,"update_time":1620726078,"id":1669214845839,"updated_at":"2022-11-23T14:47:25.838000Z","created_at":"2022-11-23T14:47:25.838000Z"},{"_id":"637e327d70db72139b0e9ad4","body":"It is full search.\r\n\r\nExample below:\r\nSearched for **geometry** in FULL SEARCH -\r\n```\r\n34 \/\/ removal of UPPERCASE geometry\r\n86 PPG_FACE(class surface *geometry\/*=NULL*\/, REVBIT sense\/*=FORWARD*\/,\r\n89 PPG_FACE(class surface *geometry\/*=NULL*\/);\r\n282 \/\/return the surface geometry (a surface)\r\n289 \/\/set the surface geometry\r\n306 inline PPG_FACE::PPG_FACE(class surface *geometry, REVBIT sense, in PPG_FACE() argument\r\n308 : FACE(geometry, sense, type) in PPG_FACE()\r\n327 inline PPG_FACE::PPG_FACE(class surface *geometry, REVBIT sense) in PPG_FACE() argument\r\n328 : FACE(geometry, sense, PPG_FACE::type) in PPG_FACE()\r\n336 inline PPG_FACE::PPG_FACE(class surface *geometry) in PPG_FACE() argument\r\n```\r\nif you see line no. 34, 282 & 289, these lines are commented out","issue_id":1660204174434,"origin_id":838306992,"user_origin_id":82038972,"create_time":1620732413,"update_time":1620734467,"id":1669214845842,"updated_at":"2022-11-23T14:47:25.841000Z","created_at":"2022-11-23T14:47:25.841000Z"},{"_id":"637e327d70db72139b0e9ad5","body":"Yes, the full search performs search in all the tokens, i.e. including those in the comments. If you want just references or definitions, use Definition\/Symbol search.","issue_id":1660204174434,"origin_id":838353550,"user_origin_id":2934284,"create_time":1620734620,"update_time":1620897140,"id":1669214845845,"updated_at":"2022-11-23T14:47:25.844000Z","created_at":"2022-11-23T14:47:25.844000Z"},{"_id":"637e327d70db72139b0e9ad6","body":"References or definitions search has a limited scope. I want full search without comments","issue_id":1660204174434,"origin_id":839414495,"user_origin_id":82038972,"create_time":1620792026,"update_time":1620792026,"id":1669214845847,"updated_at":"2022-11-23T14:47:25.847000Z","created_at":"2022-11-23T14:47:25.847000Z"},{"_id":"637e327d70db72139b0e9ad7","body":"What do you mean by \"limited scope\" ?","issue_id":1660204174434,"origin_id":839563929,"user_origin_id":2934284,"create_time":1620807229,"update_time":1620807229,"id":1669214845849,"updated_at":"2022-11-23T14:47:25.849000Z","created_at":"2022-11-23T14:47:25.849000Z"},{"_id":"637e327d70db72139b0e9ad8","body":"It does not search in string if we have set it to references or definitions","issue_id":1660204174434,"origin_id":839571270,"user_origin_id":82038972,"create_time":1620807622,"update_time":1620807622,"id":1669214845852,"updated_at":"2022-11-23T14:47:25.851000Z","created_at":"2022-11-23T14:47:25.851000Z"},{"_id":"637e327d70db72139b0e9ad9","body":"What do you mean by 'search in string' exactly ? Maybe give an example ?","issue_id":1660204174434,"origin_id":840430081,"user_origin_id":2934284,"create_time":1620897211,"update_time":1620897211,"id":1669214845854,"updated_at":"2022-11-23T14:47:25.854000Z","created_at":"2022-11-23T14:47:25.854000Z"}]

I am using OpenGrok 1.7.2 I would like to request you to add a new feature of **Comment On/Off** Usually, Opengrok searches for a keyword and shows the results having...

AmeyaBhujbal