confluencebuilder

confluencebuilder copied to clipboard

Exception in builder.py if self.legacy_pages is None

[{"_id":"6635cf32b7a25c1c6114346d","body":"Thanks for reporting this.\r\n\r\nChanges (#881) have been made to correct this exception case. This change has been added into the main branch and should be made available next stable release. At this time, this issue does not affect instances using `confluence_cleanup_archive` or `confluence_cleanup_purge`. If blocked by this issue, it is recommended to either use the [development version](https:\/\/sphinxcontrib-confluencebuilder.readthedocs.io\/en\/stable\/install\/#development-installation) of this extension, or switch back to an older version of this extension.","issue_id":1710114618820,"origin_id":1962412543,"user_origin_id":1834509,"create_time":1708791234,"update_time":1708791234,"id":1714802482160,"updated_at":"2024-05-04T06:01:22.159000Z","created_at":"2024-05-04T06:01:22.159000Z"},{"_id":"6635cf32b7a25c1c6114346e","body":"That is excellent news!\r\n\r\nI am running the main branch now so I could implement a quick fix to the\r\nproblem. I am planning to use one of the cleanup options but did not test\r\nthem right away because the doc said they are unsupported.\r\n\r\n\r\n\r\nOn Sat, Feb 24, 2024 at 11:14\u202fAM James Knight ***@***.***>\r\nwrote:\r\n\r\n> Thanks for reporting this.\r\n>\r\n> Changes (#881\r\n> <https:\/\/github.com\/sphinx-contrib\/confluencebuilder\/pull\/881>) have been\r\n> made to correct this exception case. This change has been added into the\r\n> main branch and should be made available next stable release. At this time,\r\n> this issue does not affect instances using confluence_cleanup_archive or\r\n> confluence_cleanup_purge. If blocked by this issue, it is recommended to\r\n> either use the development version\r\n> <https:\/\/sphinxcontrib-confluencebuilder.readthedocs.io\/en\/stable\/install\/#development-installation>\r\n> of this extension, or switch back to an older version of this extension.\r\n>\r\n> \u2014\r\n> Reply to this email directly, view it on GitHub\r\n> <https:\/\/github.com\/sphinx-contrib\/confluencebuilder\/issues\/880#issuecomment-1962412543>,\r\n> or unsubscribe\r\n> <https:\/\/github.com\/notifications\/unsubscribe-auth\/AASTPM5UWXSQAIN2ZRAD5IDYVIGU3AVCNFSM6AAAAABDW2U4JKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRSGQYTENJUGM>\r\n> .\r\n> You are receiving this because you authored the thread.Message ID:\r\n> ***@***.***>\r\n>\r\n","issue_id":1710114618820,"origin_id":1962616955,"user_origin_id":2439091,"create_time":1708803135,"update_time":1708803135,"id":1714802482164,"updated_at":"2024-05-04T06:01:22.164000Z","created_at":"2024-05-04T06:01:22.164000Z"},{"_id":"6635cf32b7a25c1c6114346f","body":"v2.5 is now available on PyPI -- marking as closed.","issue_id":1710114618820,"origin_id":2029073946,"user_origin_id":1834509,"create_time":1711940300,"update_time":1711940300,"id":1714802482168,"updated_at":"2024-05-04T06:01:22.168000Z","created_at":"2024-05-04T06:01:22.168000Z"}]

Sphinx version: 5.3.0 Python version: 3.11.6 (CPython) Docutils version: 0.19 Jinja2 version: 3.1.2 Last messages: done publishing assets... [ 12%] AmbaUSB-1-opened.png publishing assets... [ 25%] AmbaUSB-2-cv25.png publishing assets... [ 37%]...

jameyhicks

Proxy Error When Publishing

[{"_id":"663a7d44fa3686d6a11632cb","body":"@DanielJerrehian, sorry, I'm not sure off hand why you may be experience these connection issues.\r\n\r\nMind generating a `python -m sphinxcontrib.confluencebuilder report` output?\r\n\r\nAssuming you can access Confluence via your browser, you could try to see if you can query the REST API from your browser as well. For example, venturing to a page like `https:\/\/DomainName.CompanyName.TopLevelDomain\/rest\/api\/space` (assuming that's the proper endpoint for your Confluence instances), and you should see a JSON-like response in the browser.\r\n\r\nIf you do, next possible check would be to try the same request in the terminal\/shell you may be using. For example, invoking one of these calls and seeing a JSON-like response:\r\n```\r\ncurl https:\/\/DomainName.CompanyName.TopLevelDomain\/rest\/api\/space\r\n or\r\nwget -qO- https:\/\/DomainName.CompanyName.TopLevelDomain\/rest\/api\/space\r\n or\r\nInvoke-WebRequest -Uri https:\/\/DomainName.CompanyName.TopLevelDomain\/rest\/api\/space\r\n```\r\n\r\nAnd if those work, lastly, you can tree a publish attempt using the `confluence_publish_debug = True` option set. This may print some helpful debug messages to the standard output.","issue_id":1710114618826,"origin_id":1913768905,"user_origin_id":1834509,"create_time":1706486534,"update_time":1706486534,"id":1715109188213,"updated_at":"2024-05-07T19:13:08.213000Z","created_at":"2024-05-07T19:13:08.213000Z"},{"_id":"663a7d44fa3686d6a11632cc","body":"Remove the http from your proxy settings.","issue_id":1710114618826,"origin_id":1945478650,"user_origin_id":133498,"create_time":1707980755,"update_time":1707980755,"id":1715109188221,"updated_at":"2024-05-07T19:13:08.221000Z","created_at":"2024-05-07T19:13:08.221000Z"},{"_id":"663a7d44fa3686d6a11632cd","body":"> Remove the http from your proxy settings.\r\n\r\nThis can be valid in select scenarios. This would depend on the version of urllib3 installed (used by Requests). Newer versions of the library expect a leading scheme, where older versions do not. For users who may be experiencing proxy-related issues, it is recommended to update their Requests\/urllib3 to a more recent version (if possible).\r\n\r\n----\r\n\r\nIt has been three months since this initial request has been made, but no new information has been discussed related to OPs user experience. It is assumed that the issue has been resolved, or has been dropped. With that in mind, this issue will be closed. Feel free to create a new issue if needed in the future.","issue_id":1710114618826,"origin_id":2027869715,"user_origin_id":1834509,"create_time":1711763652,"update_time":1711763652,"id":1715109188241,"updated_at":"2024-05-07T19:13:08.240000Z","created_at":"2024-05-07T19:13:08.240000Z"}]

I am attempting to publish Sphinx documentation on an intranet Confluence page which has a server URL similar to the following: `https://DomainName.CompanyName.TopLevelDomain` When I run the command `python -m sphinx...

DanielJerrehian

Replay HTTP requests against an air-gapped Confluence server

[{"_id":"66cf9f66b3dc7ee8eb10f947","body":"This topic was discussed in https:\/\/github.com\/sphinx-contrib\/confluencebuilder\/issues\/627 last year, if you would like to read up about the opinions\/issues at the time.\r\n\r\nNote that part of the hesitation outlined in these comments were due to some prospect implementation in supporting Confluence' ADF format (in part to support the v2 editor style). The implementation would require a multi-pass publishing design, otherwise Confluence would reject uploads. Since this extension was able to support the v2 editor while still using the storage format, publishing should be as simple as it is right now in the current implementation. So, when considering the other issue's comments, it may be somewhat easier to implement a separation between building and publishing.\r\n\r\nWhile my first impressions think that a replay functionality should not be something directly implemented in this extension, I do think it's a great idea for your use case. Isolating request data made should be easily translatable to PowerShell (or whatever) scripts. While the suggested approach of wrapping on the requests library should work, I wonder if it would be simpler if this extension supported an advanced configuration or introduce extension hooks which allows a user to override either the `ConfluencePublisher` or `Rest` classes without needing to use mock implementation (but I guess it doesn't really matter what is used; whatever works for this use case). This Confluence builder extension is always willing to create hooks or support overrides from the configuration for existing features, if power users want an easier way to perform custom tweaks.\r\n\r\nThe biggest thing to consider when looking at the separation between building and publishing is not so much the generated documentation set, but the dynamic values needed in some of the data requests. We could initially ignore some features like purging old pages or publishing on updated pages\/attachments, and just focus on publishing an entire documentation\/attachment set. One example to consider is that the `version`'s [`number`](https:\/\/github.com\/sphinx-contrib\/confluencebuilder\/blob\/v2.1.1\/sphinxcontrib\/confluencebuilder\/publisher.py#L1118), which on new pages is set to one (or not provided). However, a publish attempt is updating an existing document, this value must be set to the existing version number of the page [plus one](https:\/\/github.com\/sphinx-contrib\/confluencebuilder\/blob\/v2.1.1\/sphinxcontrib\/confluencebuilder\/publisher.py#L1162). This can make it hard if you are trying to do a dumb-replay publish ~ you may have to add some smarts to know a request needs an update and tweaks the version value. Alternatively, you could also just crudely wipe the entire space and publish a new set with all versions being `1`.\r\n\r\nAs for the Confluence Builder extension, my current opinion on supporting the separation of building and publishing is as follows. I am in favor to add support for such a feature. I would not say it is a priority feature to get out by next release, but it could be the next big feature we aim to bring in this extension. The following overview is how I would imagine such a design would be done:\r\n\r\n```mermaid\r\nflowchart LR\r\n P --> S\r\n Tpp --> S\r\n E --> A\r\n A -.-> Tpp\r\n A -.-> I\r\n subgraph Sphinx\/Confluence Builder\r\n C[Initialization] --> B\r\n C --> I\r\n B[Builder] --> P\r\n B --> E\r\n E[Export]\r\n I[Import] --> P\r\n P[Publisher]\r\n end\r\n A[Archive]\r\n subgraph Third Party\r\n Tpp[Publisher]\r\n end\r\n subgraph \"Confluence\"\r\n S[Space]\r\n end\r\n```\r\n\r\nI'm welcome to other design considerations as well.\r\n\r\nFinally, I would imagine the timelines wanted for OPs requirements may not be align with the timelines of this extension providing full support for build\/publish separation. For sure it should be possible to tweak this extension in order to be flexible for a third-party replay functionality (if tweaks to this extension are needed), but a serious look at a full fledged solution may not be looked into (by this maintainer) until possible Fall, if not next year.","issue_id":1710114618831,"origin_id":1627469891,"user_origin_id":1834509,"create_time":1688843465,"update_time":1688843465,"id":1724882790813,"updated_at":"2024-08-28T22:06:30.812000Z","created_at":"2024-08-28T22:06:30.812000Z"},{"_id":"66cf9f66b3dc7ee8eb10f948","body":"Marry me! \ud83e\udd79 Thank you so much for such a quick and precise reply!\r\n\r\n> Alternatively, you could also just crudely wipe the entire space and publish a new set with all versions being 1.\r\n\r\nThat's exactly what i had in mind. \ud83d\ude04 Quick-n-Nasty!\r\n\r\nShould this become a feature and it will be implemented similar to what @jdknight proposes, i'd volunteer for providing a reference implementation of a third-party publisher. Having a plan B is always nice, so i don't see a problem in me dedicating time for this, probably Q4 '23.","issue_id":1710114618831,"origin_id":1627487984,"user_origin_id":134346897,"create_time":1688849881,"update_time":1688849881,"id":1724882790818,"updated_at":"2024-08-28T22:06:30.818000Z","created_at":"2024-08-28T22:06:30.818000Z"},{"_id":"66cf9f66b3dc7ee8eb10f949","body":"UPDATE: I've upgraded the boilerplate since the interims solution will be in place longer than i would like to. Now, the boilerplate suppresses any connection to a Confluence instance and dumps contents and attachments to seperate files (had issues with Anti-Virus when normalized). The schema is now redundant.\r\n\r\nAttached you'll find my quick-n-dirty boilerplate implementation of the wrapper. Fixing up a custom Rest Client in PowerShell\/Perl\/whatever can be achieved in less than 50 lines, hence i don't see the point in sharing.\r\n\r\nMaybe this will be useful for someone else.\r\n\r\n@jdknight if i can support the project with any feature implementations, let me know. I feel comfortable with the code now.\r\n\r\nIn addition, i've appended a schema for the output. \r\n\r\nBy default the dump will go to `sphinx.config.Config.outdir` \/ 'confluence.out'.\r\n\r\nOne can use the schema in conjunction with e.g. the `TestJSON` PowerShell cmdlet.\r\n\r\nAttachment data is decoded as *ISO-8859-1*.\r\n\r\n```python\r\n#!\/usr\/bin\/env python3\r\n\"\"\"Publishment delay wrapper sphinxcontrib.confluencebuilder\r\n\r\nThis is a lightweight pass-through wrapper for \r\n``sphinxcontrib.confluencebuilder``, which intercepts all ``store_*`` calls on \r\na ``ConfluencePublisher`` instance, dumps all data into interchange. \r\nThe index and dumps can be used in conjunction with the PowerShell \r\nhelper to delay\/replay the publishment of pages and attachments for a different \r\nConfluence instance, than what the programmatic target is.\r\n\r\nThe builder name is ``x_confluence``\r\n\r\n``Publisher``, ``Builder``, as well as ``Rest`` instances are mocked and are \r\nsupressing any HTTP connectivity should the ``confluence_publish_dry_run`` be \r\nset to ``True``. \r\n\r\n.. warning::\r\n\r\n ``confluence_publish_dry_run`` MUST be set to ``True``\r\n\r\nContent (pages) and attachments are dumped into separate files and indexed. \r\n\r\nThe output directory can be set through ``x_confluence_outdir``.\r\n\r\nThe use-case for this implementation is as follows:\r\n\r\n I am currently facing the situation where i need to publish to an \r\n air-gapped Confluence server inside a virtualised and privatised \r\n environment (over Windows VDI). In addition, company policy forbids me from \r\n using, or installing Python \r\n on the VDI. I can freeload a perl executable that came bundled with Git for \r\n Windows, but there are no other scripting means besides Windows PowerShell.\r\n\r\n\"\"\"\r\n__author__ = '[email protected]'\r\n__copyright__ = 'adesso SE'\r\n__license__ = 'DL-DE-BY-2.0'\r\n\r\nfrom dataclasses import dataclass, asdict\r\nimport json\r\nfrom mimetypes import guess_extension\r\nfrom pathlib import Path\r\nfrom typing import Any, Optional, ByteString, Dict, Tuple, List\r\nfrom unittest.mock import patch\r\nfrom uuid import uuid4\r\n\r\nfrom sphinx.application import Sphinx\r\nfrom sphinx.util import logging\r\n\r\nfrom sphinxcontrib.confluencebuilder import setup as _setup\r\nfrom sphinxcontrib.confluencebuilder.builder import (\r\n ConfluenceBuilder as _ConfluenceBuilder\r\n)\r\nfrom sphinxcontrib.confluencebuilder.publisher import (\r\n ConfluencePublisher as _ConfluencePublisher\r\n)\r\nfrom sphinxcontrib.confluencebuilder.rest import Rest as _Rest\r\n\r\n\r\nlogger = logging.getLogger(__name__)\r\n\r\n\r\n\r\n@dataclass\r\nclass ConfluenceContentMeta:\r\n \"\"\"\r\n see\r\n `https:\/\/docs.atlassian.com\/ConfluenceServer\/rest\/8.4.0\/#api\/content-createContent`_,\r\n for more information\r\n \"\"\"\r\n #:\r\n title: str\r\n\r\n #: \r\n ancestor_id: str\r\n\r\n\r\n@dataclass\r\nclass ConfluenceChildAttachmentMeta:\r\n \"\"\"\r\n\r\n \"\"\"\r\n #: \r\n container_id: str\r\n\r\n #: \r\n name: str\r\n\r\n #: \r\n mimetype: str\r\n\r\n\r\n\r\n@dataclass\r\nclass ConfluencePublisherDump:\r\n \"\"\"\r\n \"\"\"\r\n #: \r\n pages: Dict[str, ConfluenceContentMeta]\r\n\r\n #: \r\n attachments: Dict[str, ConfluenceChildAttachmentMeta]\r\n\r\n\r\nclass Rest(_Rest):\r\n \"\"\"\r\n \"\"\"\r\n def __init__(self, config):\r\n \"\"\"\r\n \"\"\"\r\n\r\n super().__init__(config)\r\n\r\n def __getattr__(self, name: str) -> Any:\r\n \"\"\"\r\n \"\"\"\r\n print('Hallo')\r\n \r\n return super().__getattribute__(name)\r\n\r\n def __setattr__(self, name: str, value: Any) -> None:\r\n \"\"\"\r\n \"\"\"\r\n return super().__setattr__(name, value)\r\n\r\n def get(self, key, params=None):\r\n \"\"\"\r\n \"\"\"\r\n from pprint import pprint\r\n\r\n return {'results': [\r\n {\r\n 'id': 776536065,\r\n 'key': self.config.confluence_space_key,\r\n 'name': 'Testitest',\r\n 'type': 'personal'\r\n }\r\n ], 'size': 1, 'limit': 1, 'start': 0}\r\n\r\n\r\nclass ConfluencePublisher(_ConfluencePublisher):\r\n \"\"\"\r\n \"\"\"\r\n def __init__(self):\r\n \"\"\"\r\n \"\"\"\r\n super().__init__()\r\n\r\n self.dump = ConfluencePublisherDump(\r\n pages = {},\r\n attachments = {}\r\n )\r\n\r\n def __getattr__(self, name: str) -> Any:\r\n \"\"\"\r\n \"\"\"\r\n return super().__getattribute__(name)\r\n\r\n def __setattr__(self, name: str, value: Any) -> None:\r\n \"\"\"\r\n \"\"\"\r\n return super().__setattr__(name, value)\r\n\r\n def connect(self):\r\n \"\"\"initialize a REST client and probe the target Confluence instance\r\n\r\n .. note::\r\n\r\n Actually, i don't want the extension to initialize a connection, \r\n but there is too much entanglement, so we're mocking the absolute\r\n minimum for the publisher object to assume everything is fine\r\n \"\"\"\r\n with patch('sphinxcontrib.confluencebuilder.publisher.Rest', Rest):\r\n\r\n return super().connect()\r\n\r\n def get_page_by_id(self, page_id, expand = 'version') -> Tuple[None, List]:\r\n \"\"\"get page information with the provided page name\r\n\r\n :param page_id: the page identifier\r\n :param expand: data to expand on\r\n\r\n :returns: page id and page object\r\n \"\"\"\r\n return (None, [])\r\n\r\n def store_attachment(\r\n self,\r\n page_id: str,\r\n name: str,\r\n data: Any,\r\n mimetype: Any,\r\n hash_: str,\r\n force: bool = False\r\n ) -> str:\r\n \"\"\"request to store an attachment on a provided page\r\n\r\n :returns: the attachment identifier\r\n \"\"\"\r\n logger.info('pass-through intercept: store_attachment')\r\n\r\n attachment_id = uuid4()\r\n\r\n mime_extension = guess_extension(mimetype, False)\r\n\r\n if mime_extension:\r\n\r\n attachment_id = f'{attachment_id}{mime_extension}'\r\n\r\n file = (Path(getattr(self.config, 'x_confluence_outdir')) \/ \r\n 'attachments' \/ attachment_id)\r\n\r\n file.parent.mkdir(parents=True, exist_ok=True)\r\n\r\n file.write_bytes(data)\r\n\r\n self.dump.attachments[attachment_id] = ConfluenceChildAttachmentMeta(\r\n container_id = page_id,\r\n name = name,\r\n mimetype= mimetype\r\n )\r\n\r\n return attachment_id\r\n\r\n def store_page(\r\n self,\r\n page_name: str,\r\n data: Any,\r\n parent_id: Optional[str] = None\r\n ) -> str:\r\n \"\"\"request to store page information to a confluence instance\r\n\r\n :param page_name: the page title to use on the updated page\r\n :param data: the page data to apply\r\n :param parent_id: the id of the ancestor to use\r\n\r\n :returns: id of uploaded page\r\n \"\"\"\r\n logger.info('pass-through intercept: store_page')\r\n\r\n content_id = str(uuid4())\r\n\r\n file = (Path(getattr(self.config, 'x_confluence_outdir')) \/ 'content' \/\r\n f'{content_id}.xml')\r\n\r\n file.parent.mkdir(parents=True, exist_ok=True)\r\n\r\n file.write_bytes(data['content'].encode('utf-8'))\r\n\r\n self.dump.pages[content_id] = ConfluenceContentMeta(\r\n title = page_name,\r\n ancestor_id = parent_id\r\n )\r\n\r\n return content_id\r\n\r\n def store_page_by_id(\r\n self,\r\n page_name: str,\r\n page_id: str,\r\n data: Any\r\n ) -> str:\r\n \"\"\"request to store page information on the page with a matching id\r\n\r\n :param page_name: the page title to use on the updated page\r\n :param data: the page data to apply\r\n :param parent_id: the id of the ancestor to use\r\n\r\n :returns: id of uploaded page\r\n \"\"\"\r\n logger.info('pass-through intercept: store_page_by_id')\r\n\r\n return 'NULL'\r\n\r\n def disconnect(self):\r\n \"\"\"terminate the REST client\r\n\r\n .. note::\r\n\r\n Freeloading this method to dump the index.\r\n \"\"\"\r\n file = Path(getattr(self.config, 'x_confluence_outdir')) \/ 'data.json'\r\n\r\n\r\n file.parent.mkdir(parents = True, exist_ok=True)\r\n\r\n raw = json.dumps(asdict(self.dump), indent=4)\r\n\r\n file.write_text(raw)\r\n\r\n logger.info(f'content dump count: {len(self.dump.pages)}')\r\n\r\n logger.info(f'attachments dump count: {len(self.dump.attachments)}')\r\n\r\n logger.info(f'dump index: {file}')\r\n\r\n\r\nclass ConfluenceBuilder(_ConfluenceBuilder):\r\n \"\"\"\r\n \"\"\"\r\n name = 'x_confluence'\r\n\r\n def __init__(self, app: Sphinx, env = None):\r\n \"\"\"\r\n \"\"\"\r\n patch_target = ('sphinxcontrib.confluencebuilder'\r\n '.builder.ConfluencePublisher')\r\n\r\n with patch(patch_target, ConfluencePublisher):\r\n\r\n super().__init__(app, env)\r\n\r\n\r\n def __getattribute__(self, name: str) -> Any:\r\n \"\"\"\r\n \"\"\"\r\n return super().__getattribute__(name)\r\n\r\n\r\n def __setattr__(self, name: str, value: Any) -> None:\r\n \"\"\"\r\n \"\"\"\r\n return super().__setattr__(name, value)\r\n\r\n\r\ndef setup(app: Sphinx):\r\n \"\"\"\r\n \"\"\"\r\n patch_target = 'sphinxcontrib.confluencebuilder.ConfluenceBuilder'\r\n\r\n app.add_config_value(\r\n name = 'x_confluence_outdir',\r\n default = str(Path(app.outdir) \/ 'confluence.out'),\r\n rebuild = True\r\n )\r\n\r\n with patch(patch_target, ConfluenceBuilder):\r\n\r\n logger.info(f'patching: {patch_target}')\r\n\r\n return _setup(app)\r\n```\r\n\r\n```json\r\n{\r\n\t\"$id\": \"https:\/\/github.com\/tiara-adessi\/confluencebuilder\/schema\/top\",\r\n \"x-authors\": [\r\n \"[email protected]\"\r\n ],\r\n \"type\": \"object\",\r\n \"properties\": {\r\n \"pages\": {\r\n \"type\": \"array\",\r\n \"items\": {\r\n \"$ref\": \"#\/definitions\/page\"\r\n }\r\n },\r\n \"attachments\": {\r\n \"type\": \"array\",\r\n \"items\": {\r\n \"$ref\": \"#\/definitions\/attachment\"\r\n }\r\n }\r\n },\r\n \"required\": [\r\n \"pages\",\r\n \"attachments\"\r\n ],\r\n \"definitions\": {\r\n \"page\": {\r\n \"type\": \"object\",\r\n \"properties\": {\r\n \"page_name\": {\r\n \"type\": \"string\"\r\n },\r\n \"page_id\": {\r\n \"type\": \"string\"\r\n },\r\n \"parent_id\": {\r\n \"type\": \"string\"\r\n },\r\n \"data\": {\r\n \"type\": \"object\",\r\n \"properties\": {\r\n \"content\": {\r\n \"type\": \"string\"\r\n },\r\n \"labels\": {\r\n \"type\": \"array\",\r\n \"items\": {\r\n \"type\": \"string\"\r\n }\r\n }\r\n },\r\n \"required\": [\r\n \"content\",\r\n \"labels\"\r\n ]\r\n }\r\n },\r\n \"required\": [\r\n \"page_name\",\r\n \"page_id\",\r\n \"parent_id\",\r\n \"data\"\r\n ]\r\n },\r\n \"attachment\": {\r\n \"type\": \"object\",\r\n \"properties\": {\r\n \"page_id\": {\r\n \"type\": \"string\"\r\n },\r\n \"name\": {\r\n \"type\": \"string\"\r\n },\r\n \"data\": {\r\n \"type\": \"string\"\r\n },\r\n \"mimetype\": {\r\n \"type\": \"string\"\r\n },\r\n \"attachment_id\": {\r\n \"type\": \"string\"\r\n }\r\n },\r\n \"required\": [\r\n \"page_id\",\r\n \"name\",\r\n \"data\",\r\n \"mimetype\",\r\n \"attachment_id\"\r\n ]\r\n }\r\n }\r\n}\r\n```","issue_id":1710114618831,"origin_id":1636884962,"user_origin_id":134346897,"create_time":1689454554,"update_time":1689790232,"id":1724882790822,"updated_at":"2024-08-28T22:06:30.822000Z","created_at":"2024-08-28T22:06:30.822000Z"},{"_id":"66cf9f66b3dc7ee8eb10f94a","body":"Thanks for the example, provides some good insights to understand what is desired.\r\n\r\nI'm curious about the schema definition, since I cannot say I've created a schema definition file for JSON before. Noticed there is the use of `type`, `properties`, `required`, etc. which looks to be able to support a way to programmatically understand the schema definition. Is this based off a common practice (e.g. using these specific keys)? Manually created or was this created with something else?","issue_id":1710114618831,"origin_id":1646662024,"user_origin_id":1834509,"create_time":1690056340,"update_time":1690056340,"id":1724882790827,"updated_at":"2024-08-28T22:06:30.827000Z","created_at":"2024-08-28T22:06:30.827000Z"},{"_id":"66cf9f66b3dc7ee8eb10f94b","body":"That was a very nice subtle hint, that i was missing the [`$schema`](https:\/\/datatracker.ietf.org\/doc\/html\/draft-bhutton-json-schema-01#name-the-schema-keyword) keyword (though it's not mandatory). \ud83d\ude04\r\n\r\nYes, these are common practices as defined in the JSONSchema specification(s).\r\n\r\n There are some pretty good validators out there (e.g., I use [this](https:\/\/pypi.org\/project\/jsonschema\/) one in Python, and [this](https:\/\/www.npmjs.com\/package\/jsonschema) one in Node.js, and [this](https:\/\/metacpan.org\/pod\/JSON::Schema) one in Perl).\r\nThe nice thing about Microsoft PowerShell is, that they have a built-in [Newtonsoft](https:\/\/www.newtonsoft.com\/json) validator through the [`Test-JSON`](https:\/\/learn.microsoft.com\/en-us\/powershell\/module\/microsoft.powershell.utility\/test-json?view=powershell-7.3) cmdlet, which is available for PowerShell Core and Desktop. So Microsoft seems to be leading the way into making JSONSchema actually more practical (it's stuck in the *forever-draft* dimension)\r\n\r\nYes, the schema was created manually and i tend to use older versions of the specification to author them, so that it stays compatible across multiple validators.\r\n\r\nBesides the [`$ref`](https:\/\/datatracker.ietf.org\/doc\/html\/draft-bhutton-json-schema-01#name-schema-references) reference resolution (the specification is vague about that, therefore there are multiple understandings and implementation approaches of that) it is pretty straight forward.\r\n\r\nBtw. i refactored the code once more as it turns out, that my temporary solution probably has to stay long-term as the promised CI\/CD environment won't suffice. I'll share the repos some day next week.\r\n","issue_id":1710114618831,"origin_id":1657120254,"user_origin_id":134346897,"create_time":1690718190,"update_time":1690718582,"id":1724882790831,"updated_at":"2024-08-28T22:06:30.831000Z","created_at":"2024-08-28T22:06:30.831000Z"},{"_id":"66cf9f66b3dc7ee8eb10f94c","body":"Sorry for the delayed (promised) notice.\r\n\r\nWe now have published two programs and made this an open-source effort:\r\n\r\n- [xconfluencebuilder](https:\/\/bitbucket.org\/victorykit\/xconfluencebuilder\/src\/master\/) (which is the patched version of `sphinxcontrib.confluencebuilder`)\r\n- [PSConfluencePublisher](https:\/\/bitbucket.org\/victorykit\/psconfluencepublisher\/src\/master\/)\r\n\r\n`xconfluencebuilder` generates the manifest (including referenced content) of archived pages\/attachments and `PSConfluencePublisher` does the publishing (also works with PowerShell 5 [Desktop]). `PSConfluencePublisher` is now also able to do unidirectional synchronization and caches all metadata locally as to reduce the amount of network chatter.\r\n\r\nWe're hoping the reference implementations suffice for you giving it a test drive.\r\n\r\nThe manifest schema (currently only part of the `PSConfluencePublisher` repo) has been adapted to:\r\n\r\n```json\r\n{\r\n \"$id\": \"https:\/\/spec.victory-k.it\/psconfluencepublisher.json\",\r\n \"x-authors\": [\r\n \"[email protected]\"\r\n ],\r\n \"type\": \"object\",\r\n \"properties\": {\r\n \"Pages\": {\r\n \"type\": \"array\",\r\n \"item\": {\r\n \"$ref\": \"#\/definitions\/page\"\r\n }\r\n },\r\n \"Attachments\": {\r\n \"type\": \"array\",\r\n \"item\": {\r\n \"$ref\": \"#\/definitions\/attachment\"\r\n }\r\n }\r\n },\r\n \"required\": [\r\n \"Pages\",\r\n \"Attachments\"\r\n ],\r\n \"definitions\": {\r\n \"page\": {\r\n \"type\": \"object\",\r\n \"description\": \"Local Confluence page\/container attachment metadata\",\r\n \"properties\": {\r\n \"Title\": {\r\n \"type\": \"string\",\r\n \"description\": \"Title of page\"\r\n },\r\n \"Id\": {\r\n \"type\": \"string\",\r\n \"description\": \"Id of attachment defined by Confluence instance. The id is generated after the publishing of a page.\"\r\n },\r\n \"Version\": {\r\n \"type\": \"string\"\r\n },\r\n \"Hash\": {\r\n \"type\": \"string\",\r\n \"description\": \"SHA512 hexadecimal content hash value\"\r\n },\r\n \"Ref\": {\r\n \"type\": \"string\",\r\n \"description\": \"Local filesystem reference\/path\"\r\n },\r\n \"AncestorTitle\": {\r\n \"type\": \"string\",\r\n \"description\": \"Title of Confluence page this page is a child of. The title must be a property key of the pages object.\"\r\n }\r\n },\r\n \"required\": [\r\n \"Title\",\r\n \"Ref\"\r\n ]\r\n },\r\n \"attachment\": {\r\n \"type\": \"object\",\r\n \"description\": \"Local Confluence page\/container attachment metadata\",\r\n \"properties\": {\r\n \"Name\": {\r\n \"type\": \"string\",\r\n \"description\": \"name of attachment, which must be unique within the container page\"\r\n },\r\n \"Id\": {\r\n \"type\": \"string\",\r\n \"description\": \"Id of attachment defined by Confluence instance. The id is generated after the publishing of an attachment.\"\r\n },\r\n \"Hash\": {\r\n \"type\": \"string\",\r\n \"description\": \"SHA512 hexadecimal attachment content hash value\"\r\n },\r\n \"MimeType\": {\r\n \"type\": \"string\",\r\n \"description\": \"MIME type of attachment\",\r\n \"default\": \"binary\/octet-stream\"\r\n },\r\n \"ContainerPageTitle\": {\r\n \"type\": \"string\",\r\n \"description\": \"Title of Confluence page this attachment is contained in. The title must be a property key of the pages object.\"\r\n },\r\n \"Ref\": {\r\n \"type\": \"string\",\r\n \"description\": \"Local filesystem reference\/path\"\r\n }\r\n },\r\n \"required\": [\r\n \"Name\",\r\n \"Hash\",\r\n \"MimeType\",\r\n \"ContainerPageTitle\",\r\n \"Ref\"\r\n ]\r\n }\r\n }\r\n}\r\n```","issue_id":1710114618831,"origin_id":1676646194,"user_origin_id":134346897,"create_time":1691986521,"update_time":1691986521,"id":1724882790835,"updated_at":"2024-08-28T22:06:30.834000Z","created_at":"2024-08-28T22:06:30.834000Z"}]

Hi all, i am currently facing the situation where i need to **publish to an air-gapped Confluence server inside a virtualised and privatised environment (over Windows VDI)**. In addition, company...

ByteB4rb1e

Uploading wrong attachments for drawio

[{"_id":"66498c7d4a8e87d14e196b80","body":"Thanks for the report -- there looks to be something up.\r\n\r\nUsing the [drawio sample set](https:\/\/github.com\/sphinx-contrib\/confluencebuilder\/tree\/main\/tests\/sample-sets\/drawio), a run appears to render drawio content as expected. When trying to match the configuration exactly, the same results. Although, when inspecting the Confluence instance being published to, both the `.drawio` and `.svg` extensions appear to be uploaded as attachment. When inspecting the output\/implementation, it looks like the following occurs:\r\n\r\n1. confluencebuilder performs its image processing check to see what images are to be uploaded. It finds (for example) an image candidate entry and registers it into the asset manager:\r\n\r\n ```\r\n <image candidates=\"{'*': 'diagram\/box.drawio'}\" classes=\"['drawio']\" uri=\"diagram\/box.drawio\"\/>\r\n ```\r\n\r\n2. At some point, the drawio extension will update the image with new candidate values. When confluencebuilder performs its second pass of finding any new images generated from other events, it then detects the SVG one.\r\n\r\n ```\r\n <image candidates=\"{'*': '.\\\\output\\\\py\\\\sample-drawio\\\\.doctrees\\\\drawio\\\\c10f10d8c2d1aeef4d59a7ca6354e021557e3d8b\\\\box.svg'}\" classes=\"['drawio']\" uri=\".\\output\\py\\sample-drawio\\.doctrees\\drawio\\c10f10d8c2d1aeef4d59a7ca6354e021557e3d8b\\box.svg\"\/>\r\n ```\r\n\r\n3. confluencebuilder will then publish both files during the attachment uploading stage.\r\n\r\nUploading both files is less than ideal. We should maybe update confluencebuilder to detect when image content is \"replaced\" and remove it from asset tracking.\r\n\r\nThat being said, it does not explain why the figures are not working for the reported use case.\r\n\r\nSome questions about your case:\r\n- For the comment \"it is referring to a xxx.svg file\", I just want to confirm that you see DrawIO generate an `svg` file in the project's output directory (I assume this is the case).\r\n- When monitoring the output of a publish event, do you see attempts to publish both `.drawio` and `.svg`?\r\n\r\n ```\r\n publishing assets... [ 33%] box.drawio\r\n publishing assets... [ 67%] box.svg\r\n publishing assets... [100%] objects.inv\r\n ```\r\n\r\n Or just the `.drawio` file?\r\n\r\n ```\r\n publishing assets... [ 50%] box.drawio\r\n publishing assets... [100%] objects.inv\r\n ```\r\n- In the case where you may see the SVG being uploaded as well, do you see it uploaded on the page's attachment?\r\n- Do you see any different results when using `png` instead of `svg`?\r\n- Can you indicate which version of `sphinxcontrib-drawio` and the version of DrawIO installed?","issue_id":1715611692785,"origin_id":2094280312,"user_origin_id":1834509,"create_time":1714838895,"update_time":1714838895,"id":1716096125567,"updated_at":"2024-05-19T05:22:05.567000Z","created_at":"2024-05-19T05:22:05.567000Z"},{"_id":"66498c7d4a8e87d14e196b81","body":"The svg is generated locally: `.doctrees\/drawio\/3801198468179d90799067f4827c526171438f1f\/kmd_shim.svg`\r\n\r\nI don't see svg being uploaded, only drawio.\r\n\r\n```\r\n[100%] Generating Sphinx documentation\r\nRunning Sphinx v7.1.2\r\n\/home\/nanwa\/.local\/lib\/python3.8\/site-packages\/sphinxcontrib\/applehelp\/__init__.py:24: RemovedInSphinx80Warning: The alias 'sphinx.util.SkipProgressMessage' is deprecated, use 'sphinx.util.display.SkipProgressMessage' instead. Check CHANGES for Sphinx API modifications.\r\n from sphinx.util import SkipProgressMessage, progress_message\r\n\/home\/nanwa\/.local\/lib\/python3.8\/site-packages\/sphinxcontrib\/applehelp\/__init__.py:24: RemovedInSphinx80Warning: The alias 'sphinx.util.progress_message' is deprecated, use 'sphinx.http_date.epoch_to_rfc1123' instead. Check CHANGES for Sphinx API modifications.\r\n from sphinx.util import SkipProgressMessage, progress_message\r\n\/home\/nanwa\/.local\/lib\/python3.8\/site-packages\/sphinxcontrib\/htmlhelp\/__init__.py:26: RemovedInSphinx80Warning: The alias 'sphinx.util.progress_message' is deprecated, use 'sphinx.http_date.epoch_to_rfc1123' instead. Check CHANGES for Sphinx API modifications.\r\n from sphinx.util import progress_message\r\nmaking output directory... done\r\nrunning in non-headless mode, not starting Xvfb\r\nbuilding [mo]: targets for 0 po files that are out of date\r\nwriting output... \r\nbuilding [confluence]: targets for 2 source files that are out of date\r\nupdating environment: [new config] 2 added, 0 changed, 0 removed\r\nreading sources... [100%] kmd_shim\r\nlooking for now-outdated files... none found\r\npickling environment... done\r\nchecking consistency... done\r\npreparing documents... done\r\ncopying assets... done\r\n(drawio) 'diagram\/kmd_shim.drawio' -> 'drawio\/3801198468179d90799067f4827c526171438f1f\/kmd_shim.svg'\r\nwriting output... [100%] kmd_shim\r\n[dryrun] updating existing page Docs (2795954634) \r\n[dryrun] updating existing page Design (2795954636) \r\n\r\nbuilding intersphinx... done\r\n[dryrun] adding new attachment kmd_shim.drawio \r\n[dryrun] adding new attachment objects.inv \r\n\r\nPublish point: ...\r\nbuild succeeded.\r\n[100%] Built target docs-design-confluence\r\n```\r\nThe resulting confluence page only has the drawio as attachment.\r\n\r\nI tried jpg as well, no difference.\r\n\r\nsphinxcontrib-drawio: sphinxcontrib-drawio==0.0.17\r\n\r\n```\r\ndrawio --version\r\nChecking for beta autoupdate feature for deb\/rpm distributions\r\nFound package-type: deb\r\n24.2.5\r\n```\r\n\r\nThanks!\r\n","issue_id":1715611692785,"origin_id":2094319909,"user_origin_id":6154433,"create_time":1714843998,"update_time":1714843998,"id":1716096125572,"updated_at":"2024-05-19T05:22:05.572000Z","created_at":"2024-05-19T05:22:05.572000Z"},{"_id":"66498c7d4a8e87d14e196b82","body":"One thing I do that might be different from others is this:\r\nI have a source directory and a build directory. I don't want to generate any temporary files in my source folder. So my conf.py is copied to build folder at build time. Then I run sphinx-build from the builder folder and provide the location of the conf.py through the -c option. Here's my cmake snippet:\r\n\r\n```\r\nadd_custom_target(\r\n docs-design-confluence ALL\r\n COMMAND\r\n ${SPHINX_EXECUTABLE} -j auto -b confluence -c ${CMAKE_CURRENT_BINARY_DIR}\r\n ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR}\/design -vvv\r\n WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}\r\n COMMENT \"Generating Sphinx documentation\"\r\n)\r\n```\r\n\r\nI suspect the builder failed to find the svg asset because of this. I don't know if I am using this feature correctly.\r\n\r\n","issue_id":1715611692785,"origin_id":2094336308,"user_origin_id":6154433,"create_time":1714846486,"update_time":1714846543,"id":1716096125577,"updated_at":"2024-05-19T05:22:05.577000Z","created_at":"2024-05-19T05:22:05.577000Z"},{"_id":"66498c7d4a8e87d14e196b83","body":"I may be completely off but I noticed that we are always looking for an asset in the srcdir? If an assert is generated (like drawio), then it will be in the build dir, right? When looking for an assert, can we check both src dir and build dir?\r\n\r\n```\r\n if normalized_path.is_absolute():\r\n abs_path = normalized_path\r\n\r\n # some generated nodes will prefix the path of an asset with `\/`,\r\n # with the intent of that path being the root from the\r\n # configured source directory instead of an absolute path on the\r\n # system -- check the environment's source directory first before\r\n # checking the full system's path\r\n if normalized_path.name[0] == os.sep:\r\n new_path = Path(env.srcdir, normalized_path.name[1:])\r\n\r\n if new_path.is_file():\r\n abs_path = new_path\r\n else:\r\n abs_path = Path(env.srcdir, normalized_path)\r\n\r\n # extensions may dump a generated asset in the output directory; if\r\n # the absolute mapping to the source directory does not find the\r\n # asset, attempt to bind the path based on the output directory\r\n if not abs_path.is_file():\r\n abs_path = out_dir \/ normalized_path\r\n\r\n # if no asset can be found, ensure a `None` path is returned\r\n if not abs_path.is_file():\r\n abs_path = None\r\n```","issue_id":1715611692785,"origin_id":2094338274,"user_origin_id":6154433,"create_time":1714846916,"update_time":1714846916,"id":1716096125580,"updated_at":"2024-05-19T05:22:05.580000Z","created_at":"2024-05-19T05:22:05.580000Z"},{"_id":"66498c7d4a8e87d14e196b84","body":"You might be on the right path. I assume confluencebuilder cannot find the path of the SVG when using [`find_env_abspath`](https:\/\/github.com\/sphinx-contrib\/confluencebuilder\/blob\/d4151d3dbfd10e6e8e5c6f9e24b567e44c6c1c49\/sphinxcontrib\/confluencebuilder\/util.py#L203) which could prevent it from being registered internally for uploading. It looks like a user can only know [that this is happening](https:\/\/github.com\/sphinx-contrib\/confluencebuilder\/blob\/d4151d3dbfd10e6e8e5c6f9e24b567e44c6c1c49\/sphinxcontrib\/confluencebuilder\/assets.py#L344C20-L347) if verbose logging is enabled.\r\n\r\nDo you by chance see any `[confluence]` prefixed log that print `failed to find path:`? This would help confirm that the path is not being found internally.\r\n\r\n> If an assert is generated (like drawio), then it will be in the build dir, right?\r\n\r\nUnfortunately, it is not always consistent. It has been observed various extensions to unique things when referencing certain files -- a mix of absolute paths, relative to a source and relative to a build directory (on a side note, no idea why drawio is generating content in `.doctrees`). The confluencebuilder implementation tries to handle all these cases -- and there may be some little thing that is not being handled gracefully for this use case (assuming the issue is in this function).\r\n\r\nI do no think you are doing anything wrong from a quick inspection of the CMake snippet you have provided.","issue_id":1715611692785,"origin_id":2094347170,"user_origin_id":1834509,"create_time":1714849055,"update_time":1714849055,"id":1716096125583,"updated_at":"2024-05-19T05:22:05.582000Z","created_at":"2024-05-19T05:22:05.582000Z"},{"_id":"66498c7d4a8e87d14e196b85","body":"Hmm.. after switching to png\/jpg, the image will be uploaded to confluence. It seems to be specific to SVG.","issue_id":1715611692785,"origin_id":2094524662,"user_origin_id":6154433,"create_time":1714869565,"update_time":1714869565,"id":1716096125585,"updated_at":"2024-05-19T05:22:05.584000Z","created_at":"2024-05-19T05:22:05.584000Z"},{"_id":"66498c7d4a8e87d14e196b86","body":"If you do not mind trying this out, but do other SVG images (non-DrawIO generated) work as expected?","issue_id":1715611692785,"origin_id":2106363218,"user_origin_id":1834509,"create_time":1715544963,"update_time":1715544963,"id":1716096125588,"updated_at":"2024-05-19T05:22:05.587000Z","created_at":"2024-05-19T05:22:05.587000Z"},{"_id":"671cc5231e128a2ff1047dae","body":"There has not been activity on this thread for some time. Performing cleanup from this recent release, this thread will be marked as closed.\r\n\r\nThe assumption from this issue is that there is some sort of restriction on OPs Confluence instance which is restricting the publication of SVG attachments. The only recommendation is to try SVG publication without the use of DrawIO to confirm if this is the case. If it turns out that this is more of a DrawIO issue than an generic SVG issue, we can re-open this issue to try to determine the cause.","issue_id":1715611692785,"origin_id":2412227710,"user_origin_id":1834509,"create_time":1728938144,"update_time":1728938144,"id":1729938723961,"updated_at":"2024-10-26T10:32:03.960000Z","created_at":"2024-10-26T10:32:03.960000Z"}]

Hi, First, thanks for this great tool! I finally don't have to deal with confluence editor anymore. I have installed both drawio and confluencebuilder extensions: ``` extensions = [ 'sphinx_rtd_theme',...

wangnangg

How to achieve confluence's scroll version in sphinx-confluencebuilder

[{"_id":"66498c7e2cfd0d66a80fbd65","body":"@rakeshcbpur, I do not believe this extension can work with [Scroll Versions](https:\/\/marketplace.atlassian.com\/apps\/1210818\/). From what I can tell, it has [its own API](https:\/\/help.k15t.com\/scroll-versions\/4.6\/rest-api-resources) to manage resources which cannot be gracefully mapped to what this extension performs.","issue_id":1715611692818,"origin_id":2065606380,"user_origin_id":1834509,"create_time":1713491422,"update_time":1713491422,"id":1716096126614,"updated_at":"2024-05-19T05:22:06.614000Z","created_at":"2024-05-19T05:22:06.614000Z"}]

Hi Team, Apologies, if this is the bug/issues forum , I am still asking this question. We use below command to publish all the pages in current directory to configured...

rakeshcbpur

"Unsupported Confluence API call" when using versions >2.4.0

[]

We use Sphinx and this extension to convert a number of `.md` and `.rst` files to HTML and then upload the files to Confluence. We have used this across a...

massdosage

Mention sphinxnotes-strike in documentation

[]

Hi After experimenting a lot with your suggestions in the documentation about how to use strikethrough I was close to give up because nothing seemed to work. Then I came...

mkonig

Facing issue with Sphinx while connecting with SSO configured Confluence

[]

Hi Team, I hope your day is going well. We are facing issue with Sphinx confluence Builder. We are unable to establish connection between Confluence and Sphinx. We are using...

tapas-baranwal

An official way to add extra Confluence nodes

[]

Right now it looks like you can add extra nodes without modifying the extension, but it depends on private methods of the translator. Example for the `panel` macro: ```python from...

neongreen-sc

"Unsupported Confluence API call" when deploying updated documents with updated confluencebuilder

[]

I'm trying to update confluencesphinxbuilder and at the same time publish documentation with new page. I'm getting a ``` Unsupported Confluence API call An unsupported Confluence API call has been...

fizyk