Error Offchain Voting Anchor: JSON decode error

[dmitrystas]

OS Your OS: Ubuntu 24.04

Versions The db-sync version (eg cardano-db-sync --version):

cardano-db-sync 13.6.0.5 - linux-x86_64 - ghc-8.10 git revision cb61094c82254464fc9de777225e04d154d9c782

PostgreSQL version: 17

Build/Install Method The method you use to build or install cardano-db-sync: static binaries

Run method The method you used to run cardano-db-sync (eg Nix/Docker/systemd/none): systemd

Problem Report When parsing this Voting Anchor https://raw.githubusercontent.com/Tulibra1510/Drep-CaFiV2-metada/refs/heads/main/CaFi-V2.jsonld, DB-Sync shows the following error:

Error Offchain Voting Anchor: JSON decode error from when fetching metadata from Just https://raw.githubusercontent.com/Tulibra1510/Drep-CaFiV2-metada/refs/heads/main/CaFi-V2.jsonld resulted in : "Error in $.body.doNotList: expected String or Object with @value but encountered String, CIP-100:Error in $.body.references[0]: reference type should be one of [\"Other\",\"GovernanceMetadata\"] but it's Link"

The doNotList field is indeed a string rather than a boolean, so that is indeed a metadata error.

However, regarding CIP-100: Error in $.body.references[0] - it looks like an DB-Sync mistake, since @id is specified as CIP119:references, not CIP100:references https://github.com/cardano-foundation/CIPs/blob/master/CIP-0119/README.md#type-link

{
  "@context": {
    "CIP100": "https://github.com/cardano-foundation/CIPs/blob/master/CIP-0100/README.md#",
    "CIP119": "https://github.com/cardano-foundation/CIPs/blob/master/CIP-0119/README.md#",
    "hashAlgorithm": "CIP100:hashAlgorithm",
    "body": {
      "@id": "CIP119:body",
      "@context": {
        "references": {
          "@id": "CIP119:references",
          "@container": "@set",
          "@context": {
            "GovernanceMetadata": "CIP100:GovernanceMetadataReference",
            "Identity": "CIP119:IdentityReference",
            "Link": "CIP119:LinkReference",
            "Other": "CIP100:OtherReference",
            "label": "CIP100:reference-label",
            "uri": "CIP100:reference-uri",
            "referenceHash": {
              "@id": "CIP119:referenceHash",
              "@context": {
                "hashDigest": "CIP119:hashDigest",
                "hashAlgorithm": "CIP100:hashAlgorithm"
              }
            }
          }
        },
        "paymentAddress": "CIP119:paymentAddress",
        "givenName": "CIP119:givenName",
        "image": "CIP119:image",
        "objectives": "CIP119:objectives",
        "motivations": "CIP119:motivations",
        "qualifications": "CIP119:qualifications",
        "doNotList": "CIP119:doNotList"
      }
    },
    "authors": {
      "@id": "CIP100:authors",
      "@container": "@set",
      "@context": {
        "name": "http://xmlns.com/foaf/0.1/name",
        "witness": {
          "@id": "CIP100:witness",
          "@context": {
            "witnessAlgorithm": "CIP100:witnessAlgorithm",
            "publicKey": "CIP100:publicKey",
            "signature": "CIP100:signature"
          }
        }
      }
    }
  },
  "authors": [],
  "hashAlgorithm": "blake2b-256",
  "body": {
    "doNotList": "false",
    "givenName": "CaFi",
    "motivations": "With a strong belief in the potential of blockchain technology and the development of the Cardano ecosystem, the CaFi team wishes to contribute to innovation and progress through supporting community projects and initiatives. The main motivation of CaFi is to make the Catalyst more accessible to Vietnamese, while creating an environment where people can learn, share and develop their skills in writing proposals, evaluating and participating in the democratic voting process.\n\nThe CaFi team understands that the blockchain ecosystem in general and Cardano in particular are in the development stage and require community participation to achieve sustainability and success. By guiding the community to participate in Catalyst, CaFi wants to promote creativity, innovation and help valuable ideas receive the necessary support to grow.",
    "objectives": "1. Community Support: CaFi team focuses on helping the Vietnam community actively participate in the Catalyst through consulting and training activities on proposal writing, proposal evaluation and voting.\n\n2. Promoting innovation: We wishe to support and develop innovative projects in the Cardano ecosystem, especially those with practical value and sustainable development potential.\n\n3. Building the dRep role: We aim to become a responsible and reputable dRep in Cardano's decentralized governance model, contributing to the long-term development of the ecosystem.\n\n4. Community development: We want to build a dynamic, creative and self-directed community in the Blockchain ecosystem through participating in funding opportunities from Catalyst.\n\nWith the experience gained, CaFi team believes that working with the community will bring many values, while helping projects from Vietnam to have better access to resources from Catalyst to develop strongly on the Cardano platform.",
    "paymentAddress": "addr1q9mrk85srqxk88239chlxd73ct2s4nl20jzfwgu9k7tzxe60knquflpw8mtql842hyundh3vdy4e6zhw8f6z7afxjvys34qg2p",
    "qualifications": "1.  Do Viet Cuong\n● Master, high school teacher\n● Blockchain research since 2017\n● Cardano Vietnam community manager\n● Co-host of 9 Catalyst funded proposals\n● Admin of youtube channel, podcast, discussion group about Cardano and Catalyst project\n● Certified CBCA\n\n2. Tu Nguyen\n● Vietnam Drep Pioneer Program #1\n● FIMI Translator team\n● CBCA Alpha Program 2, Certificate ID: 64898843661a2dbd9402d27b\n● CBCA Course, Certificate ID: 5351588\n● Catalyst Funded Proposer\n● Admin of the Catalyst discussion group\n\n3.  Duc Nguyen\n● Cardano Blockchain Researcher from 2018 to present: Research and develop project related to Cardano Blockchain\n● Skill: Financial analysis; IT system management, database; Graphic design\n● Language: Vietnamese; English\n\n4.  Hoang Phuong Linh\n● Translation and research in Cardano Blockchain since 2017\n● Marketer, content creator, trainer, event organizer, community connector\n● Certified CBCA\n● Catalyst Funded Proposer\n● Languages: Vietnamese, English",
    "references": [
      {
        "@type": "Link",
        "label": "Catalyst Discussion Group",
        "uri": "https://t.me/Fimi_PA"
      },
      {
        "@type": "Link",
        "label": "Youtube Channel",
        "uri": "https://www.youtube.com/@taichinh-taman5516"
      },
      {
        "@type": "Link",
        "label": "General Cardano Discussion Group",
        "uri": "https://t.me/StakingADA"
      },
      {
        "@type": "Link",
        "label": "Cardano Podcast Channel",
        "uri": "https://t.me/fimi_podcast"
      },
      {
        "@type": "Link",
        "label": "IOG Reseacher Paper Library in Vietnamese Version",
        "uri": "https://cardano.vn/docs/Cardano360/TechDocs/intro/"
      },
      {
        "@type": "Identity",
        "label": "Do Viet Cuong",
        "uri": "https://t.me/dovietcuong"
      },
      {
        "@type": "Identity",
        "label": "Tu Nguyen",
        "uri": "https://t.me/Tulibra"
      },
      {
        "@type": "Identity",
        "label": "Duc Nguyen",
        "uri": "https://www.linkedin.com/in/minh-983aa4241/"
      },
      {
        "@type": "Identity",
        "label": "Hoang Phuong Linh",
        "uri": "https://www.linkedin.com/in/phuong-linh-hoang-a318bb138/"
      }
    ]
  }
}

[rdlrt]

Think it would be nice for dbsync to not worry about correctness/validity of anchor/voting and leave it to consumers. The specs will not be static, and adds unnecessary (possibly invalid) validation on dbsync end

[dmitrystas]

Yes, I fully agree. I would also prefer that the data be written to the db regardless of its 'validity'. We can still mark it as invalid (e.g., with an is_valid column in the DB or so) if DB-Sync believes this is the case, but we would still be able to ignore the warning if DB-Sync is mistaken

[kderme]

Agree, dbsync should still include the whole offchain data, if it manages to download them.

[kderme]

However, regarding CIP-100: Error in $.body.references[0] - it looks like an DB-Sync mistake, since @id is specified as CIP119:references, not CIP100:references

Actually looking at the specific error, I don't think DBSync has any mistakes. It tries to parse it as CIP-100, but this is only a fallback, since parsing as CIP-119 fails correctly. Both errors are printed.

[kderme]

We could insert the data if it has the correct hash, even if the json-ld doesn't parse (which is already too hard to do properly). In that case db-sync would stop retrying to fetch the data. is_valid would be false (no need to change that).

[Cmdv]

what should we store inside the table off_chain_vote_data when we hit this new case (valid hash/failed parsing)? With the field json do we:

1. make it nullable
2. populate it with the bytestring that we were unable to parse? (problem there is this field can be jsonb)

[kderme]

I guess in most cases it will be a valid json, but possible not a valid json-ld, because it's not very well defined. So in most cases we will be able to populate the json field. For the cases where it's not a valid json but still has valid hash, we can make it nullable.

[rdlrt]

Could potentially use a text field instead of json and dump raw content, dont worry about format/parsing it. Less CPU cycles spent on validating multiple formats across what could be a moving goal post?