jsonforms
jsonforms copied to clipboard
Ajv loading requires `unsafe-eval` policy for `script-src` Content-Security-Policy
Is your feature request related to a problem? Please describe.
For secure environments, the content security policy requires the unsafe-eval
policy directive for the script-src
policy. This requirement is primarily due to the use of Ajv's use of the Function
constructor per https://github.com/epoberezkin/ajv/issues/406
Describe the solution you'd like
The best resolution to this is to enable pre-compilation of the JSON schemas so that they don't have to be compiled at runtime. This can be accomplished using ajv-pack
https://github.com/epoberezkin/ajv-pack
Describe alternatives you've considered Alternatives or workarounds is to lazy load the Ajv validator so that it only loads when JsonForms components are rendered. This would allow compiling a version that disables jsonforms for environment with strict CSP's.
Describe for which setup you like to have the improvement Framework: Material-UI (or any of the others, this should apply)
Additional context
See screen shots below for some examples of this error

I like the look of ajv-pack, this is also related to a potential performance enhancement discussed here. Pre-complilation is great because avoids a penalty for doing this at runtime which is non-trivial for large schemas. https://spectrum.chat/jsonforms/general/performance-enhancements-for-ref-resolving~4fb7b288-ddad-43b4-9697-07e923a1e667
Although ajv-pack does not support recursive references which is a feature the eclipsesource team are currently looking to support. https://github.com/eclipsesource/jsonforms/issues/1476
I wonder if its possible to use ajv-pack already by passing an object to init which implements the methods of ajv that JSON Forms uses but uses ajv-pack instead Take a look a the init action https://github.com/eclipsesource/jsonforms/blob/master/packages/core/src/actions/index.ts#L83
and example of use https://github.com/eclipsesource/jsonforms-react-seed/blob/master/src/index.tsx
Something like
{
compile: () => prepackedAjvModule
}
An inital test suggests more compatibility work would be required to use ajv-pack.
I used the validate function created by dispatching init action
import validate from "../validate_schema.js"
store.dispatch(Actions.init(defaultData, schema, uischema, { compile: validate } as any))
First big difference is that I only see 1st error, not array of errors from the function created by ajv-pack.
Second difference which breaks the displaying of validation errors is that dataPath is missing
Ajv-pack validate
{
"keyword": "required",
"dataPath": "",
"schemaPath": "#/required",
"params": {
"missingProperty": "name"
},
"message": "should have required property 'name'"
}
AJV default validation (I removed schema and data properties)
{
"keyword": "required",
"dataPath": "name",
"schemaPath": "#/required",
"params": {
"missingProperty": "name"
},
"message": "is a required property"
}
Okay first issue is fixed by compling schema with --all-errors flag
ajv compile -s schema/Formulary/SchemaFlat.json -o validate_schema.js --all-errors
Thanks @Lily418 for the great responses.
I think it makes sense to support ajv-pack, either directly or at least as some sort of configuration option in the future. We should also check whether we can reduce the bundle size for people who want to use Ajv this way.
I checked the code for the usages of Ajv and found:
- Actions:
Ajv.compile(schema)
is called duringinit
,setSchema
andsetAjv
. All of these actions are only called from the outside so you know which schemas are coming in - Combinators:
Ajv.compile(schema)
is also called in the cases ofoneOf
,anyOf
andallOf
for each of the subschemas - Rules: To evaluate rules we create a new Ajv instance and call
validate
on it, which also callscompile
.
These three places need to be modified for a proper support of ajv-pack. Items 1 & 2 could be configured using @Lily418's suggestion with a custom Ajv "mock" (assuming a packed
validate behaves the same as the normal one). With item 3 however you will run into the error once you start using rules.
We would welcome a PR providing a configuration option for ajv-pack.
My issue was caused by not compiling schema with correct flags. I added error-data-path
ajv compile -s schema/Formulary/SchemaFlat.json -o validate_schema.js --all-errors --error-data-path=property
Here's a little script based off the one provided by ajv-pack to compile schema with default Json Forms Ajv options
https://gist.github.com/Lily418/0fb96b6dd4ae87885f0c788b070383ac
Thanks again @Lily418 for your great input.
@clayroach when using this compile script and creating your own "Ajv mock" as outlined above, you should be able to use JSON Forms (except for the Rule support) without hitting an error caused by the configured policies.
@Lily418 @sdirix Thanks for jumping on this. I will try out the suggestion above and open a PR on this for any changes.
This issue will be closed as it has been open for a long time with no activity. Feel free to reopen this issue if needed.
@sdirix This is still a real issue. If you could re-open that would be amazing. I'm using JSON forms for a project right now and just ran into this. I see ways it can be resolved and I think I may be able to create a PR at some point in the next few weeks or so.
I found that using the method suggested by @Lily418 above to generate the ajv-pack
'ed module, setting up my own instance of AJV and replacing the ajv.compile
function with something like ajv.compile = () => myPackedModule
gets me 90% of the way there. There are just some issues with the way AJV is used in the @jsonforms/core
package. One main issue that means this method doesn't work 100% is that createAjv uses addMetaSchema
which triggers an unsafe eval, and this function is called on file load by /packages/core/src/util/runtime.ts
. If this instead was able to use the injected ajv instance instead of one it creates on file load I could hack it all together in a way that works for me.
Ideally, it would be amazing if we could inject the ajv-pack
'd module as is so it could be used instead of ajv altogether. I'm contemplating cloning the core package into my project's repository and making the tweaks I need to get this working, then building a PR from there, but I need to talk to my team first.
EDIT: After digging in a bit further, it looks like the main issue with createAjv
is that it adds a custom meta-schema for draft 4 that only adds 2 lines:
"id":{
"type":"string",
"format":"uri" // This one
},
"$schema":{
"type":"string",
"format":"uri" // and this one
},
If JSON Forms could function with the default draft-4 schema, and add it with getSchema('http://json-schema.org/draft-04/schema#')
instead of addMetaSchema
, there would be no unsafe evals outside of ajv.compile
calls (as far as I can tell). I guess my question is: how important is it to JSON Forms that the draft-4 schema forces "id" and "$schema" to be URI formatted strings? If I understand how this works correctly, this just allows AJV to output a more useful error when they aren't URIs.
Hi @DBosley, thanks for your message. I agree we should keep this issue open.
I'm absolutely fine with changing runtime.ts
as I don't like that we construct an own Ajv instance here anyway. I would prefer reusing the Ajv instance from the state/context. The easiest way to achieve this would be to just add Ajv as a parameter to all exported functions. Maybe we can think of a slightly nicer pattern.
Regarding the API support: Maybe it would be easiest to just provide a wrapper util for packed modules which mocks / delegates all Ajv functions which JSON Forms uses. The usage would be straightforward and we don't need to distinguish between "normal" Ajv and "packed" Ajv in the core module, reducing complexity. Do you see any problems with that or is there something we would miss out upon with this approach?
I think passing the ajv instance into runtime.ts
would get me past the blocking issue with my project. I was trying to think of a better pattern myself, but was at a loss as these utility functions don't have a good way to access the reducer state.
If all usages of AJV could use the one provided via config, there would at least be a workaround that would allow users to get around unsafe-eval, even if it's a little hacky. Some things to note: ajv-pack
'ed exports only 1 thing, a validate function. The schema is available via a property attached directly to the exported function. It apparently also doesn't support recursive references in the schema, but I feel like there could be an easy workaround to this by just having multiple instances of the JSON Form that you then compose together into a single data object on submit. Here's a basic usage example if I were to pack it into a file called "packedValidate":
import { validate } from "./packedValidate";
const schema = validate.schema;
...
validate(formData); // this is functionally the same as Ajv.compile(schema)(formData)
I like your idea for the mock/delegate approach. I think it could get the job done. Is this something you want to take the initiative on, or would it help if I took a stab at it in a PR or two?
I think 2 PRs make sense for this:
- Change any ajv usage to use the one supplied, even if it's a bit ugly to pass it in to get it down to the utility functions
- Create the wrapper utility to allow for injection of a pre-compiled schema without having to Frankenstein it together yourself before injecting AJV into the project.
I can most likely get the first PR done pretty quickly. (I may need a little help if this is something that affects tests or needs a new test, but otherwise I'm fairly familiar with the code at this point. I've had to develop a full set of renderers and cells for my project). The 2nd PR should be simple as well, but I'd love some guidance on how you'd like it done, any tests it may need, and documentation improvements that would be required.
This does seem like it may be related to #1557
This topic is currently not on our priority list so we wouldn't tackle it in the near future ourselves (except when a client requests it via the support). However we'll definitively take a look at a contribution from the community (i.e. you :wink: ). Of course if there are any questions we'll gladly help.
- I saw that you already opened the first PR! Thanks for the initiative, very appreciated!
- Let's see what we have to do to get the "Ajv packed" use case fully working. I guess we not only need to pack the original schema but also all rule schemas from the ui schema and then dispatch between them? Regarding the implementation I would add the wrapper util to the core package, however I would prefer not including the
ajv-pack
dependency, so it should also be handed in. To be merged it should also contain some unit tests to make sure it behaves as expected. It would also be nice to add an example to the example app which uses a packed Ajv (but not required). As a cherry on top you could also document it on the jsonforms website.
I would prefer not including the ajv-pack dependency
good news! ajv-pack
is purely needed to build the module. It's not needed as a dependency after compiling the schema
I don't know how in-depth I will be able to go for making the ajv-packed improvements myself. I'm happy to assist, but I need to shift back to my own project once I have a way to move forward (hopefully this first PR will do the trick). I'll document anything I need to do to get this working that may be useful for building the ajv-packed injector.
That's fine of course ;) Yes it would be very helpful to know what you had to do to get it working.
I was able to use the latest beta version with my precompiled AJV schema validator and I didn't get an unsafe-eval error with CSP enabled!
Here's how I got it to work:
- add
ajv
andajv-pack
to my project - create a folder called
./schemas
and place all schemas to precompile in there - create a file sibling to the schemas folder with the following node script:
const Ajv = require('ajv');
const pack = require('ajv-pack');
const fs = require('fs');
const path = require('path');
const process = require('process');
const ajv = new Ajv({
sourceCode: true,
schemaId: 'auto',
allErrors: true,
jsonPointers: true,
errorDataPath: 'property',
verbose: true
});
const schemaPath = path.join(__dirname, './schemas');
const generatedFileText =
'// THIS FILE IS GENERATED. DO NOT EDIT\n\n';
let indexFile = '';
fs.readdir(schemaPath, (err, files) => {
if (err) {
console.error('Could not list the directory.', err);
process.exit(1);
}
files.forEach((file) => {
const filePath = path.join(schemaPath, file);
const schema = require(filePath);
const validate = ajv.compile(schema);
const moduleCode = pack(ajv, validate);
const validatorName = file.split('.')[0];
const outFile = validatorName + '.js';
fs.writeFileSync(
path.join(__dirname, './validators', outFile),
generatedFileText + moduleCode
);
indexFile += `export { default as ${validatorName} } from './validators/${validatorName}'\n`;
});
fs.writeFileSync(path.join(__dirname, './index.js'), generatedFileText + indexFile);
});
- run this file with node. It will create a new folder with a validator file named the same as the schema and it will export all created validators from a single index file
- import the validator you want to use with JSON forms and replace AJV's compile method response to return it:
import Ajv from 'ajv';
import { MySchema as validate } from './path/to/generated/index';
const ajv = new Ajv({
sourceCode: true,
schemaId: 'auto',
allErrors: true,
jsonPointers: true,
errorDataPath: 'property',
verbose: true
});
ajv.compile = () => validate;
- Pass ajv and all other necessary props to the JsonForms component (note: The schema is part of the precompiled validate function, so you can access it via
validate.schema
):<JsonForms ajv={ajv} schema={validate.schema} {..otherJsonFormsProps} />
@DBosley Thanks for the extensive write-up, this is very helpful! Did you also manage to get the schema-based rule support working?
I don't actually use that in my implementation. I don't see that being a problem, though. The unsafe-eval issues stem from compiling JSON Schemas and meta schemas with AJV. As far as I can tell, nothing in the UI Schema is compiled. I do use some UI Schema settings to organize layout, so I know the reference selectors still work fine. I may be using some of the ui schema rules soon, but I'm not sure yet. I'll let you know if I have time to test that out
Small update: I got this working with multiple schemas that have $ref
pointers. When building the generated output, you need to use ajv.addSchema
for each schema. Then when using ajv-pack, you do something like this for each one you added:
const pack = require('ajv-pack');
const { join, dirname } = require('path');
const fs = require('fs');
const { mkdirSync, writeFileSync } = fs;
// id is /path/to/my/schema.json
// I also use this as the $id string in the schema
const validate = ajv.getSchema(id);
const moduleCode = pack(ajv, validate);
const validatorName = id.split('.')[0];
const outFile = join(__dirname, '../packed', validatorName + '.js');
mkdirSync(dirname(outFile), { recursive: true });
writeFileSync(outFile, generatedFileText + moduleCode, {});
There's an issue with just replacing ajv.compile
with the root schema's validate, however. You need to instead replace it with a function that will find the correct validator based on a prop in the schema is passes in. I use $id:
_ajv.compile = (schema) => {
if (schema['$id']) {
const validators = [
rootValidate,
ref1Validate,
ref2Validate
];
for (const validator of validators) {
if (validator.schema['$id'] === schema['$id']) {
return validator;
}
}
}
return rootValidate;
};
Also, you need to create resolvers for each schema to pass into the JSON Forms component:
const resolve = {
ref1: {
order: 1,
canRead: (file) => {
return file.url.indexOf('name-of-my-ref1-file') !== -1;
},
read: () => {
return JSON.stringify(ref1Validate.schema);
}
},
ref2: {
order: 2,
canRead: (file) => {
return file.url.indexOf('name-of-my-ref2-file') !== -1;
},
read: () => {
return JSON.stringify(ref2Validate.schema);
}
}
};
Then this is just handed in like it is in the documentation here: https://jsonforms.io/docs/ref-resolving
ajv-pack is now deprecated and its functionality has been integrated/moved to ajv itself and ajv-cli. The output of ajv-cli seems to differ slightly from ajv-pack (at least I wasn't able to directly use @DBosley's workaround, https://github.com/eclipsesource/jsonforms/issues/1498#issuecomment-679257857)
Are there any plans to move forward with this issue?
Here's some output produced by ajv-cli on an example schema.
Click to expand
'use strict' module.exports = validate20 module.exports.default = validate20 const schema22 = { type: 'object', properties: { name: { type: 'string', minLength: 3, description: 'Please enter your name', }, vegetarian: { type: 'boolean' }, birthDate: { type: 'string', format: 'date' }, nationality: { type: 'string', enum: ['DE', 'IT', 'JP', 'US', 'RU', 'Other'], }, personalData: { type: 'object', properties: { age: { type: 'integer', description: 'Please enter your age.' }, height: { type: 'number' }, drivingSkill: { type: 'number', maximum: 10, minimum: 1, default: 7 }, }, required: ['age', 'height'], }, occupation: { type: 'string' }, postalCode: { type: 'string', maxLength: 5 }, }, required: ['occupation', 'nationality'], } const func8 = require('ajv/dist/runtime/ucs2length').default const func0 = require('ajv/dist/runtime/equal').default const formats0 = require('ajv-formats/dist/formats').fullFormats.date function validate20( data, { instancePath = '', parentData, parentDataProperty, rootData = data } = {} ) { let vErrors = null let errors = 0 if (errors === 0) { if (data && typeof data == 'object' && !Array.isArray(data)) { let missing0 if ( (data.occupation === undefined && (missing0 = 'occupation')) || (data.nationality === undefined && (missing0 = 'nationality')) ) { validate20.errors = [ { instancePath, schemaPath: '#/required', keyword: 'required', params: { missingProperty: missing0 }, message: "must have required property '" + missing0 + "'", }, ] return false } else { if (data.name !== undefined) { let data0 = data.name const _errs1 = errors if (errors === _errs1) { if (typeof data0 === 'string') { if (func8(data0) 10 || isNaN(data7)) { validate20.errors = [ { instancePath: instancePath + '/personalData/drivingSkill', schemaPath: '#/properties/personalData/properties/drivingSkill/maximum', keyword: 'maximum', params: { comparison: '=', limit: 1, }, message: 'must be >= 1', }, ] return false } } } else { validate20.errors = [ { instancePath: instancePath + '/personalData/drivingSkill', schemaPath: '#/properties/personalData/properties/drivingSkill/type', keyword: 'type', params: { type: 'number' }, message: 'must be number', }, ] return false } } var valid1 = _errs15 === errors } else { var valid1 = true } } } } } else { validate20.errors = [ { instancePath: instancePath + '/personalData', schemaPath: '#/properties/personalData/type', keyword: 'type', params: { type: 'object' }, message: 'must be object', }, ] return false } } var valid0 = _errs9 === errors } else { var valid0 = true } if (valid0) { if (data.occupation !== undefined) { const _errs17 = errors if (typeof data.occupation !== 'string') { validate20.errors = [ { instancePath: instancePath + '/occupation', schemaPath: '#/properties/occupation/type', keyword: 'type', params: { type: 'string' }, message: 'must be string', }, ] return false } var valid0 = _errs17 === errors } else { var valid0 = true } if (valid0) { if (data.postalCode !== undefined) { let data9 = data.postalCode const _errs19 = errors if (errors === _errs19) { if (typeof data9 === 'string') { if (func8(data9) > 5) { validate20.errors = [ { instancePath: instancePath + '/postalCode', schemaPath: '#/properties/postalCode/maxLength', keyword: 'maxLength', params: { limit: 5 }, message: 'must NOT have more than 5 characters', }, ] return false } } else { validate20.errors = [ { instancePath: instancePath + '/postalCode', schemaPath: '#/properties/postalCode/type', keyword: 'type', params: { type: 'string' }, message: 'must be string', }, ] return false } } var valid0 = _errs19 === errors } else { var valid0 = true } } } } } } } } } else { validate20.errors = [ { instancePath, schemaPath: '#/type', keyword: 'type', params: { type: 'object' }, message: 'must be object', }, ] return false } } validate20.errors = vErrors return errors === 0 }
Could someone explain why rules does not working with standalone(CLI) compiled JSON schema ? Or maybe someone have some workaround now?
I haven't been able to get @DBosley's approach to work in my project using Ajv standalone validators. The issue for me is that the compiled validator code contains require
statements for runtime Ajv
modules (such as ucs2length
, which is included when using a length constraint such as minLength
or maxLength
). @joschaschmiedt's recent comment also reflects this. Because I am using Vite, my project uses native browser ES modules and import
statements.
I think the compiled validators should be dependency-free and contain all required code bundled in, or at least there should be an option to bundle everything in.
I came up with a hacky workaround: find-and-replace on the compiled code to replace require
statements with a blob satisfying the requirement. For example:
const replacements = [
{
search: /require\("ajv\/dist\/runtime\/ucs2length"\)\.default/g,
replace: 'function(c){let t=c.length,f=0,n=0,d;for(;n<t;)f++,(d=c.charCodeAt(n++))>=55296&&d<=56319&&n<t&&(64512&(d=c.charCodeAt(n)))==56320&&n++;return f}',
},
// ...
];
for (const { search, replace } of replacements) {
validatorCode = validatorCode.replace(search, replace);
}
if (validatorCode.indexOf('require(') !== -1) {
throw new Error('Unreplaced require() statement in compiled validator code');
}
With this, the compiled validators run in the browser. I am still working on jerry-rigging the ajv
instance so that JsonForms
doesn't call the built-in ajv.compile()
.
A lot of work currently goes into using this library and Ajv within the context of a secure CSP.
I was able to get it working with the hack in my previous comment and:
export default function Form({
data,
enabled,
onChange,
schema,
validator,
uischema,
validationMode,
}) {
const ajvRef = useRef(createAjv());
ajvRef.current.compile = () => validator;
return (
<JsonForms
ajv={ajvRef.current}
cells={cells}
data={data}
readonly={!enabled}
onChange={onChange}
renderers={renderers}
schema={schema}
uischema={uischema}
validationMode={validationMode}
/>
);
}
Could someone explain why rules does not working with standalone(CLI) compiled JSON schema ? Or maybe someone have some workaround now?
It uses a different API. ajv.validate
for rules.
https://github.com/eclipsesource/jsonforms/blob/1e44159b4e9e7aaf4f5324b6f67636e5e35bd777/packages/core/src/util/runtime.ts#L82
If we implement a similar "mock" for ajv.validate
, it should work with rules too, correct?
createAjv
is no longer used to create another Ajv
instance inside runtime.ts
. I don't know when this changed but I see ajv
instance being passed in as an argument.
I have an idea after reading the source.
<JsonForms
ajv={undefined}
cells={cells}
data={data}
readonly={!enabled}
onChange={onChange}
renderers={renderers}
schema={schema}
uischema={uischema}
validationMode={"NoValidation"}
additionalErrors={(data) => validator(data)} // validator is compiled externally.
/>
You can see this working at https://jsonforms.io/docs/validation#external-validation-errors
I was able to get it working with the hack in my previous comment and:
export default function Form({ data, enabled, onChange, schema, validator, uischema, validationMode, }) { const ajvRef = useRef(createAjv()); ajvRef.current.compile = () => validator; return ( <JsonForms ajv={ajvRef.current} cells={cells} data={data} readonly={!enabled} onChange={onChange} renderers={renderers} schema={schema} uischema={uischema} validationMode={validationMode} /> ); }
Nothing here gets my rules working. Has anyone else had success with that?
Nothing here gets my rules working. Has anyone else had success with that?
Hi @cbprice, You can use the same workaround as a previous poster, but make it a bit smarter: Instead of blindly returning the validator
for the overall JSON Schema, you need to check what schema
you are given (i.e. overall JSON Schema or a rule schema) and then return the corresponding validator
. Then your rules should work.
Hey @sdirix, first of all, thank you for working on this! I think this is a very valuable library, especially due to it not being tied to a single frontend framework's ecosystem. IMHO it'd be even more flexible (and more future-proof as restrictive CSP settings become more common) if validation was not tied to Ajv.
Now, (unless I'm missing something here) the outlined workarounds solve the issue for mostly static JSON schemas. However, when introducing more dynamic schemas (think building a form builder with dynamic schemas etc.) or when layering schemas, things become difficult. It's a bit unfortunate you'd have to either expose your users to XSS risks by going with unsafe-eval
or build some elaborate precompilation logic into the backend.
I'm aware of #1557 but I think that's more about making Ajv updates more flexible. Any chance we're going to see support for something like hyperjump/json-schema or similar packages not requiring unsafe-eval
? I'm not that familiar with the JS JSON schema package landscape but I see mostly two options here if you'd decide for decoupling from Ajv:
- The 'proper' way: Introduce some package-independent intermediate layer for translating validation / errors
- The 'hacky' way: implement the subset of the Ajv interface used by jsonforms for another library (if this is done outside jsonforms this may be very brittle)
What are your thoughts on this?
@sdirix - how we can avoid same error while evaluating rules to control layout rendering e.g show/hide or enabled/disabled element dynamically based on payload value? we have overcome unsafe-eval issue to validate the payload againts schema by writing our custom validator but how we can handle the same issue with rules?
We could not go with complied schema approach as we more than 100 layout schema and json schema which we are fetching dynamically and rendering it on our UI, those schemas are frequently changed using dashboard so in this case how we can resolve this issue.?