redwood
redwood copied to clipboard
GraphQL and Scaffolding does not support Prisma Bytes data type
My database is postgres server.
I made model Asset following the tutorial in schema.prisma
// Asset Model
// to hold binary
model Asset {
id Int @id @default(autoincrement())
title String
comments String
createdAt DateTime @default(now())
blob Bytes
coordinates Json
}
I went to generate my scaffold Yarn rw g scaffold asset
then my error
PS E:\HalmetsRedWood\Hamlet> Yarn rw g scaffold asset
× Generating scaffold files...
→ Cannot read properties of undefined (reading 'replace')
Adding layout import...
Adding set import...
Adding scaffold routes...
Adding scaffold asset imports...
Generating types ...
Cannot read properties of undefined (reading 'rep
The only related issue i could find is https://github.com/redwoodjs/redwood/issues/3214 this is kinda too deep for me Some sort of mapping issue https://github.com/redwoodjs/redwood/pull/3218
So reading this issue more its a scalar https://www.graphql-scalars.dev/docs/scalars/byte
'bytes' I have no idea if this works
test('scenarioFieldValue returns a float for Float types', () => {
const field = { type: 'Float' }
const value = service.scenarioFieldValue(field)
expect(value).toEqual(parseFloat(value))
expect(typeof value).toBe('number')
})
- Installed the framework for the project
- we need to access a local database like postgres to reproduce bug
- changed the redwood-project .env with
DATABASE_URL="postgresql://postgres:<PASSWORD>@localhost:5432/testDB?connection_limit=1"
psql -U postgres
createdb testDB
- Change the 'schema.prisma' with
// How to connect prisma postgres server
datasource db {
provider = "postgresql"
url = env("DATABASE_URL")
}
- Updated
schema.primsa
with blob type from Prisma to recreate the steps above.
// Asset Model
// to hold binary
model Asset {
id Int @id @default(autoincrement())
title String
comments String
createdAt DateTime @default(now())
blob Bytes
coordinates Json
}
we apply yarn rw prisma migrate dev
- ok now for scaffold
yarn rw g scaffold asset
we get error
× Generating scaffold files...
→ Cannot read properties of undefined (reading 'replace')
Adding layout import...
Adding set import...
Adding scaffold routes...
Adding scaffold asset imports...
Generating types ...
Cannot read properties of undefined (reading 'replace')
Now to fix it
Ok looking at old issue we need to parse the 'value' in a scenario.test.js?
Well good news is that Nexus-Prisma has already implemented a bytes scalar support https://github.com/prisma/nexus-prisma/issues/68 from the GraphQL Scalars Library
See code here https://github.com/prisma/nexus-prisma/pull/75/files
Is there a way to parse Binary?
nodeJS Buffers and blob.arrayBuffer() https://nodejs.org/api/buffer.html
Ok so 'Byte' Type Scalar for Graphql
The Byte scalar type represents byte value as specified by NodeJS Buffer type
So a byte value is basically a integer between 0 and 255
If value is a number, it will be coerced to a valid byte value, an integer between 0 and 255.
What the hell is a buffer in JS?
Node provides Buffer class which provides instances to store raw data similar to an array of integers but corresponds to a raw memory allocation outside the V8 heap.
link
So Graphql scenarioFieldValue will generate a test of a buffer by passing in random bytes to type check
Graphql inside 'The Byte
scalar type represents byte value as a Buffer', Byte type expects a buffer in its code
https://github.com/Urigo/graphql-scalars/blob/728827f133d6448d6e032231df6db06af8e92414/src/scalars/Byte.ts
So we wrote our own form of Byte using the GraphQL Scalar library in scenario.test.js
test('scenarioFieldValue returns a byte for Byte types NodeJS Buffer type', () => {
const field = { type: 'Byte' }
const value = service.scenarioFieldValue(field)
// GraphQLByte.parseValue(encodedValue) examples https://github.com/Urigo/graphql-scalars/blob/aeb8aebc9738d533e0cac4cf6ddbad7d1555814e/tests/Byte.test.ts
expect(value).toEqual(GraphQLByte.parseValue(value))
expect(typeof value).toBe('Buffer')
})
We use expect(typeof value).tBe('Buffer') because if you look in line 11 of https://github.com/Urigo/graphql-scalars/blob/aeb8aebc9738d533e0cac4cf6ddbad7d1555814e/src/scalars/Byte.ts
type BufferJson = { type: 'Buffer'; data: number[] };
case 'Byte': {
newValue = concatenate(Uint8Array,newValue, Uint8Array.of(1, 1))
break
}
Where we left over
Recommend Longterm solutions Graphql and Redwood cannot either cover all custom Scalar types like bytes. However project maintained by The Guild has good Library called GraphQL Scalars that covers near most use cases
We import this new library called GraphQL Scalars https://github.com/urigo/graphql-scalars
we install this yarn add graphql-scalars
and import the Type Scalars inside of GraphSQL Scalars Library because they have validation code already and covers go use cases
https://github.com/Urigo/graphql-scalars/blob/728827f133d6448d6e032231df6db06af8e92414/src/scalars/Byte.ts
For this problem with Bytes they have
const ByteMock = () => new Uint8Array([1988, 1981, 1965, 1963, 1959, 1955]);
Oh it comes with Apollo I am dumb https://www.apollographql.com/docs/apollo-server/schema/custom-scalars/
Ah
const ByteMock = () => new Uint8Array([1988, 1981, 1965, 1963, 1959, 1955]);
This was the tricky part that I hard problems with when trying to implement Bytes in the scenarios and tests.
Maybe that will work.
Ah
const ByteMock = () => new Uint8Array([1988, 1981, 1965, 1963, 1959, 1955]);
This was the tricky part that I hard problems with when trying to implement Bytes in the scenarios and tests.
Maybe that will work.
Well am stuck basically its not passing a "Buffer" type I am too inexperience can we PR this we need collaboration
Ok When I test typeof BufferObject it appears as a 'object' because in JS everything is Object but how do you check that a object is buffer, even though we used GraphQLByte.parseValue(value) and jest is expecting a Buffer
we have to check with a different test
code
const randByte = Buffer.from([1, 2, 3, 4]);
const testBye = new Uint8Array([1988, 1981, 1965, 1963, 1959, 1955]);
const buf1 = Buffer.alloc(10);
const buf = Buffer.from('hello world', 'utf8');
console.log(typeof randByte)
console.log(randByte)
console.log(typeof testBye)
console.log(testBye)
console.log(typeof buf1)
console.log(buf1)
console.log(typeof buf)
console.log(buf)
// how to test is is this a buffer?
console.log("randByte buffer?", Buffer.isBuffer(randByte))
console.log("testBye buffer?", Buffer.isBuffer(testBye))
console.log("buf1 buffer?", Buffer.isBuffer(buf1))
console.log("buf buffer?",Buffer.isBuffer(buf))
output
node .\main.js
object
<Buffer 01 02 03 04>
object
Uint8Array(6) [ 196, 189, 173, 171, 167, 163 ]
object
<Buffer 00 00 00 00 00 00 00 00 00 00>
object
<Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64>
true
false
true
true
we edit scenario.test.js
Ok its passible it worked @dthyresson however new bug and I cannot run yarn e2e I am on Windows
PS E:\RedWoodIssues\redwoodblog> yarn rw g scaffold asset
√ Generating scaffold files...
√ Successfully wrote file `./web\src\components\Asset\EditAsse
tCell\EditAssetCell.js`
√ Successfully wrote file `./web\src\components\Asset\Asset\As
set.js`
√ Successfully wrote file `./web\src\components\Asset\AssetCel
l\AssetCell.js`
√ Successfully wrote file `./web\src\components\Asset\AssetFor
m\AssetForm.js`
√ Successfully wrote file `./web\src\components\Asset\Assets\A
√ Generating scaffold files...
√ Successfully wrote file `./web\src\components\Asset\EditAsse
tCell\EditAssetCell.js`
√ Successfully wrote file `./web\src\components\Asset\Asset\As
set.js`
√ Successfully wrote file `./web\src\components\Asset\AssetCel
l\AssetCell.js`
√ Successfully wrote file `./web\src\components\Asset\AssetFor
m\AssetForm.js`
√ Successfully wrote file `./web\src\components\Asset\Assets\A
ssets.js`
√ Successfully wrote file `./web\src\components\Asset\AssetsCe
ll\AssetsCell.js`
√ Successfully wrote file `./web\src\components\Asset\NewAsset
\NewAsset.js`
√ Successfully wrote file `./api\src\graphql\assets.sdl.js`
√ Successfully wrote file `./api\src\services\assets\assets.js
`
√ Successfully wrote file `./api\src\services\assets\assets.sc
enarios.js`
√ Successfully wrote file `./api\src\services\assets\assets.te
st.js`
√ Successfully wrote file `./web\src\scaffold.css`
√ Successfully wrote file `./web\src\layouts\AssetsLayout\Asse
tsLayout.js`
√ Successfully wrote file `./web\src\pages\Asset\EditAssetPage
\EditAssetPage.js`
√ Successfully wrote file `./web\src\pages\Asset\AssetPage\Ass
etPage.js`
√ Successfully wrote file `./web\src\pages\Asset\AssetsPage\As
setsPage.js`
√ Successfully wrote file `./web\src\pages\Asset\NewAssetPage\
NewAssetPage.js`
√ Adding layout import...
√ Adding set import...
√ Adding scaffold routes...
√ Adding scaffold asset imports...
| Generating types ...
Schema loading failed. Unknown type: "Bytes".
√ Generating scaffold files...
√ Successfully wrote file `./web\src\components\Asset\EditAsse
tCell\EditAssetCell.js`
√ Successfully wrote file `./web\src\components\Asset\Asset\As
set.js`
√ Successfully wrote file `./web\src\components\Asset\AssetCel
l\AssetCell.js`
√ Successfully wrote file `./web\src\components\Asset\AssetFor
m\AssetForm.js`
√ Successfully wrote file `./web\src\components\Asset\Assets\A
√ Generating scaffold files...
√ Successfully wrote file `./web\src\components\Asset\EditAsse
tCell\EditAssetCell.js`
√ Successfully wrote file `./web\src\components\Asset\Asset\As
set.js`
√ Successfully wrote file `./web\src\components\Asset\AssetCel
l\AssetCell.js`
√ Successfully wrote file `./web\src\components\Asset\AssetFor
m\AssetForm.js`
√ Successfully wrote file `./web\src\components\Asset\Assets\A
√ Generating scaffold files...
√ Successfully wrote file `./web\src\components\Asset\EditAsse
tCell\EditAssetCell.js`
√ Successfully wrote file `./web\src\components\Asset\Asset\As
set.js`
√ Successfully wrote file `./web\src\components\Asset\AssetCel
l\AssetCell.js`
√ Successfully wrote file `./web\src\components\Asset\AssetFor
m\AssetForm.js`
√ Successfully wrote file `./web\src\components\Asset\Assets\A
ssets.js`
√ Successfully wrote file `./web\src\components\Asset\AssetsCe
ll\AssetsCell.js`
√ Successfully wrote file `./web\src\components\Asset\NewAsset
\NewAsset.js`
√ Successfully wrote file `./api\src\graphql\assets.sdl.js`
√ Successfully wrote file `./api\src\services\assets\assets.js
`
√ Successfully wrote file `./api\src\services\assets\assets.sc
enarios.js`
√ Successfully wrote file `./api\src\services\assets\assets.te
st.js`
√ Successfully wrote file `./web\src\scaffold.css`
√ Successfully wrote file `./web\src\layouts\AssetsLayout\Asse
tsLayout.js`
√ Successfully wrote file `./web\src\pages\Asset\EditAssetPage
\EditAssetPage.js`
√ Successfully wrote file `./web\src\pages\Asset\AssetPage\Ass
etPage.js`
√ Successfully wrote file `./web\src\pages\Asset\AssetsPage\As
setsPage.js`
√ Successfully wrote file `./web\src\pages\Asset\NewAssetPage\
NewAssetPage.js`
√ Adding layout import...
√ Adding set import...
√ Adding scaffold routes...
√ Adding scaffold asset imports...
\ Generating types ...
Error: Could not generate GraphQL type definitions (web)
AggregateError: GraphQL Document Validation failed with 9 errors;
Error 0: GraphQLDocumentError: Unknown type "Int".
at E:/RedWoodIssues/redwoodblog/web/src/components/Asset/Asset/Asset.js:1:35
Error 1: GraphQLDocumentError: Unknown type "Int".
at E:/RedWoodIssues/redwoodblog/web/src/components/Asset/AssetCell/AssetCell.js:1:26
Error 2: GraphQLDocumentError: Cannot query field "asset" on type "Query".
at E:/RedWoodIssues/redwoodblog/web/src/components/Asset/AssetCell/AssetCell.js:2:3
Error 3: GraphQLDocumentError: Cannot query field "assets" on type
"Query".
at E:/RedWoodIssues/redwoodblog/web/src/components/Asset/AssetsCell/AssetsCell.js:2:3
Error 4: GraphQLDocumentError: Unknown type "Int".
at E:/RedWoodIssues/redwoodblog/web/src/components/Asset/EditAssetCell/EditAssetCell.js:1:26
Error 5: GraphQLDocumentError: Cannot query field "asset" on type "Query".
at E:/RedWoodIssues/redwoodblog/web/src/components/Asset/EditAs √ Generating scaffold files...
√ Successfully wrote file `./web\src\components\Asset\EditAsse
tCell\EditAssetCell.js`
√ Successfully wrote file `./web\src\components\Asset\Asset\As
set.js`
√ Successfully wrote file `./web\src\components\Asset\AssetCel
l\AssetCell.js`
√ Successfully wrote file `./web\src\components\Asset\AssetFor
m\AssetForm.js`
√ Successfully wrote file `./web\src\components\Asset\Assets\A
ssets.js`
√ Successfully wrote file `./web\src\components\Asset\AssetsCe
ll\AssetsCell.js`
√ Successfully wrote file `./web\src\components\Asset\NewAsset
\NewAsset.js`
√ Successfully wrote file `./api\src\graphql\assets.sdl.js`
√ Successfully wrote file `./api\src\services\assets\assets.js
`
√ Successfully wrote file `./api\src\services\assets\assets.sc
enarios.js`
√ Successfully wrote file `./api\src\services\assets\assets.te
st.js`
√ Successfully wrote file `./web\src\scaffold.css`
√ Successfully wrote file `./web\src\layouts\AssetsLayout\Asse
tsLayout.js`
√ Successfully wrote file `./web\src\pages\Asset\EditAssetPage
\EditAssetPage.js`
√ Successfully wrote file `./web\src\pages\Asset\AssetPage\Ass
etPage.js`
√ Successfully wrote file `./web\src\pages\Asset\AssetsPage\As
setsPage.js`
√ Successfully wrote file `./web\src\pages\Asset\NewAssetPage\
NewAssetPage.js`
√ Adding layout import...
√ Adding set import...
√ Adding scaffold routes...
√ Adding scaffold asset imports...
√ Generating types ...
GraphQL Server crashbed its not passing Bytes with GraphQL check
Hi @Gresliebear and thanks for the progress? I was planning to work on this issue the week and maybe we can collaborate using the work you've done as I bet it is close.
Could you submit a Draft PR with your work in progress and I can review and see what else might be needed?
Thanks!
Hi @Gresliebear and thanks for the progress? I was planning to work on this issue the week and maybe we can collaborate using the work you've done as I bet it is close.
Could you submit a Draft PR with your work in progress and I can review and see what else might be needed?
Thanks!
@dthyresson yes give me 30 minutes because I don't know how to PR quickly this is my 2nd time so my apologizes,
Where I left off I made changes and then yarn cross-env RWFW_PATH=E:\RedWoodIssues\redwood rwfw project:sync
Then did yarn rw dev
database postgres primsa has no issues migrating the Byte to postgres
Recommended feature scaffold should autogenerated a drag and dropbox or click here to upload files button
Hi @Gresliebear and thanks for the progress? I was planning to work on this issue the week and maybe we can collaborate using the work you've done as I bet it is close.
Could you submit a Draft PR with your work in progress and I can review and see what else might be needed?
Thanks!
@dthyresson Alright the PR is made https://github.com/redwoodjs/redwood/pull/5874
Re-assigned based on review of PR #5874
Completing this https://github.com/redwoodjs/redwood/pull/5874 PR can be a label:hackoberfest item
Hi, I found the same error (not sure if this is related) when trying to generate SDL for UserCredentials when using AuthDB with WebAuthn on Windows (I cannot test this on other systems).
There should be either note NOT to use WebAuthn with Windows or it needs a fix as it's preventing the feature from working. Unless it is possible to change type from Bytes to something else.
model UserCredential {
id String @id
user User @relation(fields: [userId], references: [id])
userId String
publicKey Bytes
transports String?
counter BigInt
}
I also posted this comment over here, but you shouldn't create an SDL file for UserCredential
, just add to the Prisma model and then migrate the database. UserCredential
is referenced internally by the WebAuthn code, but it should not be exposed via your GraphQL API!
We can't create a scaffold for a table with a relation with User if we don't create the sdl for UserCredential, what I understand with scaffold command is that we have to create all the relations, User have a relation with UserCredential, my table Product have a relation with User ...
here the message when I want to go to /products page
![image](https://user-images.githubusercontent.com/10152935/195292675-79e361f4-d6cc-4060-ab73-0da2960393bd.png)
web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/graphql to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
You shouldn't create a scaffold for UserCredential
either, you can't really manually create records for that table, so there's no need to have a UI for it.
But you're saying that once you create the UserCredential
model and migrate the database, that you can't create a scaffold for User
and it blows up?
That error sounds like the API server stopped running, can you check back further in the console output and see what the error was that crashed it?
As I'm thinking about this more, is it that the User
SDL gets created fine, but when you try to query it you get an error about UserCredential
type not existing? Check your api/src/graphql/user.sdl.js
file, and see if there's a reference to UserCredential
in there. If so, remove that line—you shouldn't be able to query on that model at all. If that's the fix please let me know and I'll add that to the documentation!
Unfortunately there's no easy way to exclude that relationship from the SDL generating logic at this point. We're talking about creating an allow/block list of attribute names, and could include UserCredential
in that list.
I can confirm I've wiped out the db, migrate without UserCredential and now I can scaffold my Product table who have a relation with User who don't have a relation with UserCredential
As I'm thinking about this more, is it that the
User
SDL gets created fine, but when you try to query it you get an error aboutUserCredential
type not existing? Check yourapi/src/graphql/user.sdl.js
file, and see if there's a reference toUserCredential
in there. If so, remove that line—you shouldn't be able to query on that model at all. If that's the fix please let me know and I'll add that to the documentation!Unfortunately there's no easy way to exclude that relationship from the SDL generating logic at this point. We're talking about creating an allow/block list of attribute names, and could include
UserCredential
in that list.
I'll try this
Looks like we give a warning message now, telling you that there's a relationship defined in an SDL but no other SDL file that actually defines the type:
So if you're seeing an error like that, you should just be able to remove any reference to UserCredential
and userCredentialId
from your User
SDL file. Watch the console and see if the API server restarts itself, otherwise you might just need to kill everything and restart yarn rw dev
.
web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/auth?method=getToken to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/graphql to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/auth?method=getToken to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/graphql to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/graphql to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/graphql to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
The errors when I do nothing just create sdl for relations with my Product table
![image](https://user-images.githubusercontent.com/10152935/195406506-b2677561-6848-4cbd-b86b-8aca7452ea3e.png)
api | 19:07:57 🚨 graphql-server Error building context. Error: Exception in getAuthenticationContext: The `Authorization` header is not valid.
The error when I remove the relation with UserCredential in api/src/graphql/user.sdl.ts
If I comment also webAuthnChallenge
all is fine
Huh...not sure why you'd need to comment that one out, although it's a good practice to do so (that's another field that shouldn't be accessed via GraphQL). You should really comment out hashedPassword
and salt
as well (those are two fields we're going to add to the allow/block list by default).
Sometimes it takes a couple tries for the API server to realize something has changed and reload itself, maybe commenting out webAuthnChallenge
was just a coincidence! You could try commenting it back IN and see if it still works.
No it's a wrong refresh, it' work witout commenting ! sorry but yeah we need to comment credentials
to create scaffold that have a relation with User
Okay sounds good, I'm updating the documentation now!
PR merged here: https://github.com/redwoodjs/redwood/pull/6672