GraphQL and Scaffolding does not support Prisma Bytes data type

[Gresliebear]

My database is postgres server.

I made model Asset following the tutorial in schema.prisma

// Asset Model
// to hold binary
model Asset { 
  id Int @id @default(autoincrement())
  title String
  comments String
  createdAt DateTime @default(now())
  blob Bytes
  coordinates Json
}

I went to generate my scaffold Yarn rw g scaffold asset

then my error

PS E:\HalmetsRedWood\Hamlet> Yarn rw g scaffold asset  
  × Generating scaffold files...
    → Cannot read properties of undefined (reading 'replace')
    Adding layout import...
    Adding set import...
    Adding scaffold routes...
    Adding scaffold asset imports...
    Generating types ...
Cannot read properties of undefined (reading 'rep

[Gresliebear]

The only related issue i could find is https://github.com/redwoodjs/redwood/issues/3214 this is kinda too deep for me Some sort of mapping issue https://github.com/redwoodjs/redwood/pull/3218

[Gresliebear]

So reading this issue more its a scalar https://www.graphql-scalars.dev/docs/scalars/byte

'bytes' I have no idea if this works

  test('scenarioFieldValue returns a float for Float types', () => {
    const field = { type: 'Float' }
    const value = service.scenarioFieldValue(field)

    expect(value).toEqual(parseFloat(value))
    expect(typeof value).toBe('number')
  })

[Gresliebear]

1. Installed the framework for the project
2. we need to access a local database like postgres to reproduce bug
3. changed the redwood-project .env with

DATABASE_URL="postgresql://postgres:<PASSWORD>@localhost:5432/testDB?connection_limit=1"

psql -U postgres

createdb testDB

4. Change the 'schema.prisma' with

// How to connect prisma postgres server
datasource db {
provider = "postgresql"
  url = env("DATABASE_URL")
}

5. Updated schema.primsa with blob type from Prisma to recreate the steps above.

// Asset Model
// to hold binary
model Asset { 
  id Int @id @default(autoincrement())
  title String
  comments String
  createdAt DateTime @default(now())
  blob Bytes
  coordinates Json
}

we apply yarn rw prisma migrate dev

6. ok now for scaffold yarn rw g scaffold asset we get error

  × Generating scaffold files...
    → Cannot read properties of undefined (reading 'replace')
    Adding layout import...
    Adding set import...
    Adding scaffold routes...
    Adding scaffold asset imports...
    Generating types ...
Cannot read properties of undefined (reading 'replace')

[Gresliebear]

Now to fix it

Ok looking at old issue we need to parse the 'value' in a scenario.test.js?

Well good news is that Nexus-Prisma has already implemented a bytes scalar support https://github.com/prisma/nexus-prisma/issues/68 from the GraphQL Scalars Library

See code here https://github.com/prisma/nexus-prisma/pull/75/files

Is there a way to parse Binary?

nodeJS Buffers and blob.arrayBuffer() https://nodejs.org/api/buffer.html

Ok so 'Byte' Type Scalar for Graphql

The Byte scalar type represents byte value as specified by NodeJS Buffer type

So a byte value is basically a integer between 0 and 255

If value is a number, it will be coerced to a valid byte value, an integer between 0 and 255.

What the hell is a buffer in JS?

Node provides Buffer class which provides instances to store raw data similar to an array of integers but corresponds to a raw memory allocation outside the V8 heap. link

So Graphql scenarioFieldValue will generate a test of a buffer by passing in random bytes to type check

Graphql inside 'The Byte scalar type represents byte value as a Buffer', Byte type expects a buffer in its code https://github.com/Urigo/graphql-scalars/blob/728827f133d6448d6e032231df6db06af8e92414/src/scalars/Byte.ts

So we wrote our own form of Byte using the GraphQL Scalar library in scenario.test.js

  test('scenarioFieldValue returns a byte for Byte types NodeJS Buffer type', () => {
    const field = { type: 'Byte' }
    const value = service.scenarioFieldValue(field)
    // GraphQLByte.parseValue(encodedValue) examples https://github.com/Urigo/graphql-scalars/blob/aeb8aebc9738d533e0cac4cf6ddbad7d1555814e/tests/Byte.test.ts
    expect(value).toEqual(GraphQLByte.parseValue(value))
    expect(typeof value).toBe('Buffer')
  })

We use expect(typeof value).tBe('Buffer') because if you look in line 11 of https://github.com/Urigo/graphql-scalars/blob/aeb8aebc9738d533e0cac4cf6ddbad7d1555814e/src/scalars/Byte.ts

type BufferJson = { type: 'Buffer'; data: number[] };

    case 'Byte': {
        newValue = concatenate(Uint8Array,newValue, Uint8Array.of(1, 1))
        break
      }

Where we left over

[Gresliebear]

Recommend Longterm solutions Graphql and Redwood cannot either cover all custom Scalar types like bytes. However project maintained by The Guild has good Library called GraphQL Scalars that covers near most use cases

We import this new library called GraphQL Scalars https://github.com/urigo/graphql-scalars

we install this yarn add graphql-scalars

and import the Type Scalars inside of GraphSQL Scalars Library because they have validation code already and covers go use cases

https://github.com/Urigo/graphql-scalars/blob/728827f133d6448d6e032231df6db06af8e92414/src/scalars/Byte.ts

For this problem with Bytes they have

const ByteMock = () => new Uint8Array([1988, 1981, 1965, 1963, 1959, 1955]);

Oh it comes with Apollo I am dumb https://www.apollographql.com/docs/apollo-server/schema/custom-scalars/

[dthyresson]

Ah

const ByteMock = () => new Uint8Array([1988, 1981, 1965, 1963, 1959, 1955]);

This was the tricky part that I hard problems with when trying to implement Bytes in the scenarios and tests.

Maybe that will work.

[Gresliebear]

Ah

const ByteMock = () => new Uint8Array([1988, 1981, 1965, 1963, 1959, 1955]);

This was the tricky part that I hard problems with when trying to implement Bytes in the scenarios and tests.

Maybe that will work.

Well am stuck basically its not passing a "Buffer" type I am too inexperience can we PR this we need collaboration

[Gresliebear]

Ok When I test typeof BufferObject it appears as a 'object' because in JS everything is Object but how do you check that a object is buffer, even though we used GraphQLByte.parseValue(value) and jest is expecting a Buffer

we have to check with a different test

code

const randByte = Buffer.from([1, 2, 3, 4]);
const testBye = new Uint8Array([1988, 1981, 1965, 1963, 1959, 1955]);
const buf1 = Buffer.alloc(10);
const buf = Buffer.from('hello world', 'utf8');

console.log(typeof randByte)
console.log(randByte)
console.log(typeof testBye)
console.log(testBye)
console.log(typeof buf1)
console.log(buf1)
console.log(typeof buf)
console.log(buf)

// how to test is is this a buffer?

console.log("randByte buffer?", Buffer.isBuffer(randByte))
console.log("testBye buffer?", Buffer.isBuffer(testBye))
console.log("buf1 buffer?", Buffer.isBuffer(buf1))
console.log("buf buffer?",Buffer.isBuffer(buf))

output

                     node .\main.js

object

<Buffer 01 02 03 04>
object
Uint8Array(6) [ 196, 189, 173, 171, 167, 163 ]
object
<Buffer 00 00 00 00 00 00 00 00 00 00>
object
<Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64>
true
false
true
true

we edit scenario.test.js

[Gresliebear]

Ok its passible it worked @dthyresson however new bug and I cannot run yarn e2e I am on Windows

PS E:\RedWoodIssues\redwoodblog> yarn rw g scaffold asset
  √ Generating scaffold files...
    √ Successfully wrote file `./web\src\components\Asset\EditAsse 
tCell\EditAssetCell.js`
    √ Successfully wrote file `./web\src\components\Asset\Asset\As 
set.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetCel 
l\AssetCell.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetFor 
m\AssetForm.js`
    √ Successfully wrote file `./web\src\components\Asset\Assets\A 
  √ Generating scaffold files...
    √ Successfully wrote file `./web\src\components\Asset\EditAsse 
tCell\EditAssetCell.js`
    √ Successfully wrote file `./web\src\components\Asset\Asset\As 
set.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetCel 
l\AssetCell.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetFor 
m\AssetForm.js`
    √ Successfully wrote file `./web\src\components\Asset\Assets\A 
ssets.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetsCe 
ll\AssetsCell.js`
    √ Successfully wrote file `./web\src\components\Asset\NewAsset 
\NewAsset.js`
    √ Successfully wrote file `./api\src\graphql\assets.sdl.js`    
    √ Successfully wrote file `./api\src\services\assets\assets.js 
`
    √ Successfully wrote file `./api\src\services\assets\assets.sc 
enarios.js`
    √ Successfully wrote file `./api\src\services\assets\assets.te 
st.js`
    √ Successfully wrote file `./web\src\scaffold.css`
    √ Successfully wrote file `./web\src\layouts\AssetsLayout\Asse 
tsLayout.js`
    √ Successfully wrote file `./web\src\pages\Asset\EditAssetPage 
\EditAssetPage.js`
    √ Successfully wrote file `./web\src\pages\Asset\AssetPage\Ass 
etPage.js`
    √ Successfully wrote file `./web\src\pages\Asset\AssetsPage\As 
setsPage.js`
    √ Successfully wrote file `./web\src\pages\Asset\NewAssetPage\ 
NewAssetPage.js`
  √ Adding layout import...
  √ Adding set import...
  √ Adding scaffold routes...
  √ Adding scaffold asset imports...
  | Generating types ...

Schema loading failed. Unknown type: "Bytes".

  √ Generating scaffold files...
    √ Successfully wrote file `./web\src\components\Asset\EditAsse 
tCell\EditAssetCell.js`
    √ Successfully wrote file `./web\src\components\Asset\Asset\As 
set.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetCel 
l\AssetCell.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetFor 
m\AssetForm.js`
    √ Successfully wrote file `./web\src\components\Asset\Assets\A 
  √ Generating scaffold files...
    √ Successfully wrote file `./web\src\components\Asset\EditAsse 
tCell\EditAssetCell.js`
    √ Successfully wrote file `./web\src\components\Asset\Asset\As 
set.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetCel 
l\AssetCell.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetFor 
m\AssetForm.js`
    √ Successfully wrote file `./web\src\components\Asset\Assets\A 
  √ Generating scaffold files...
    √ Successfully wrote file `./web\src\components\Asset\EditAsse 
tCell\EditAssetCell.js`
    √ Successfully wrote file `./web\src\components\Asset\Asset\As 
set.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetCel 
l\AssetCell.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetFor 
m\AssetForm.js`
    √ Successfully wrote file `./web\src\components\Asset\Assets\A 
ssets.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetsCe 
ll\AssetsCell.js`
    √ Successfully wrote file `./web\src\components\Asset\NewAsset 
\NewAsset.js`
    √ Successfully wrote file `./api\src\graphql\assets.sdl.js`    
    √ Successfully wrote file `./api\src\services\assets\assets.js 
`
    √ Successfully wrote file `./api\src\services\assets\assets.sc 
enarios.js`
    √ Successfully wrote file `./api\src\services\assets\assets.te 
st.js`
    √ Successfully wrote file `./web\src\scaffold.css`
    √ Successfully wrote file `./web\src\layouts\AssetsLayout\Asse 
tsLayout.js`
    √ Successfully wrote file `./web\src\pages\Asset\EditAssetPage 
\EditAssetPage.js`
    √ Successfully wrote file `./web\src\pages\Asset\AssetPage\Ass 
etPage.js`
    √ Successfully wrote file `./web\src\pages\Asset\AssetsPage\As 
setsPage.js`
    √ Successfully wrote file `./web\src\pages\Asset\NewAssetPage\ 
NewAssetPage.js`
  √ Adding layout import...
  √ Adding set import...
  √ Adding scaffold routes...
  √ Adding scaffold asset imports...
  \ Generating types ...

Error: Could not generate GraphQL type definitions (web)
AggregateError: GraphQL Document Validation failed with 9 errors;  
  Error 0: GraphQLDocumentError: Unknown type "Int".
    at E:/RedWoodIssues/redwoodblog/web/src/components/Asset/Asset/Asset.js:1:35

Error 1: GraphQLDocumentError: Unknown type "Int".
    at E:/RedWoodIssues/redwoodblog/web/src/components/Asset/AssetCell/AssetCell.js:1:26

Error 2: GraphQLDocumentError: Cannot query field "asset" on type "Query".
    at E:/RedWoodIssues/redwoodblog/web/src/components/Asset/AssetCell/AssetCell.js:2:3

Error 3: GraphQLDocumentError: Cannot query field "assets" on type 
"Query".
    at E:/RedWoodIssues/redwoodblog/web/src/components/Asset/AssetsCell/AssetsCell.js:2:3

Error 4: GraphQLDocumentError: Unknown type "Int".
    at E:/RedWoodIssues/redwoodblog/web/src/components/Asset/EditAssetCell/EditAssetCell.js:1:26

Error 5: GraphQLDocumentError: Cannot query field "asset" on type "Query".
    at E:/RedWoodIssues/redwoodblog/web/src/components/Asset/EditAs  √ Generating scaffold files...
    √ Successfully wrote file `./web\src\components\Asset\EditAsse 
tCell\EditAssetCell.js`
    √ Successfully wrote file `./web\src\components\Asset\Asset\As 
set.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetCel 
l\AssetCell.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetFor 
m\AssetForm.js`
    √ Successfully wrote file `./web\src\components\Asset\Assets\A 
ssets.js`
    √ Successfully wrote file `./web\src\components\Asset\AssetsCe 
ll\AssetsCell.js`
    √ Successfully wrote file `./web\src\components\Asset\NewAsset 
\NewAsset.js`
    √ Successfully wrote file `./api\src\graphql\assets.sdl.js`    
    √ Successfully wrote file `./api\src\services\assets\assets.js 
`
    √ Successfully wrote file `./api\src\services\assets\assets.sc 
enarios.js`
    √ Successfully wrote file `./api\src\services\assets\assets.te 
st.js`
    √ Successfully wrote file `./web\src\scaffold.css`
    √ Successfully wrote file `./web\src\layouts\AssetsLayout\Asse 
tsLayout.js`
    √ Successfully wrote file `./web\src\pages\Asset\EditAssetPage 
\EditAssetPage.js`
    √ Successfully wrote file `./web\src\pages\Asset\AssetPage\Ass 
etPage.js`
    √ Successfully wrote file `./web\src\pages\Asset\AssetsPage\As 
setsPage.js`
    √ Successfully wrote file `./web\src\pages\Asset\NewAssetPage\ 
NewAssetPage.js`
  √ Adding layout import...
  √ Adding set import...
  √ Adding scaffold routes...
  √ Adding scaffold asset imports...
  √ Generating types ...

[Gresliebear]

GraphQL Server crashbed its not passing Bytes with GraphQL check

[dthyresson]

Hi @Gresliebear and thanks for the progress? I was planning to work on this issue the week and maybe we can collaborate using the work you've done as I bet it is close.

Could you submit a Draft PR with your work in progress and I can review and see what else might be needed?

Thanks!

[Gresliebear]

Hi @Gresliebear and thanks for the progress? I was planning to work on this issue the week and maybe we can collaborate using the work you've done as I bet it is close.

Could you submit a Draft PR with your work in progress and I can review and see what else might be needed?

Thanks!

@dthyresson yes give me 30 minutes because I don't know how to PR quickly this is my 2nd time so my apologizes,

Where I left off I made changes and then yarn cross-env RWFW_PATH=E:\RedWoodIssues\redwood rwfw project:sync

Then did yarn rw dev database postgres primsa has no issues migrating the Byte to postgres

Recommended feature scaffold should autogenerated a drag and dropbox or click here to upload files button

[Gresliebear]

Hi @Gresliebear and thanks for the progress? I was planning to work on this issue the week and maybe we can collaborate using the work you've done as I bet it is close.

Could you submit a Draft PR with your work in progress and I can review and see what else might be needed?

Thanks!

@dthyresson Alright the PR is made https://github.com/redwoodjs/redwood/pull/5874

[dthyresson]

Re-assigned based on review of PR #5874

[dthyresson]

Completing this https://github.com/redwoodjs/redwood/pull/5874 PR can be a label:hackoberfest item

[mc-spieczynski]

Hi, I found the same error (not sure if this is related) when trying to generate SDL for UserCredentials when using AuthDB with WebAuthn on Windows (I cannot test this on other systems).

There should be either note NOT to use WebAuthn with Windows or it needs a fix as it's preventing the feature from working. Unless it is possible to change type from Bytes to something else.

model UserCredential {
  id         String  @id
  user       User    @relation(fields: [userId], references: [id])
  userId     String
  publicKey  Bytes
  transports String?
  counter    BigInt
}

[cannikin]

I also posted this comment over here, but you shouldn't create an SDL file for UserCredential, just add to the Prisma model and then migrate the database. UserCredential is referenced internally by the WebAuthn code, but it should not be exposed via your GraphQL API!

[snettah]

We can't create a scaffold for a table with a relation with User if we don't create the sdl for UserCredential, what I understand with scaffold command is that we have to create all the relations, User have a relation with UserCredential, my table Product have a relation with User ...

here the message when I want to go to /products page

web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/graphql to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)

[cannikin]

You shouldn't create a scaffold for UserCredential either, you can't really manually create records for that table, so there's no need to have a UI for it.

But you're saying that once you create the UserCredential model and migrate the database, that you can't create a scaffold for User and it blows up?

That error sounds like the API server stopped running, can you check back further in the console output and see what the error was that crashed it?

[cannikin]

As I'm thinking about this more, is it that the User SDL gets created fine, but when you try to query it you get an error about UserCredential type not existing? Check your api/src/graphql/user.sdl.js file, and see if there's a reference to UserCredential in there. If so, remove that line—you shouldn't be able to query on that model at all. If that's the fix please let me know and I'll add that to the documentation!

Unfortunately there's no easy way to exclude that relationship from the SDL generating logic at this point. We're talking about creating an allow/block list of attribute names, and could include UserCredential in that list.

[snettah]

I can confirm I've wiped out the db, migrate without UserCredential and now I can scaffold my Product table who have a relation with User who don't have a relation with UserCredential

[snettah]

As I'm thinking about this more, is it that the User SDL gets created fine, but when you try to query it you get an error about UserCredential type not existing? Check your api/src/graphql/user.sdl.js file, and see if there's a reference to UserCredential in there. If so, remove that line—you shouldn't be able to query on that model at all. If that's the fix please let me know and I'll add that to the documentation!

Unfortunately there's no easy way to exclude that relationship from the SDL generating logic at this point. We're talking about creating an allow/block list of attribute names, and could include UserCredential in that list.

I'll try this

[cannikin]

Looks like we give a warning message now, telling you that there's a relationship defined in an SDL but no other SDL file that actually defines the type:

So if you're seeing an error like that, you should just be able to remove any reference to UserCredential and userCredentialId from your User SDL file. Watch the console and see if the API server restarts itself, otherwise you might just need to kill everything and restart yarn rw dev.

[snettah]

web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/auth?method=getToken to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/graphql to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/auth?method=getToken to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/graphql to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/graphql to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
web | <e> [webpack-dev-server] [HPM] Error occurred while proxying request localhost:8910/graphql to http://[::1]:8911/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)

The errors when I do nothing just create sdl for relations with my Product table

api | 19:07:57 🚨 graphql-server Error building context. Error: Exception in getAuthenticationContext: The `Authorization` header is not valid.

The error when I remove the relation with UserCredential in api/src/graphql/user.sdl.ts

[snettah]

If I comment also webAuthnChallenge all is fine

[cannikin]

Huh...not sure why you'd need to comment that one out, although it's a good practice to do so (that's another field that shouldn't be accessed via GraphQL). You should really comment out hashedPassword and salt as well (those are two fields we're going to add to the allow/block list by default).

Sometimes it takes a couple tries for the API server to realize something has changed and reload itself, maybe commenting out webAuthnChallenge was just a coincidence! You could try commenting it back IN and see if it still works.

[snettah]

No it's a wrong refresh, it' work witout commenting ! sorry but yeah we need to comment credentials to create scaffold that have a relation with User

[cannikin]

Okay sounds good, I'm updating the documentation now!

[cannikin]

PR merged here: https://github.com/redwoodjs/redwood/pull/6672