Describe the bug

As shown in https://github.com/agoric-labs/hardened-create-cosmos-app, when I try using create-cosmos-app to make a cosmos app (which runs on Next.js), I cannot successfully invoke lockdown and run the app.

Error Details:

See the README of the example repo for error details.

Steps to reproduce

Clone https://github.com/agoric-labs/hardened-create-cosmos-app
yarn && yarn dev to install deps and run locally
Load the url outputted by yarn dev
Observe the error

Expected behavior

lockdown can be invoked in the top-level of the application without error.

Platform environment

What OS are you using? MacOS 13.3.1 (22E261)
What version of Node.js? v18.17.1
Is there anything special/unusual about your platform? No
What version of Endo are you using? "ses": "^1.1.0"

Additional context

Create-cosmos-app is a tool for scaffolding cosmos apps with Next.js. It would be beneficial to be able to use SES and Endo APIs in apps built using this tool. The error is perhaps more general to Next.js as a whole, but this tool is of particular interest.

Screenshots

See https://github.com/agoric-labs/hardened-create-cosmos-app

Feb 05 '24 06:02 samsiegart

This smells like a CommonJS interop issue and I would like to summon @naugtur, if we can borrow his eyes.

Feb 09 '24 20:02 kriskowal

@samsiegart Capturing the error in the description will help us diagnose.

Feb 09 '24 20:02 kriskowal

@samsiegart Capturing the error in the description will help us diagnose.

There's a screenshot and two stack traces in the example repo's README. I've updated the description here to point it out more explicitly

Feb 09 '24 20:02 samsiegart

Oh, it's private, I should make it public

Edit: done

Feb 09 '24 20:02 samsiegart

@kriskowal , I've assigned it to the two of us.

Feb 09 '24 22:02 erights

@naugtur , you too ;)

Feb 09 '24 22:02 erights

Another data point- shouldn't be a surprise given I'm using the same create-cosmos-app, just with a slightly different template: I tried to initialize an NFT example with create-cosmos-app --name hardened-nft-example --example nft and I got the same error as @samsiegart, see https://github.com/agoric-labs/hardened-nft-example?tab=readme-ov-file#bug-report

Feb 12 '24 18:02 LuqiPan

Reproduced locally! Thanks for the clear and simple instructions.

Feb 13 '24 03:02 erights

Yeah, @naugtur @kriskowal , it is all about the cjs loader. I will definitely need your help looking at this. It was easy to reproduce locally, so could you have a look? If the bug is interesting, perhaps we could discuss it at the upcoming endo meeting?

Feb 13 '24 04:02 erights

Actually, this seems kinda urgent for us. Could we meet tomorrow after you've taken a first look? I am free tomorrow anytime after 2pm Pacific.

Feb 13 '24 04:02 erights

@tgrecojs you mentioned on the call you might be familiar with this issue

Feb 14 '24 19:02 kumavis

I just took a moment to spin up an example repo which integrates @endo/init into a next.js application. This code can be found here - https://github.com/tgrecojs/nextjs-with-endo-init

Pasting the contents of the README file from that repository below. @samsiegart let me know if this helps as all. One more thing I want to note - I think your repo may be using a version of past next.js. FWIW I have wired up past versions to work with endo so I don't believe that is the issue here but I'm not entirely sure.

README.md

Next.js X @endo/init

View this incredibly rich and interactive demo here - https://hardened-nextjs-repo.vercel.app/.

This repository demonstrates the integration of Endo and Knit within a Next.js application.

Below you can view contents of the project's _document.js file.

import { Html, Head, Main, NextScript } from "next/document";
import Script from "next/script";

export default function Document() {
  return (
    <Html>
      <Head />
      <body>
        <Main />
        <NextScript />
        <Script
          async
          type="module"
          src="https://esm.sh/@endo/[email protected]"
          strategy="beforeInteractive"
        />
      </body>
    </Html>
  );
}

The key component within this file is the Next.js specific <Script /> element as it is responsible for loading @endo/init. When used outside of Next.js' _document.js file, this component has been the source of issues when loading the page such as the one seen here (although it is not necessarily responsible for this issue).

Checking for tamper-proof globals.

Below is a screenshot of interaction from https://hardened-nextjs-repo.vercel.app

ss_02142024_000144

Feb 14 '24 20:02 tgrecojs

I reproduced the error and the error starts on the server

$ next dev
  ▲ Next.js 13.5.6
  - Local:        http://localhost:3000

 ✓ Ready in 1626ms
 ✓ Compiled / in 1937ms (2269 modules)
 ⨯ [ReferenceError: __webpack_require__ is not defined
  at Object.eval (webpack-internal:///./node_modules/next/dist/build/webpack/loaders/next-route-loader/index.js?kind=PAGES&page=%2F&preferredRegion=&absolutePagePath=.%2Fpages%2Findex.tsx&absoluteAppPath=private-next-pages%2F_app&absoluteDocumentPath=private-next-pages%2F_document&middlewareConfigBase64=e30%3D!:1:1)
  at eval (eval at makeEvaluate (file:///lab/samples/hardened-create-cosmos-app/node_modules/ses/src/make-evaluate.js:92:27), <anonymous>:12:22)
  at ./node_modules/next/dist/build/webpack/loaders/next-route-loader/index.js?kind=PAGES&page=%2F&preferredRegion=&absolutePagePath=.%2Fpages%2Findex.tsx&absoluteAppPath=private-next-pages%2F_app&absoluteDocumentPath=private-next-pages%2F_document&middlewareConfigBase64=e30%3D! (/lab/samples/hardened-create-cosmos-app/.next/server/pages/index.js:22:1)
  at __webpack_require__ (/lab/samples/hardened-create-cosmos-app/.next/server/webpack-runtime.js:33:42)
  at __webpack_exec__ (/lab/samples/hardened-create-cosmos-app/.next/server/pages/index.js:361:39)
  at /lab/samples/hardened-create-cosmos-app/.next/server/pages/index.js:362:114
  at __webpack_require__.X (/lab/samples/hardened-create-cosmos-app/.next/server/webpack-runtime.js:185:21)
  at /lab/samples/hardened-create-cosmos-app/.next/server/pages/index.js:362:47
  at Object.<anonymous> (/lab/samples/hardened-create-cosmos-app/.next/server/pages/index.js:365:3)] {
  page: '/'
}
 ✓ Compiled /_error in 228ms (2271 modules)
[ReferenceError: exports is not defined
  at Object.eval (webpack-internal:///./node_modules/next/dist/pages/_document.js:2:23)
  at eval (eval at makeEvaluate (file:///lab/samples/hardened-create-cosmos-app/node_modules/ses/src/make-evaluate.js:92:27), <anonymous>:12:22)
  at ./node_modules/next/dist/pages/_document.js (/lab/samples/hardened-create-cosmos-app/.next/server/vendor-chunks/next.js:30:1)
  at __webpack_require__ (/lab/samples/hardened-create-cosmos-app/.next/server/webpack-runtime.js:33:42)
  at __webpack_exec__ (/lab/samples/hardened-create-cosmos-app/.next/server/pages/_document.js:52:39)
  at /lab/samples/hardened-create-cosmos-app/.next/server/pages/_document.js:53:83
  at __webpack_require__.X (/lab/samples/hardened-create-cosmos-app/.next/server/webpack-runtime.js:185:21)
  at /lab/samples/hardened-create-cosmos-app/.next/server/pages/_document.js:53:47
  at Object.<anonymous> (/lab/samples/hardened-create-cosmos-app/.next/server/pages/_document.js:56:3)]

the exports reported on the page might as well be a sourcemap translation back from webpack_require OR abug caused by it being missing

Feb 15 '24 15:02 naugtur

@naugtur did you get to that error using the _document.js file I referenced? If so, then I am definitely intersted in hearing as much.

I reproduced the error as well in this repo and it did it by changing the strategy attribute to beforeInteractive. (I've since arrived at the belief that _document.js approach is the best solution.)

https://github.com/tgrecojs/hardened-nextjs/blob/b0c77d97f90b3b536e75390c62655afd9ee61c3a/source/shared/HOCs/withHardenedJs.hoc.js#L3-L15

I can see this my solution fixes this in the repo related to this issue if it would be most helpful. cc @samsiegart @erights

Feb 15 '24 17:02 tgrecojs

@tgrecojs The difference between yours and this repo seems to be this one is running lockdown on the serverside and yours is running it on clientside.

What is the intended result?
re your way of adding it to the clientside - this reminds me of a similar situation in our work on lockdown for react native where we were trying to find the right place to run it. Are you sure this way you're not running lockdown too late?

Feb 20 '24 14:02 naugtur

What is the intended result?

This question is best suited for @samsiegart. It's my understanding that the intended result is for this error to go away, and for this application to properly run lockdown.

The difference between yours and this repo seems to be this one is running lockdown on the serverside and yours is running it on clientside.

I‘m assuming you are referring to the repo referenced in my last comment, and not the one I recreated last week. If you're referring to this project, then let me know how you were able to confirm lockdown was being called on the client.

I pushed up a change last night after doing a bit of investigating now it seems as though @endo/init is being loaded on the server side however lockdown doesn't seem to get invoked on the server. If it does, then the behavior isn't consistent.

the next.js docs on Script shows the following:

Although the default behavior of next/script allows you to load third-party scripts in any page or layout, you can fine-tune its loading behavior by using the strategy property:

beforeInteractive: Load the script before any Next.js code and before any page hydration occurs.

afterInteractive: (default) Load the script early but after some hydration on the page occurs.

lazyOnload: Load the script later during browser idle time. worker: (experimental) Load the script in a web worker.

This indicates that next.js to load this code before any hydration is done leading me to believe that hardened javascript would be loaded on the server before any other code (including next.js internals). There was one big issue though - beforeInteractive without async leads to the very error @samsiegart is mentioning.

The following code loads the script properly, but it still doesn't consistently download endo before next.js code.

    <Script
          async
          type="module"
          src="https://esm.sh/@endo/[email protected]"
          strategy="afterInteractive"
        />

Here's a repo to the latest implementation - https://github.com/tgrecojs/nextjs-with-endo-init/blob/main/pages/_document.js

A live URL can be viewed here - https://nextjs-with-endo-init-kpj7t18sm-thomasgreco.vercel.app/

Feb 21 '24 15:02 tgrecojs

It would also be preferred to be able to load ses from a local bundle as opposed to the url https://esm.sh/@endo/[email protected]

Feb 21 '24 18:02 samsiegart

I'm getting the impression that you may have viewed the repo I spun up as a potential solution to the issues your facing with create-cosmos-app. I want to quickly clarify that this was not my intention at all. It was only meant to provide some added context into integrating next.js with SES.

So, as for your ask...

It would also be preferred to be able to load ses from a local bundle as opposed to the url https://esm.sh/@endo/[email protected]

I can reference @endo/init locally without issue. But is that the only ask here, or am I missing something. If it's not, then please let me know what your end goal is so I can try and assist.

Feb 21 '24 19:02 tgrecojs

Yea the intended result is that we can use endo APIs inside the application on top of next.js+create-cosmos-app. I guess if the local @endo/init is a non-factor, then we're just on the question of whether we need beforeInteractive or not. I can try based on your example and see if it works to call makeAgoricWalletConnection without beforeInteractive. If so your <Script> tag solution could suffice.

Feb 22 '24 02:02 samsiegart

I came across this when trying to fork dapp-offer-up. I went around it by disabling server side rendering.

demo: https://dapp-offer-up-next-ui.vercel.app/

Mar 06 '24 08:03 gacogo

The goal here seems to be using next.js' SSR capabilities and from what i've gathered the Script tag is the vehicle that will get us to the desired outcome. I've just drafted a PR for this here.. I'm adding the comments from that here for visibility.

I'm not completely over the hill yet but I think there is some solid progress made. For one, i'm almost certain that the approach that needs to be taken is to use beforeInteractive with _document.js. This change makes use of this approach, and everything seems to render properly. There is only one issue - @endo/init doesn't seem to be doing anything?

Below is a screen grab of the application booting up. You can see that @endo/init is the first script loaded out of the bunch, which is the expected behavior of beforeInteractive. But it doesn't seem to actually be calling lockdown.

https://github.com/agoric-labs/hardened-create-cosmos-app/assets/6646552/5e213ddf-5a15-4814-91bb-0849a8bd16f5

Here is the URL for this deployment - https://create-cosmos-app-three.vercel.app/.

@endo/init seems to be getting imported, but it just stops there resulting in Object.isFrozen(Array.prototype) returns false.

I'm wondering if anyone can give additional input on how @endo/init is functioning here. I'm not sure why lockdown doesn't seem to be getting called? And how can I make sure the @endo/init invokes lockdown (I expected it to invoke lockdown automatically).

It seems like there is something I'm missing here but not exactly sure what 🤔 cc @naugtur @kriskowal @kumavis

Mar 07 '24 00:03 tgrecojs

Hey @samsiegart . I've created a pull request] intended to fix some of these bugs, using a slightly modified installSesLockdown.ts. I hope you find it helpful.

Issues that still persist during development: Once you make a change on a file and attempt to reload page manually, WebPack will throw errors. After making changes you'll have to run yarn dev manually after making changes. Live reload seems to work however.

Edit: I've hosted it here https://hardened-create-cosmos-app.vercel.app/

Mar 16 '24 14:03 gacogo

@gacogo this is a really interesting solution! I do have one question - are you still disabling SSR in your PR?

Also, i'm not sure if you read any of my comments on this issue but I provide a fairly detailed look into how next.js suggests loading scripts (without disabling SSR).

Mar 16 '24 20:03 tgrecojs

@tgrecojs Am not disabling ssr with this solution. I also tried dynamic imports with {ssr: true} and it worked.

Mar 17 '24 04:03 gacogo

What's the status of this? Should this be tracked as an endo ecosystem compat issue https://github.com/endojs/endo/issues/2037 ?

Apr 01 '24 00:04 erights

@gacogo I tried to use you solution with:

(async () => {
  await import("ses"); // adds lockdown, harden, and Compartment
  const consoleTaming = process.env.NEXT_PUBLIC_CONSOLE_TAMING  ? 'unsafe' : 'safe';

  lockdown({
    errorTaming: "unsafe",
    overrideTaming: "severe",
    consoleTaming,
  });

  Error.stackTraceLimit = Infinity;

})();

export default function initiateLockdown() {
  return Promise.resolve();
}

But I still get the error:

[SyntaxError: Possible import expression rejected at [module]:32. (SES_IMPORT_REJECTED)
  at rejectImportExpressions (webpack-internal:///(rsc)/./node_modules/ses/src/transforms.js:155:65)
  at mandatoryTransforms (webpack-internal:///(rsc)/./node_modules/ses/src/transforms.js:243:12)
  at applyTransforms (webpack-internal:///(rsc)/./node_modules/ses/src/transforms.js:257:14)
...

does it work for you now?

May 28 '24 16:05 RicFer01

endo
endo copied to clipboard

App scaffolded with create-cosmos-app cannot call lockdown successfully

Describe the bug

Error Details:

Steps to reproduce

Expected behavior

Platform environment

Additional context

Screenshots

README.md

Next.js X @endo/init

Checking for tamper-proof globals.

endo endo copied to clipboard

App scaffolded with create-cosmos-app cannot call lockdown successfully

Describe the bug

Error Details:

Steps to reproduce

Expected behavior

Platform environment

Additional context

Screenshots

README.md

Next.js X @endo/init

Checking for tamper-proof globals.

endo
endo copied to clipboard