Customize paths to cache

[alex289]

trafficstars

Add own paths to the cache

I'm using the cypress orb to test my Nextjs application. A build is required to run the app so it would make it faster to cache the created build (and restore it). In my case it is .next/cache.

[bahmutov]

Ohhh this is an interesting idea, definitely worth exploring. Could it be done using some kind of post install command right now though?

Sent from my iPhone

On Sep 3, 2020, at 13:59, Alex [email protected] wrote:

 Add own paths to the cache

I'm using the cypress orb to test my Nextjs application. A build is required to run the app so it would make it faster to cache the created build (and restore it). In my case it is .next/cache.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[alex289]

Okey, so now it is working, but nothing will be cached because the build command is executed after the post step save_cache.

[kylemh]

Where the recommendation comes from: https://github.com/vercel/next.js/blob/master/errors/no-cache.md

My current CircleCI Config:

version: 2.1

orbs:
  cypress: cypress-io/cypress@1

executors:
  default:
    resource_class: large
    docker:
      - image: cypress/base:14.10.1 # keep in sync with .nvmrc and any other executors

parameters:
  workspace_root:
    type: string
    default: '~/'
  primary_cache_key:
    type: string
    default: 'air/next-{{ .Branch }}-{{ checksum "yarn.lock" }}'
  backup_cache_key:
    type: string
    default: 'air/next-{{ .Branch }}-'
  org_level_cache_key:
    type: string
    default: 'air/next-'

exclude_main: &exclude_main
  filters:
    branches:
      ignore: main

aliases:
  - &attach_workspace
    attach_workspace:
      at: << pipeline.parameters.workspace_root >>

  - &restore_cache
    restore_cache:
      keys:
        - << pipeline.parameters.primary_cache_key >>
        # Fallback in case checksum fails.
        - << pipeline.parameters.backup_cache_key >>
        - << pipeline.parameters.org_level_cache_key >>
  - &yarn_install
    run:
      name: 'Installing dependencies...'
      command: yarn install --non-interactive --frozen-lockfile --cache_folder << pipeline.parameters.workspace_root >>.cache/yarn

  - &create_npmrc
    run:
      name: 'Create .npmrc to set NPM_TOKEN...'
      command: echo "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > ~/.npmrc

jobs:
  install_dependencies:
    executor: default
    steps:
      - checkout
      - *attach_workspace
      - *create_npmrc
      - *restore_cache
      - *yarn_install
      - run: node --version
      - run: yarn --version
      - save_cache:
          key: << pipeline.parameters.primary_cache_key >>
          paths:
            - node_modules
            - << pipeline.parameters.workspace_root >>.cache/yarn
            - << pipeline.parameters.workspace_root >>.cache/Cypress

workflows:
  ci:
    jobs:
      - install_dependencies:
          <<: *exclude_main

      - cypress/run:
          <<: *exclude_main
          name: integration_tests
          requires:
            - install_dependencies
          executor: default
          pre-steps:
            - checkout
            - *attach_workspace
            - *restore_cache
            - *yarn_install
          store_artifacts: true
          attach-workspace: true
          record: true # record results on Cypress Dashboard
          parallel: true # split all specs across machines
          parallelism: 4 # use X number of CircleCI machines
          command-prefix: yarn # dont use npx
          group: 'all tests' # name this group "all tests" on the dashboard
          start: 'yarn build && yarn start' # start server before running tests
          wait-on: http://localhost:3000 # wait until server is ready

I'm sure big improvements can happen once webpack 5 is used because of partial builds from cache, but I'm curious how saving a cache may even improve anything...

I would need to build first, send the build over as part of the workspace to the orb steps, and THEN start... right?

[alex289]

Yes, not only that your CircleCI config is correct but you need to first build the app and then save it as part of the workspace. It is not necessary to make this part of the orb but I thought it would make it easier for the users and make. I thought caching the build of NextJs will make the next build even faster because the components that have not changed are going to be recovered from the cache instead of being recreated completely.

[kylemh]

Was this done @Alex289 ?

[alex289]

No, but I don't use this anymore and it doesn't seem like it will be done.

[kylemh]

🙏🏼 will you reopen and then unsubscribe instead. I guess I can just copy-paste your your issue into a new one too 😂

[alex289]

Do you plan on working on this or why do you want this to be open?

[kylemh]

cuz it's a legit issue! i guess I think to myself: it may not be your issue any longer, but it's still a problem for all CircleCI users.

it's possible it never gets done, but leaving it up here could help ensure it gets done sooner than it might otherwise

[jennifer-shehane]

Closing this as this is not be something that we deliver for the orb. The orb is intended for simple use cases where Cypress handles caching your node modules installation and caching of Cypress itself as well as running Cypress. If you need customization beyond what seems simple to accomplish in the orb, we suggest using a customized circle yml configuration.