Can't get the REST persister to work with any of the server options

[adguy360]

So I've been using Polly for a while now as an API mocking solution for my app's dev environment (React app). I've been using the LS persister and everything is working just great, thanks for the tool :)

Here's my LS Polly config:

Polly.register(FetchAdapter);
Polly.register(LocalStoragePersister);

new Polly('Development mode', {
  adapters: ['fetch'],
  persister: 'local-storage',
  persisterOptions: {
    'local-storage': {
       key: '__pollyjs__',
     },
  },
  logging: true,
  });
}

I wanted to start using the REST persister so that I could share the recordings with the rest of my team as well as use them as our cypress fixtures.

My app uses a simple custom webpack config (which I inherited) and webpack-dev-server (3.11.0) to run, so I edited the above Polly config to register the REST perstiter and use it, and used @pollyjs/node-server as recommended with registerExpressAPI:

const webpack = require('webpack');
const WebpackDevServer = require('webpack-dev-server');
const open = require('open');
const url = require('url');
const { registerExpressAPI } = require('@pollyjs/node-server');

const config = require('../webpack-config/webpack.config.environments');

config.entry.application.unshift('webpack/hot/only-dev-server');
config.entry.application.unshift(`webpack-dev-server/client?${config.output.publicPath}`);
config.plugins.push(new webpack.HotModuleReplacementPlugin());
config.plugins.push(new webpack.NamedModulesPlugin());

const compiler = webpack(config);

const port = process.env.NODE_PORT || 443;
const publicPath = config.output.publicPath;
const host = '0.0.0.0';

const devServerOptions = {
  host,
  public: `${url.parse(publicPath).hostname}:${port}`,
  headers: {
    'Access-Control-Allow-Origin': '*',
    'Accept-Ranges': 'bytes',
  },
  overlay: {
    errors: true,
    warnings: true,
  },
  historyApiFallback: true,
  clientLogLevel: 'warning',
  quiet: true,
  hot: true,
  port,
  publicPath,
  before: function(app) {
    registerExpressAPI(app);
  },
};

new WebpackDevServer(compiler, devServerOptions).listen(port, host, (err, result) => {
  if (err) {
    console.log(err);
  }
  open(publicPath);
});

I'm getting a network error on, what I assume to be, the first record attempt:

pollyjs-persister-rest.js:9028 GET http://localhost:3000/polly/Development-mode_795867713 net::ERR_CONNECTION_REFUSED

Now I understand this may well be due to something in my webpack/dev-server configs, but since my webpack knowledge is relatively poor 😢 I would love if you could offer some assistance here 🙏

Note - I also tried running the cli instead of the node-server, the browser's console says calls are recorded though I can't find the recordings dir anywhere nor does it ever replay.

Additional info

Dependencies

{
  "@pollyjs/adapter-fetch": "5.0.0",
   "@pollyjs/cli": "5.0.0",
   "@pollyjs/core": "5.0.0",
   "@pollyjs/node-server": "5.0.0",
   "@pollyjs/persister-local-storage": "5.0.0",
   "@pollyjs/persister-rest": "5.0.0",
}

Environment

OS: MacOS 10.15.6 Browser: Chrome 84 Node: 10.15.3 Npm: 6.10.0

Thanks

[jasonmit]

const port = process.env.NODE_PORT || 443;

Looks like Polly is mounted on the Webpack server running on whatever process.env.NODE_PORT is (or 443). Either change the port to 3000 or configure Polly to talk to the correct port using the host option

new Polly('Development mode', {
  adapters: ['fetch'],
  persister: 'local-storage',
  persisterOptions: {
    'local-storage': {
       key: '__pollyjs__',
     },
  },
  logging: true,
  });
}

You're running the local-storage persister, you want to configure Polly to use the REST server.

[adguy360]

@jasonmit I just gave the working example with LS but changed it to work against the REST persister - I also note that in the issue description above ^^. I also tried both configuring the port on the polly node server to match mine (3001) and change mine to 3000 and it still doesn't work.

Can you notice anything else I might be missing here?

[adguy360]

@jasonmit @offirgolan could we reopen or at least continue discussion here?

[offirgolan]

@adguy360 if you can create a reproduction of what you're seeing, we'd be better able to help you.

[bejalane]

I have some related issue. I didn't integrate node server to webpack, just running it manually. And I see it is trying to get mocks from server, all good, I see GET requests to mock-server in the network log. But the problem it doesn't record there. I can't see any POST request or something else. I tried to make custom persister and I see saveRecording() hook is not triggering.

My code here:

import FetchAdapter from '@pollyjs/adapter-fetch';
import XhrAdapter from '@pollyjs/adapter-xhr';
import RESTPersister from '@pollyjs/persister-rest';

const polly = (() => {
  let instance;

  function createInstance() {
    Polly.register(FetchAdapter);
    Polly.register(XhrAdapter);
    Polly.register(RESTPersister);

    return new Polly('DevMocks', {
      adapters: ['fetch', 'xhr'],
      persister: 'rest',
      persisterOptions: {
        rest: {
          host: 'http://localhost:4002',
        },
      },
      logging: true,
    });
  }

  if (!instance) {
    instance = createInstance();
  }

  return instance;
})();

const setupMockedAPI = () => {
  const { server } = polly;

  return server
    .any()
    .filter(req => !req.url.includes('myapidomain.com'))
    .passthrough();
};

export default setupMockedAPI;

Server

const path = require('path');

const server = new Server({
  quiet: true,
  port: 4002,
  apiNamespace: '/polly',
  recordingsDir: path.join(__dirname, '/recordings'),
});

// Start listening and attach extra logic to the http server
server
  .listen()
  .on('error', () => { /* Add http server error logic */ });

Server is working on post requests, i checked it with postman.

[jasonmit]

@bejalane are you calling await polly.stop(); after each test? If so, can you share what is being emitted from the logger?

[bejalane]

@jasonmit Oops, sorry some code was not copied.

window.addEventListener('unload', async () => {
  await polly.stop();
});

Back to REST persister, in console I see the log and it says requests are recorded.. But as I said above there is no request to server to add the recording. Only when it is trying to find one.

One more thing, I tried LocalStorage persister, and it worked and recorded to LS. And If I make custom persister extended from LS storage saveRecording() hook is still not triggering. And for example findRecording() is triggered with no problem.

[patrickshaughnessy]

@adguy360 I think I came across a similar issue when first trying to use the REST persister. The problem was that the out-of-the-box persister expects to run in a browser context, which is evidenced by the ajax method looking for the native XHR class: https://github.com/Netflix/pollyjs/blob/master/packages/%40pollyjs/persister-rest/src/ajax.js#L8

We wanted to save / retrieve recordings from multiple node servers used during tests to a single central location (also a node server). So we needed a REST persister, but not one that uses a native browser XHR request to send / request recordings. The solution was creating a custom Node REST persister, and the only thing that really needed to change was the way AJAX was performed. Here's an example (mostly taken from the REST persister code: https://github.com/Netflix/pollyjs/blob/master/packages/%40pollyjs/persister-rest/src/index.js):

'use strict';

const Persister = require('@pollyjs/persister');
const { buildUrl } = require('@pollyjs/utils');

const request = require('request');

class NodeRESTPersister extends Persister {
	static get id() {
		return 'noderest';
	}

	static get name() {
		// NOTE: deprecated in 4.1.0 but proxying since it's possible "core" is behind
		// and therefore still referencing `name`.  Remove in 5.0.0
		return this.id;
	}

	get defaultOptions() {
		return {
			host: 'https://localhost.paypal.com:9999',
			apiNamespace: '/polly'
		};
	}

	async ajax(url, args) {
		const { host, apiNamespace } = this.options;
		console.log('buildUrl(host, apiNamespace, url)', buildUrl(host, apiNamespace, url));

		return new Promise(resolve => {
			const uri = buildUrl(host, apiNamespace, url);
			const payload = {
				...args,
				uri,
				rejectUnauthorized: false,
				json: true
			};
			request(payload, (err, response, body) => {
				console.log('Polly AJAX', err, response, body);
				return resolve(response || {});
			});
		});
	}

	async findRecording(recordingId) {
		const response = await this.ajax(`/${encodeURIComponent(recordingId)}`, {
			Accept: 'application/json; charset=utf-8'
		});

		return this._normalize(response);
	}

	async saveRecording(recordingId, data) {
		await this.ajax(`/${encodeURIComponent(recordingId)}`, {
			method: 'POST',
			body: this.stringify(data),
			headers: {
				'Content-Type': 'application/json; charset=utf-8',
				Accept: 'application/json; charset=utf-8'
			}
		});
	}

	async deleteRecording(recordingId) {
		await this.ajax(`/${encodeURIComponent(recordingId)}`, {
			method: 'DELETE'
		});
	}

	_normalize(response = {}) {
		/**
		 * 204 - No Content. Polly uses this status code in place of 404
		 * when interacting with our Rest server to prevent throwing
		 * request errors in consumer's stdout (console.log)
		 */
		if (response.statusCode === 204) {
			/* return null when a record was not found */
			return null;
		}

		return response.body;
	}
}

module.exports = NodeRESTPersister;

I think if you're trying to set up a persister from the webpack-dev-server (a node server), the persister needs to run in a node context, rather than a browser context.