KeplerGlSchema.load can't reload output from KeplerGlSchema.save

[derek-nascent]

Describe the bug saving-loading-w-schema.md reads like the way to persist a map's state across sessions is to call KeplerGlSchema.getDatasetToSave or KeplerGlSchema.save which internally calls KeplerGlSchema.getDatasetToSave. With the result of that function call in hand, the suggested solution is to stash that json blob in a db until needed and then feed it back to KeplerGlSchema.load. In my case I am not persisting to a db, but rather the store so that I can reload the map where the user left off, after a page navigation.

However, passing the results of KeplerGlSchema.save via the store to KeplerGlSchema.load and then passing the results of that to addDataToMap fails silently. The map is black and no error messages are logged in the console. Interestingly, I can pass the results of KeplerGlSchema.save to loadFiles and then pass that to addDataToMap, and the map will render. I am still investigating, but wanted to document this here because I am not sure if there is really a bug, or unclear documentation, or just a PEBKAC issue ;)

To Reproduce Steps to reproduce the behavior:

1. Add this code to your basic kepler gl page.

convertToBlob(state: any): Array<any> {
    let files = [];
    
    let blob = new Blob([JSON.stringify(state)], {type: "application/json"});
    files.push(new File([blob], "blob.json"));

    return files;
  }

addDataViaKeplerApi(): void {
    const { datasets, config } = this.props.lastKeplerGl;
    const mapToLoad = KeplerGlSchema.load(datasets, config);
    this.props.dispatch(wrapTo("findView", addDataToMap(mapToLoad)));
  }
 
componentDidMount() {
    if(this.props.lastKeplerGl) {
      console.log("loading previous kepler state");
      this.addDataViaKeplerApi(); // comment this line to test the next.
      //this.props.dispatch(wrapTo("findView", loadFiles(this.convertToBlob(this.props.lastKeplerGl))));  //uncomment this to see how to load previous state with loadFiles.
    }
  }

componentWillUnmount() {
    // setKeplerLastState is an action that triggers the saving of the current keplerGl state to the store.
    this.props.dispatch(setLastKeplerState(KeplerGlSchema.save(this.props.keplerGl.findView)));
  }

Expected behavior I expected the map to load in exactly the same state as when the user navigated away from the view.

Desktop (please complete the following information):

• macOS Mojave 10.14.5
• Browser: Chrome 75.0.3770.80

Additional context Here's my dependencies from package.json. I am using yarn as package manager

"dependencies": {
    "@antv/data-set": "0.10.2",
    "@antv/g2-brush": "0.0.2",
    "@material-ui/core": "4.1.1",
    "@material-ui/icons": "4.2.0",
    "@material-ui/styles": "4.1.1",
    "@types/googlemaps": "3.30.19",
    "@types/jest": "24.0.13",
    "@types/markerclustererplus": "2.1.33",
    "@types/material-ui": "0.21.6",
    "@types/node": "12.0.2",
    "@types/prop-types": "15.7.1",
    "@types/react": "16.8.18",
    "@types/react-dom": "16.8.4",
    "@types/react-redux": "7.0.9",
    "@types/react-router": "5.0.1",
    "@types/react-router-dom": "4.3.3",
    "@types/react-virtualized": "9.21.1",
    "@types/recompose": "0.30.6",
    "@types/redux-actions": "2.6.1",
    "ajv": "6.10.0",
    "animate.css": "3.7.0",
    "apollo-boost": "0.3.1",
    "babel-preset-env": "1.7.0",
    "bizcharts": "3.5.2",
    "chartist": "0.10.1",
    "d3-force": "1.0.6",
    "deck.gl": "6.2.4",
    "googlemaps": "1.12.0",
    "graphql": "14.3.0",
    "highcharts": "7.1.1",
    "history": "4.7.2",
    "kepler.gl": "1.0.0",
    "lodash": "4.17.11",
    "luma.gl": "6.4.3",
    "moment": "2.24.0",
    "node-sass": "4.11.0",
    "nouislider": "13.1.0",
    "perfect-scrollbar": "1.4.0",
    "prop-types": "15.7.2",
    "react": "16.8.6",
    "react-animate-on-scroll": "2.1.5",
    "react-big-calendar": "0.20.3",
    "react-bootstrap-sweetalert": "4.4.1",
    "react-chartist": "0.13.3",
    "react-datetime": "2.16.3",
    "react-dom": "16.8.6",
    "react-gojs": "4.1.0",
    "react-google-charts": "3.0.11",
    "react-google-maps": "9.4.5",
    "react-image-gallery": "0.8.12",
    "react-jsx-highmaps": "1.2.0",
    "react-jsx-highstock": "3.5.0",
    "react-jvectormap": "0.0.6",
    "react-map-gl": "4.0.0",
    "react-palm": "3.0.0",
    "react-redux": "5.1.1",
    "react-router-dom": "5.0.1",
    "react-scripts": "3.0.1",
    "react-slick": "0.23.2",
    "react-swipeable-views": "0.13.1",
    "react-table": "6.9.2",
    "react-tagsinput": "3.19.0",
    "react-tooltip": "3.10.0",
    "react-virtualized": "9.19.1",
    "react-vis": "1.11.7",
    "recompose": "0.30.0",
    "redux": "4.0.1",
    "styled-components": "4.2.0",
    "typescript": "3.4.5"
  }

[heshan0131]

@derek-nascent KeplerGLSchema.save returns a single json blob {datasets: [], config: {}, info: {}}, KeplerGLSchema.load takes in datasets and config as 2 arguments KeplerGLSchema.load(datasets, config). Did you directly pass the saved json blob to load? That might be the problem

[derek-nascent]

Thanks for the response!

I think so. When I am saving my state I am saving it to the store as a single object via this action (findView is my map's id)

this.props.dispatch(setLastKeplerState(KeplerGlSchema.save(this.props.keplerGl.findView)));

Then I try to reload by destructuring that object and then passing the result to load. I console.log them and they both have values.

const { datasets, config } = this.props.lastKeplerGl;
const mapToLoad = KeplerGlSchema.load(datasets, config);
this.props.dispatch(wrapTo("findView", addDataToMap(mapToLoad)));

I was able to keep step through the code for about an hour, but then work priorities and stuff. I'll keep trying when I have spare cycles as we're using kepler extensively, and I'd like to follow the intended design rather than hacking until it works. That almost always means I'm doing something wrong ;P

It is weird to me though that the exact same object when blobified => fileArray and then sent to loadFiles renders just fine.

[tjstavenger-pnnl]

I think I've encountered the same problem. I export data & config from Kepler and then try to reload the keplergl.json using addDataToMap(). I get a map centered in the right geographic location, but no data, layers, & filters. I've skimmed through the JSON and all the dataIds, etc. match as expected. I can use the Add Data dialog successfully with the same JSON file -- map, data, & filters are loaded. If I separate out my data & config into CSV & JSON I can build my own JSON object to send into addDataToMap, as is done by several demos (like http://vis.academy/#/kepler.gl/3-load-config). I haven't been able to figure out how to send the full keplergl.json from export in successfully.

[albcunha]

In case someone stumbles on this problem. In short: Change dataset this way: Old: datasets = [ {id, label, color, allData, fields} ] New: datasets = [{version: "v1", data: {id, label, color, allData, fields}]

Here is the explanation: If you look in source code (src > schemas > schema-manager.ts > KeplerGLSchema) You will see a comment on how state to save is supposed to be:

/**
   * stateToSave = {
   *   datasets: [
   *     {
   *       version: 'v0',
   *       data: {id, label, color, allData, fields}
   *     },
   *     {
   *       version: 'v0',
   *       data: {id, label, color, allData, fields}
   *     }
   *   ],
   *   config: {
   *     version: 'v0',
   *     config: {}
   *   },
   *   info: {
   *     app: 'kepler.gl',
   *     create_at: 'Mon May 28 2018 21:04:46 GMT-0700 (PDT)'
   *   }
   * }
   *
   * Get config and data of current map to save
   * @param state
   * @returns app state to save
   */

KeplerGLSchema is saving wrong. It wont add version to the datasets, neither fields are attached to a data key. It will only do it to config. If you will, you may save save datasets normally and when you reload it, you can do something like this:

const { datasets, config } = this.props.lastKeplerGl;
const newdatasets = dataset.map((dataset: any) => ({ version: "v1", data: dataset });
const mapToLoad = KeplerGlSchema.load(datasets, props.record.Mapa.config);
props.dispatch(addDataToMap(mapToLoad));