[more of an improvement to documentation really] DataHandler

[elChapoSing]

I think the DataHandler option is underappreciated in dygraphs. It would make sense to include it in the documentation in a more detailed way.

• How to have a dygraph point to a custom DataHandler ? (Is it a function ? Does one need to write the DataHandler in a separate file at a specific path for dygraphs to pick it up automatically by name?)

• In the definition of the DataHandler, what absolutely needs to be included and what is optional ?

• How to use the extra information provided by the DataHandler to build a different graph ? (this is already somewhat achieved by the bars examples)

• How to use the extra information provided by the DataHandler in all the events function provided by dygraphs ?

[BinaryBolt]

I would like to bring this issue back to life again. Is there anything to work with or how to use the DataHandler? Is there anybody who is able to provide a simple example on how one can use it?

[mirabilos]

BinaryBolt dixit:

I would like to bring this issue back to life again. Is there anything to work with or how to use the DataHandler? Is there anybody who is able to provide a simple example on how one can use it?

There’s the proposal, linked from:

https://dygraphs.com/options.html#dataHandler

Maybe that helps?

[BinaryBolt]

@mirabilos

There’s the proposal, linked from: https://dygraphs.com/options.html#dataHandler Maybe that helps?

That is how I actually stumbled upon this feature. I am still figuring out how I am able to use it. The proposal is clear but doesn't explicitly state an example which is ready to use. Additionally it is a little confusing as the title suggests that this proposal hasn't been implemented yet. I would like to have a minimal reproducible example to help me out. Currently I am still unsure if the dataHandler feature is actually something which helps me out in my work.

[mirabilos]

BinaryBolt dixit:

That is how I actually stumbled upon this feature.

Ah, okay.

I am still figuring out how I am able to use it.

Right. Try and error, and read the source code to get the current API because the methods have gained extra parameters in the meantime :|

The proposal is clear but doesn't explicitly state an example which is ready to use.

I don’t see one either (with grepping the source), but there is extras/rebase.js which implements one, plus the ones in core.

If you can figure out a self-contained example we could add to the demos by all means contribute it ;-)

Additionally it is a little confusing as the title suggests that this proposal hasn't been implemented yet.

As far as I can tell it not only has been implemented but also enhanced later on. The initial proposal is the “best” it has for documentation though, which is probably why it was still linked (and I have made a local copy of the document so it won’t go away).

Unfortunately, src/datahandler/datahandler.js still has errors from jsdoc so we don’t even have an autogenerated API documentation. The file does have informative comments though.

I would like to have a minimal reproducible example to help me out.

I totally get that. Having more of these examples, not just for this feature, would help. Sometimes the mailing list archives or old issues contain useful info…

Currently I am still unsure if the dataHandler feature is actually something which helps me out in my work.

Right, that’s got to be annoying, not even knowing that. The proposal (especially the stuff on PDF pages 4‥5) doesn’t help?

I could suggest a web search to see if people have implemented things using custom data handlers. The rebase example notably does…

RebaseHandler.prototype = new Dygraph.DataHandlers.DefaultHandler();

… for inheritance, so maybe search for something like that?

Good luck, //mirabilos

«MyISAM tables -will- get corrupted eventually. This is a fact of life. » “mysql is about as much database as ms access” – “MSSQL at least descends from a database” “it's a rebranded SyBase” “MySQL however was born from a flatfile and went downhill from there” – “at least jetDB doesn’t claim to be a database” ‣‣‣ Please, http://deb.li/mysql and MariaDB, finally die!

[BinaryBolt]

@mirabilos I think I was able to successfully piece together the information and implement a custom data handler. Unfortunately I am still missing something because I was assuming the following: The data I would like to pass into Dygraphs is an 2d-array of the following form

const myData = [
    [0, 1, {metadata: 'My Extra Metadata to use in the DataHandler'}],
    [1, 2, {metadata: 'My Extra Metadata to use in the DataHandler'}],
    [2, 3, {metadata: 'My Extra Metadata to use in the DataHandler'}]
];

const myChart = {
    labels: ['X', 'Y']
};

So in the beginning I was thinking that I could use the first 2 indexes for x & y and the object could then be used somewhere in the DataHandler to enable more sophisticated visualizations. But when I pass myData into the graph it will crash as it can't parse the object (Error: Expected number or array but got object: [object Object].). Working through the DataHandler and the appropriate source I was able to draw the chart but only without the object. So what exactly can I then do with the DataHandler when I am not even able to pass in extra metadata? According to the source it is only possible to add additional data in the extracSeries function (see datahandler.js#L91). That is not what I was expecting as the place where I receive the extra metadata is the same where I receive the actual data to visualize. Am I missing something? What is the point of the DataHandler when I am not actually customize how the incoming data is handled?

[mirabilos]

Hm, can you link to your WIP? Then I can jump in more easily (at the moment I’ve not got much spare time for this), maybe I can have a look.

[BinaryBolt]

@mirabilos Here is a fiddle: https://jsfiddle.net/2x59jdgt/
On line 180 I am pushing just regular data into the array and on line 182 I left something to demonstrate what I am trying to push into it. Here is the source too: HTML:

<div id="div_g" style="width:600px; height:300px;"></div>
<p>This test is modeled after a 
<a href="http://www.highcharts.com/demo/?example=dynamic-update&amp;theme=default">highcharts
test</a>. New points should appear once per second. Try zooming and 
panning over to the right edge to watch them show up.</p>

class MyDataHandler {
    X = 0;
    Y = 1;
    EXTRAS = 2;
    /**
     * DataHandler proposal: https://dygraphs.com/datahandler-proposal.pdf
     */
    constructor() {}

    /**
     * TAKEN FROM: https://github.com/danvk/dygraphs/blob/master/src/datahandler/datahandler.js#L94
     *
     * Extracts one series from the raw data (a 2D array) into an array of the
     * unified data format.
     * This is where undesirable points (i.e. negative values on log scales and
     * missing values through which we wish to connect lines) are dropped.
     * TODO(danvk): the "missing values" bit above doesn't seem right.
     *
     * EXPLANATION FROM THE PROPOSAL:
     * The extract series method is responsible for extracting a single series data from the general data
     * array. It must return the series in the unified data format. It may or may not add extras for later
     * usage.
     *
     * @param rawData - The raw data passed into dygraphs where rawData[i] = [x,ySeries1,...,ySeriesN].
     * @param seriesIndex - Index of the series to extract. All other series should be ignored.
     * @param options - Dygraph options.
     * @returns The series in the unified data format where series[i] = [x,y,{extras}].
     */
    extractSeries(rawData, seriesIndex, options) {
       
        const series = [];
        for (let i = 0; i < rawData.length; i++) {
            // const extras = { name: 'My Extra', data: Math.random() };
            series.push([rawData[i][0], rawData[i][seriesIndex]]);
        }
        return series;
    }

    /**
     * TAKEN FROM: https://github.com/danvk/dygraphs/blob/master/src/datahandler/datahandler.js#L176
     *
     * Computes the range of the data series (including confidence intervals).
     *
     * EXPLANATION FROM THE PROPOSAL:
     * This method computes the extremes of the supplied rolledData. It may be pruned compared to the
     * data returned by the DataHandler.rollingAverage method, but generally contains the data returned
     * from it. The given dateWindow must be considered for the computation of the extreme values.
     * Extras may be used if needed.
     *
     * @param series - series The series in the unified data format where series[i] = [x, y, {extras}].
     * @param dateWindow - dateWindow The x-value range to display with the format: [min, max].
     * @param stepPlot - stepPlot Whether the stepPlot option is set.
     * @returns The low and high extremes of the series in the given window with the format: [low, high].
     */
    getExtremeYValues(series, dateWindow, stepPlot) {
        let min = series[0][1];
        let max = series[0][1];
        let i = series.length;
        while (i--) {
            min = series[i][1] < min ? series[i][1] : min;
            max = series[i][1] > max ? series[i][1] : max;
        }
        return [min, max];
    }

    /**
     * TAKEN FROM: https://github.com/danvk/dygraphs/blob/master/src/datahandler/datahandler.js#L108
     *
     * Converts a series to a Point array. The resulting point array must be returned in increasing order of idx property.
     *
     * MY NOTES:
     * Point array needs to be of the PointType: https://github.com/danvk/dygraphs/blob/ea40d17cd1d9f5f4f0a12b509615aac72420d33f/src/dygraph-types.js#L31
     * @typedef {{
     *     idx: number,
     *     name: string,
     *     x: ?number,
     *     xval: ?number,
     *     y_bottom: ?number,
     *     y: ?number,
     *     y_stacked: ?number,
     *     y_top: ?number,
     *     yval_minus: ?number,
     *     yval: ?number,
     *     yval_plus: ?number,
     *     yval_stacked
     * }}
     *
     * @param series - The series in the unified data format where series[i] = [x,y,{extras}].
     * @param setName - Name of the series.
     * @param boundaryIdStart - Index offset of the first point, equal to the number of skipped points left of the date window minimum (if any).
     * @returns List of points for this series.
     */
    seriesToPoints(series, setName, boundaryIdStart) {
        const points = [];
        for (let i = 0; i < series.length; i++) {
            const item = series[i];
            const yRaw = item[1];
            const yVal = yRaw === null ? null : this.parseFloat(yRaw);
            const point = {
                x: NaN,
                y: NaN,
                xval: this.parseFloat(item[0]),
                yval: yVal,
                name: setName,
                idx: i + boundaryIdStart,
                canvasx: NaN,
                canvasy: NaN
            };
            points.push(point);
        }
        this.onPointsCreated_(series, points);
        return points;
    }

    /**
     * TAKEN FROM: https://github.com/danvk/dygraphs/blob/master/src/datahandler/datahandler.js#L149
     *
     * Callback called for each series after the series points have been generated
     * which will later be used by the plotters to draw the graph.
     * Here data may be added to the seriesPoints which is needed by the plotters.
     * The indexes of series and points are in sync meaning the original data
     * sample for series[i] is points[i].
     *
     * @param series - The series in the unified data format where series[i] = [x,y,{extras}].
     * @param points - The corresponding points passed to the plotter.
     */
    onPointsCreated_(series, points) {
        // console.log('onPointsCreated_', series, points);
    }

    rollingAverage(series, rollPeriod, options, seriesIndex) {
        // console.log('rollingAverage');
        return [0, 1, 2, 3, 4];
    }

    onLineEvaluated() {
        // console.log('onLineEvaluated');
        return [0, 1, 2, 3, 4];
    }

    /**
     * TAKEN FROM: https://github.com/danvk/dygraphs/blob/master/src/datahandler/datahandler.js#L199
     *
     * Optimized replacement for parseFloat, which was way too slow when almost
     * all values were type number, with few edge cases, none of which were strings.
     *
     * @param val
     * @returns
     */
    parseFloat(val) {
        // console.log('parse', val);
        if (val === null) return NaN;

        return val;
    }
}


$(document).ready(function() {
      var data = [];
      var t = new Date();
      for (var i = 10; i >= 0; i--) {
        var x = new Date(t.getTime() - i * 1000);
        data.push([x, Math.random()]);
      }

      var g = new Dygraph(document.getElementById("div_g"), data,
                          {
                            drawPoints: true,
                            showRoller: true,
                            valueRange: [0.0, 1.2],
                            labels: ['Time', 'Random'],
                            dataHandler: MyDataHandler
                          });
      // It sucks that these things aren't objects, and we need to store state in window.
      window.intervalId = setInterval(function() {
        var x = new Date();  // current time
        var y = Math.random();
        // Works
        data.push([x, y]);
        // Doesn't work
        // data.push([x, y, {myExtra: 'myExtraData'}]);
        g.updateOptions( { 'file': data } );
      }, 1000);
    }
);