banana icon indicating copy to clipboard operation
banana copied to clipboard

banana 1.5 - heatmap Solr version dependency?

Open dougmcclure opened this issue 10 years ago • 17 comments
trafficstars

I've tried the best I could to get the heatmap panel to work against my Solr 4.7.1. Does this panel need a newer version (eg 4.10 with some Pivot Facet improvements)?

Tks!

dougmcclure avatar Jan 17 '15 05:01 dougmcclure

@dougmcclure I tested it with Solr 4.7.0 and it works fine for me did you check your configuration for the panel?

tarek-salah avatar Jan 19 '15 10:01 tarek-salah

Yes, I've tried to use this a dozen different times with no luck rendering anything. Are there any requirements for what field types are selected for "Row Field" and "Column Field"? I'm most used to having a count type metric to drive the value for heatmap coloring.

On Mon, Jan 19, 2015 at 5:00 AM, Tarek Salah [email protected] wrote:

@dougmcclure https://github.com/dougmcclure I tested it with Solr 4.7.0 and it works fine for me did you check your configuration for the panel?

— Reply to this email directly or view it on GitHub https://github.com/LucidWorks/banana/issues/108#issuecomment-70469576.

dougmcclure avatar Jan 19 '15 14:01 dougmcclure

@dougmcclure What are the types of fields that you are using for "Row Field" and "Column Field"?

tarek-salah avatar Jan 22 '15 11:01 tarek-salah

For the two fields I've added in my last attempt, the schema shows this:

dynamicField name="*_ds" type="tdouble" indexed="true" stored="true" required="false" multiValued="false" docValues="true"

fieldType name="tdouble" class="solr.TrieDoubleField" precisionStep="8" positionIncrementGap="0"

Doug

dougmcclure avatar Jan 27 '15 02:01 dougmcclure

Any insights into this yet?

dougmcclure avatar Feb 09 '15 21:02 dougmcclure

Is your collection sharded?

nickvasilyev avatar Feb 09 '15 21:02 nickvasilyev

Yes, each collection has two shards.

On Mon, Feb 9, 2015 at 4:31 PM, Nick V [email protected] wrote:

Is your collection sharded?

— Reply to this email directly or view it on GitHub https://github.com/LucidWorks/banana/issues/108#issuecomment-73594383.

dougmcclure avatar Feb 09 '15 21:02 dougmcclure

Try adding distrib=false to heat map query and see if that helps. On Feb 9, 2015 4:54 PM, "Doug McClure" [email protected] wrote:

Yes, each collection has two shards.

On Mon, Feb 9, 2015 at 4:31 PM, Nick V [email protected] wrote:

Is your collection sharded?

— Reply to this email directly or view it on GitHub https://github.com/LucidWorks/banana/issues/108#issuecomment-73594383.

— Reply to this email directly or view it on GitHub https://github.com/LucidWorks/banana/issues/108#issuecomment-73598539.

nickvasilyev avatar Feb 09 '15 21:02 nickvasilyev

That got me to a grid, but nothing appears to be showing up in a given cell. I'm expecting seeing some number value and shading representing count.

dougmcclure avatar Feb 09 '15 22:02 dougmcclure

Alright, that's progress. See what query it is running and run it manually in sole to make sure you are getting data back. On Feb 9, 2015 5:37 PM, "Doug McClure" [email protected] wrote:

That got me to a grid, but nothing appears to be showing up in a given cell. I'm expecting seeing some number value and shading representing count.

— Reply to this email directly or view it on GitHub https://github.com/LucidWorks/banana/issues/108#issuecomment-73605909.

nickvasilyev avatar Feb 09 '15 22:02 nickvasilyev

Ok, ran the query in browser and here's a sample of the results returned. Looks like I have some results which should be plotting.

{ "responseHeader":{ "status":0, "QTime":6, "params":{ "facet":"true", "indent":"true", "q":":", "facet.limit":"10", "distrib":"false", "facet.pivot.mincount":"0", "wt":["json", "json"], "facet.pivot":"collectionName_ts,numberSuccessful_ds", "fq":["timestamp:[NOW/MINUTE-5MINUTE TO NOW/MINUTE+1MINUTE]", "_datasource:"S-STATS-153""], "rows":"0"}}, "response":{"numFound":26,"start":0,"docs":[] }, "facet_counts":{ "facet_queries":{}, "facet_fields":{}, "facet_dates":{}, "facet_ranges":{}, "facet_pivot":{ "collectionName_ts,numberSuccessful_ds":[{ "field":"collectionName_ts", "value":"GRStats-Collecti", "count":13, "pivot":[{ "field":"numberSuccessful_ds", "value":2.0, "count":12}, { "field":"numberSuccessful_ds", "value":3.0, "count":1}, { "field":"numberSuccessful_ds", "value":0.0, "count":0}, { "field":"numberSuccessful_ds", "value":1.0, "count":0}, { "field":"numberSuccessful_ds", "value":4.0, "count":0}, { "field":"numberSuccessful_ds", "value":5.0, "count":0}, { "field":"numberSuccessful_ds", "value":6.0, "count":0}, { "field":"numberSuccessful_ds", "value":7.0, "count":0}, { "field":"numberSuccessful_ds", "value":8.0, "count":0}, { "field":"numberSuccessful_ds", "value":9.0, "count":0}]}, { "field":"collectionName_ts", "value":"GRLogs-Collectio", "count":12, "pivot":[{ "field":"numberSuccessful_ds", "value":8.0, "count":12}, { "field":"numberSuccessful_ds", "value":0.0, "count":0}, { "field":"numberSuccessful_ds", "value":1.0, "count":0}, { "field":"numberSuccessful_ds", "value":2.0, "count":0}, { "field":"numberSuccessful_ds", "value":3.0, "count":0}, { "field":"numberSuccessful_ds", "value":4.0, "count":0}, { "field":"numberSuccessful_ds", "value":5.0, "count":0}, { "field":"numberSuccessful_ds", "value":6.0, "count":0}, { "field":"numberSuccessful_ds", "value":7.0, "count":0}, { "field":"numberSuccessful_ds", "value":9.0, "count":0}]},

dougmcclure avatar Feb 09 '15 23:02 dougmcclure

Ok, hover over the blocks that should have values and see if any data shows up in tooltip. On Feb 9, 2015 6:13 PM, "Doug McClure" [email protected] wrote:

Ok, ran the query in browser and here's a sample of the results returned. Looks like I have some results which should be plotting.

{ "responseHeader":{ "status":0, "QTime":6, "params":{ "facet":"true", "indent":"true", "q":":", "facet.limit":"10", "distrib":"false", "facet.pivot.mincount":"0", "wt":["json", "json"], "facet.pivot":"collectionName_ts,numberSuccessful_ds", "fq":["timestamp:[NOW/MINUTE-5MINUTE TO NOW/MINUTE+1MINUTE]", "_datasource:"S-STATS-153""], "rows":"0"}}, "response":{"numFound":26,"start":0,"docs":[] }, "facet_counts":{ "facet_queries":{}, "facet_fields":{}, "facet_dates":{}, "facet_ranges":{}, "facet_pivot":{ "collectionName_ts,numberSuccessful_ds":[{ "field":"collectionName_ts", "value":"GRStats-Collecti", "count":13, "pivot":[{ "field":"numberSuccessful_ds", "value":2.0, "count":12}, { "field":"numberSuccessful_ds", "value":3.0, "count":1}, { "field":"numberSuccessful_ds", "value":0.0, "count":0}, { "field":"numberSuccessful_ds", "value":1.0, "count":0}, { "field":"numberSuccessful_ds", "value":4.0, "count":0}, { "field":"numberSuccessful_ds", "value":5.0, "count":0}, { "field":"numberSuccessful_ds", "value":6.0, "count":0}, { "field":"numberSuccessful_ds", "value":7.0, "count":0}, { "field":"numberSuccessful_ds", "value":8.0, "count":0}, { "field":"numberSuccessful_ds", "value":9.0, "count":0}]}, { "field":"collectionName_ts", "value":"GRLogs-Collectio", "count":12, "pivot":[{ "field":"numberSuccessful_ds", "value":8.0, "count":12}, { "field":"numberSuccessful_ds", "value":0.0, "count":0}, { "field":"numberSuccessful_ds", "value":1.0, "count":0}, { "field":"numberSuccessful_ds", "value":2.0, "count":0}, { "field":"numberSuccessful_ds", "value":3.0, "count":0}, { "field":"numberSuccessful_ds", "value":4.0, "count":0}, { "field":"numberSuccessful_ds", "value":5.0, "count":0}, { "field":"numberSuccessful_ds", "value":6.0, "count":0}, { "field":"numberSuccessful_ds", "value":7.0, "count":0}, { "field":"numberSuccessful_ds", "value":9.0, "count":0}]},

— Reply to this email directly or view it on GitHub https://github.com/LucidWorks/banana/issues/108#issuecomment-73611700.

nickvasilyev avatar Feb 09 '15 23:02 nickvasilyev

I see the tooltip, all values are (0). None of the counts show up like they do for the ones with results in the test query.

dougmcclure avatar Feb 09 '15 23:02 dougmcclure

I tried your attached data which is:

{
  "responseHeader": {
    "status": 0,
    "QTime": 6,
    "params": {
      "facet": "true",
      "indent": "true",
      "q": ":",
      "facet.limit": "10",
      "distrib": "false",
      "facet.pivot.mincount": "0",
      "wt": [
        "json",
        "json"
      ],
      "facet.pivot": "collectionName_ts,numberSuccessful_ds",
      "fq": [
        "timestamp:[NOW/MINUTE-5MINUTE TO NOW/MINUTE+1MINUTE]",
        "_datasource:\"S-STATS-153\""
      ],
      "rows": "0"
    }
  },
  "response": {
    "numFound": 26,
    "start": 0,
    "docs": []
  },
  "facet_counts": {
    "facet_queries": {},
    "facet_fields": {},
    "facet_dates": {},
    "facet_ranges": {},
    "facet_pivot": {
      "collectionName_ts,numberSuccessful_ds": [
        {
          "field": "collectionName_ts",
          "value": "GRStats-Collecti",
          "count": 13,
          "pivot": [
            {
              "field": "numberSuccessful_ds",
              "value": 2,
              "count": 12
            },
            {
              "field": "numberSuccessful_ds",
              "value": 3,
              "count": 1
            },
            {
              "field": "numberSuccessful_ds",
              "value": 0,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 1,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 4,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 5,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 6,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 7,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 8,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 9,
              "count": 0
            }
          ]
        },
        {
          "field": "collectionName_ts",
          "value": "GRLogs-Collectio",
          "count": 12,
          "pivot": [
            {
              "field": "numberSuccessful_ds",
              "value": 8,
              "count": 12
            },
            {
              "field": "numberSuccessful_ds",
              "value": 0,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 1,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 2,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 3,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 4,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 5,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 6,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 7,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 9,
              "count": 0
            }
          ]
        },
                {
          "field": "collectionName_ts",
          "value": "GRLogs-Collectio",
          "count": 1,
          "pivot": [
            {
              "field": "numberSuccessful_ds",
              "value": 8,
              "count": 1
            },
            {
              "field": "numberSuccessful_ds",
              "value": 0,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 1,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 2,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 3,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 4,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 5,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 6,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 7,
              "count": 0
            },
            {
              "field": "numberSuccessful_ds",
              "value": 9,
              "count": 0
            }
          ]
        }
      ]
    }
  }
}

And I found the following result: Alt heatmap

Which indicates it works well, so where is your problem?

karimfikry avatar Feb 10 '15 10:02 karimfikry

Ok, I think I've got a something here. Something appears to be off in how queries are passed and results returned when using a collection name versus a collection alias (what I'm using).

When I pass in a query URL which includes the very specific collection name, I see results in the browser's JSON display like I hared above.

When I pass in a query URL which includes my collection alias (which is default for all my banana dashboards), all results coming back as zero which explains why I do not see any count data in the heat map.

Any ideas on why this is behaving like it is?

dougmcclure avatar Feb 10 '15 14:02 dougmcclure

Hi @dougmcclure - I setup a test on my machine with Solr 4.7.1 in SolrCloud mode and use two shards. Then, I created two collections (hvac and hvac2) and one alias (hvacalias) that point to them. Then I create a Banana dashboard using 'hvacalias' and everything works fine. However, when I tried to add the Heatmap panel, it will come up empty at first because Solr 4.7.1 does not support distributed facet.pivot (https://issues.apache.org/jira/browse/SOLR-2894). And therefore, I have to go to Heatmap Settings => Queries tab => Add Customer Queries textbox, and add this: &distrib=false. This will make the Heatmap panel (facet.pivot) works properly.

Is this the same setup in your environment? Are you running Banana inside your Solr server? Or you are accessing Solr on remote machines?

These are some screenshots from my test settings:

heatmap_settings_queries heatmap_settings_panel test_dashboard dashboard_settings

athana avatar Feb 11 '15 07:02 athana

Yes, exactly the same. I have banana running inside the Solr server.

dougmcclure avatar Feb 11 '15 13:02 dougmcclure