GoatCounter CSV exports – what are your use cases?

[arp242]

I'm going to make a change that disables the storing of individual pageviews; instead it will rely only on "aggregate counts" as you see on the dashboard. These are already stored separately, and the only functionality where individual pageviews are ever used is for the CSV export.

There are two reasons I'd like to remove this:

• Collecting only "aggregate counts" fits better with GoatCounter's view of "we just count a few things rather than track people".

• This is a somewhat expensive feature to support, both in terms of hosting goatcounter.com, but also in terms of development – removing it would be a lot easier than keeping it as an optional feature.

---

But removing this will break the CSV export; that data simply won't be available any more. For most uses, I think that's a good thing because it wasn't using this data in the first place.

I don't necessarily mind keeping this as an optional feature, but it's not entirely clear to me this is actually useful.

So my question is: What are your use cases for the CSV exports? Would the API cover this? If not, is there something that can be changed/added in the API? I have a suspicion that most uses of the CSV export are really due to deficiencies elsewhere that can be fixed/added, but I need some more information on real-world use cases to be sure.

[TigersWay]

That would be a real big trouble for myself! I use the CSV to be able to group urls together and get different results, "section" results for example. I don't need each unique view per se but I don't see how I could re-group without. We talked about it a few months ago!

[arp242]

I remember you used the API for some things @TigersWay, but don't specifically remember CSV exports off-hand, sorry.

Can you give me a little bit more information about what kind of results exactly? Is this something that could be added to GoatCounter, either in the dashboard proper or only via the API? I don't mind adding things if that would eliminate your need for the CSV exports, provided it's possible of course.

[TigersWay]

😄 Don't worry, I'm not going to be upset for you not remembering! To be honest the dashboard is already very useful, but at the time, the csv was the only entry possible for me, to be able to build stats (visits/visitors/bounce ratio/etc) for some set of urls, set per categories/countries or tags, etc. I may need to deep dive into the actual API to see if I can get the simple cumulated counters per pathname and day.

[TigersWay]

Just had a quick look on the different stats API...

• If I am not wrong the start date for the "hits" part is not working correctly
• And it seems I have no way to get a visitors/visits count

But I also can see I am the only who reacted on this thread.... So maybe not so important 😞

[stacybrock]

For a while, I attempted to use CSV exports out of a site hosted on goatcounter.com, but ran into issues where I could not download them consistently: https://github.com/arp242/goatcounter/issues/674

Since that issue was never resolved, I switched to self-hosting goatcounter to obtain direct access to the db, for the purposes of doing my own simple analyses beyond what's included in the goatcounter UI: unique visits per day, aggregate hits per path (or combinations of path + location + browser, etc) over various time ranges, etc.

I've already migrated to 2.5.0, and while the db schema change was inconvenient, I still have direct access to my apache logs so I can use other tools to meet my needs.

[naddika]

Collecting only "aggregate counts" fits better with GoatCounter's view of "we just count a few things rather than track people".

What's wrong with tracking people? How then will you prove that you track black and white people, men and women, LGBT and horses equally? Goats too.

Extract this into a feature. Let it be there for some time.

[redstreet]

FWWI, I would love for the CSV export to remain. Perhaps at least for the last day. i do my own aggregations from the CSV, though I guess I could give that up if all possible aggregations are available directly. I also use it to do debug. For example, to see what a call to the REST API resulted in.

[redstreet]

One more issue: currently, screen size is obtained from javascript. If I instead use the backend API to send goatcounter page counts, the user-agent might have "Mobile" or some such in some cases to distinguish between desktops and tablets. The aggregations don't use this info. Having a pageview export with the user-agent would help with this.

[arp242]

The solution in 2.6 (already live on goatcounter.com) where it no longer collects individual pageviews by default (but can be enabled in settings) is a decent middle-ground. So closing this.