appdailysales icon indicating copy to clipboard operation
appdailysales copied to clipboard

Published 20 hours ago • verified publisher icon kirbyt

Use Autoingestion.class instead of screen scraping.

Open danieldickison opened this issue 13 years ago • 5 comments

The new Autoingestion Java program from Apple obviates the need to do scraping. appdailysales.py is still useful because it automates fetching multiple dates and unzipping with a nice interface for naming the output files etc.

This patch updates the script to use the Autoingestion program instead of screen scraping. Not sure if you would want to incorporate to master, but I think this is the way to go for the future.

danieldickison avatar Aug 24 '11 13:08 danieldickison

Nice! It didn't occur to me to replace the screen scraping with the new Autoingestion. I'll take a look at your patch (may take me a few days to get to it) and pull it in if it doesn't break anything. I would love to get away from the while screen scraping mess!

Thanks, -KIRBY

kirbyt avatar Aug 24 '11 14:08 kirbyt

Nice take on apples autoingestion feature. I'd be really interested to keep the screen scraping as the autoingestion is limited on the number of requests/time that you can do. I use to download reports for multiple apps, as I understand this can make problems.

philippb avatar Aug 30 '11 22:08 philippb

All-in-all, Apple's autoingest is fairly worthless. I guess it's better than the aforementioned screen scraping, but common, the biggest company in the world can't make some thing a hair more sophisticated than a sleazy command line that get's you a text file of all things? Worthless! It's a sleazy solution to something that's not very good to begin with. At the very least they could have thrown us a CSV file or something we could compile some statistics from.

resonant16 avatar Sep 02 '11 09:09 resonant16

@resonant16, I'm not sure what you mean. Downloading CSV files is exactly what Autoingest does and, wrapped in a script like appdailysales, it's quite handy. Definitely beats screen scraping (though I wasn't aware of rate limiting issues that philippb mentioned).

danieldickison avatar Sep 02 '11 17:09 danieldickison

Thanks for this, nice to have my appsales going again :)

subprocess.check_output was added in python v2.7. (https://github.com/pazz/alot/issues/16) The current ubuntu LTS (lucid) ships with python 2.6. Could this be updated to use something that is available in that old version?

I've installed python2.7 from https://launchpad.net/~fkrull/+archive/deadsnakes to get it going for now.

danzel avatar Oct 16 '11 21:10 danzel