admidio icon indicating copy to clipboard operation
admidio copied to clipboard

Admidio Stats

Open ximex opened this issue 9 years ago • 15 comments

It would be nice if we could get some statistics about the in use admidio instances like wordpress did: https://wordpress.org/about/stats/

So we can see if we could drop a old PHP, MySQL, ... version

Must have is an opt-out checkbox at the installation process and in the admin-settings (privacy reasons)

this infos should only be send to us (admidio.org) if anything changed (PHP, DB, Admdio) and sould have a anonymous identifier (like cyanogenmod did)

this infos should be shown on a statistics site at admidio.org

ximex avatar May 19 '15 18:05 ximex

Perhaps this is also interesting. http://w3techs.com/technologies/details/pl-php/5/all

Bullnados avatar May 19 '15 19:05 Bullnados

Also stats from phpinfo() would be interesting to see what functions, modules are supported.

Examples:

  • mb_strtolower (fallback check with admStrToLower())
  • mb_strtoupper (fallback check with admStrToUpper())

one more usefull function: get_loaded_extensions (https://php.net/manual/en/function.get-loaded-extensions.php)

ximex avatar May 21 '15 08:05 ximex

Possible Lib: https://github.com/gabrielbull/php-browser

ximex avatar Aug 18 '15 07:08 ximex

repository link of the above lib change to: https://github.com/sinergi/php-browser-detector

ximex avatar Sep 08 '15 07:09 ximex

Things we need:

  • create hash or uuid to get an unique identifier for an installation (url or installation based)
  • collect the data below at every admidio update
  • send the data to admidio.org (https) at every admidio update
  • Get https for admidio.org
  • Create API to send data to/get data from admidio.org
  • Create storage backend on admidio.org
  • Create statistics page on admidio.org
  • data to send:
    • identifier
    • analysis date/time
    • Admidio version
    • PHP version
    • Database type and version
    • PHP modules/extensions available
    • Plugins installed and their versions

ximex avatar Sep 14 '16 18:09 ximex

👍

Fasse avatar Sep 14 '16 18:09 Fasse

first fast impl for data collection:

<?php
/**
 ***********************************************************************************************
 * @copyright 2004-2016 The Admidio Team
 * @see http://www.admidio.org/
 * @license https://www.gnu.org/licenses/gpl-2.0.html GNU General Public License v2.0 only
 ***********************************************************************************************
 */

/**
 * @class Statistics
 * @brief Collect and manage all the data for the Admidio statistics
 */
class Statistics
{
    /**
     * @param string $domain
     * @return string
     */
    public static function createUUID($domain = 'localhost')
    {
        $dateTime = Statistics::getDateTime();
        $random = PasswordHashing::genRandomPassword();

        return sha1($dateTime . $domain . $random);
    }

    /**
     * @return array
     */
    public static function getInfoData()
    {
        return array(
            'uuid' => Statistics::getUUID(),
            'timestamp' => Statistics::getDateTime(),
            'php' => array(
                'version' => Statistics::getPHPVersion(),
                'extensions' => Statistics::getPHPExtensions()
            ),
            'database' => array(
                'available' => Statistics::getAvailableDatabaseTypes(),
                'type' => Statistics::getDatabaseType(),
                'version' => Statistics::getDatabaseVersion()
            ),
            'admidio' => array(
                'version' => Statistics::getAdmidioVersion(),
                'plugins' => Statistics::getAdmidioPlugins()
            )
        );
    }

    /**
     * @return string
     */
    private static function getUUID()
    {
        global $gAdmidioUUID;

        if (!$gAdmidioUUID)
        {
            $gAdmidioUUID = Statistics::createUUID();
        }

        return $gAdmidioUUID;
    }

    /**
     * @return string
     */
    private static function getDateTime()
    {
        $dateTime = new DateTime('now', new DateTimeZone('UTC'));

        return $dateTime->format('Y-m-d H:i:s');
    }

    /**
     * @return string
     */
    private static function getPHPVersion()
    {
        return PHP_VERSION;
    }

    /**
     * @return string[]
     */
    private static function getPHPExtensions()
    {
        $extensions = get_loaded_extensions();
        asort($extensions);

        $extensionsVersions = array();
        foreach ($extensions as $extension)
        {
            $extensionsVersions[$extension] = phpversion($extension);
        }

        return $extensionsVersions;
    }

    /**
     * @return string[]
     */
    private static function getAvailableDatabaseTypes()
    {
        return PDO::getAvailableDrivers();
    }

    /**
     * @return string
     */
    private static function getDatabaseType()
    {
        global $gDbType;

        return $gDbType;
    }

    /**
     * @return string
     */
    private static function getDatabaseVersion()
    {
        global $gDb;

        $versionStatement = $gDb->query('SELECT version()');

        return $versionStatement->fetchColumn();
    }

    /**
     * @return string
     */
    private static function getAdmidioVersion()
    {
        return ADMIDIO_VERSION_TEXT;
    }

    /**
     * @return string[]
     */
    private static function getAdmidioPlugins()
    {
        $plugins = array();

        $pluginsFolder = SERVER_PATH . DIRECTORY_SEPARATOR . 'adm_plugins';

        $dir = scandir($pluginsFolder);
        foreach ($dir as $entry)
        {
            $pluginFolder = $pluginsFolder . DIRECTORY_SEPARATOR . $entry;
            $pluginFile = $pluginFolder . DIRECTORY_SEPARATOR . $entry . '.php';

            if (!in_array($entry, array('.', '..'), true) && is_dir($pluginFolder) && is_file($pluginFile))
            {
                $handle = fopen($pluginFile, 'r');
                while (($line = fgets($handle)) !== false)
                {
                    preg_match('/^ \* Version (\d+\.\d+\.\d+.*$)/', $line, $matches);

                    if (count($matches) === 2)
                    {
                        $plugins[$entry] = $matches[1];
                    }
                }
                fclose($handle);
            }
        }

        return array($plugins);
    }
}

my results in json

{
    "uuid": "c87c629ea7d2176c6c4312ea39834c7a9b8df06e",
    "timestamp": "2016-09-14 19:59:22",
    "php": {
        "version": "5.6.21-1+donate.sury.org~trusty+4",
        "extensions": {
            "Core": "5.6.21-1+donate.sury.org~trusty+4",
            "PDO": "1.0.4dev",
            "Phar": "2.0.2",
            "Reflection": "$Id: fbcf7a77ca8e3d4cd7501de8025235b947b8240f $",
            "SPL": "0.2",
            "SimpleXML": "0.1",
            "Zend OPcache": "7.0.6-devFE",
            "apache2handler": false,
            "apc": "4.0.10",
            "apcu": "4.0.10",
            "bcmath": false,
            "bz2": false,
            "calendar": false,
            "ctype": false,
            "curl": false,
            "date": "5.6.21-1+donate.sury.org~trusty+4",
            "dba": false,
            "dom": "20031129",
            "ereg": false,
            "exif": "1.4 $Id: db007ca2e6d0b4513ae77990972997fad8bfe9c9 $",
            "fileinfo": "1.0.5",
            "filter": "0.11.0",
            "ftp": false,
            "gd": false,
            "gettext": false,
            "hash": "1.0",
            "iconv": false,
            "imagick": "3.3.0",
            "intl": "1.1.0",
            "json": "1.3.9",
            "ldap": false,
            "libxml": false,
            "mbstring": false,
            "mcrypt": false,
            "mhash": false,
            "mysql": "1.0",
            "mysqli": "0.1",
            "mysqlnd": "mysqlnd 5.0.11-dev - 20120503 - $Id: 76b08b24596e12d4553bd41fc93cccd5bac2fe7a $",
            "openssl": false,
            "pcre": false,
            "pdo_mysql": "1.0.2",
            "pdo_pgsql": "1.0.2",
            "pdo_sqlite": "1.0.1",
            "pgsql": false,
            "posix": false,
            "readline": "5.6.21-1+donate.sury.org~trusty+4",
            "session": false,
            "shmop": false,
            "soap": false,
            "sockets": false,
            "sqlite3": "0.7-dev",
            "standard": "5.6.21-1+donate.sury.org~trusty+4",
            "sysvmsg": false,
            "sysvsem": false,
            "sysvshm": false,
            "tokenizer": "0.1",
            "wddx": false,
            "xml": false,
            "xmlreader": "0.1",
            "xmlwriter": "0.1",
            "zip": "1.12.5",
            "zlib": "2.0"
        }
    },
    "database": {
        "available": [
            "mysql",
            "pgsql",
            "sqlite"
        ],
        "type": "mysql",
        "version": "10.1.17-MariaDB-1~trusty"
    },
    "admidio": {
        "version": "3.2.0 Beta 1",
        "plugins": [
            {
                "birthday": "1.9.0",
                "calendar": "2.1.0",
                "login_form": "1.8.0",
                "random_photo": "1.8.0",
                "sidebar_announcements": "1.9.0",
                "sidebar_dates": "1.7.1",
                "sidebar_downloads": "1.5.1",
                "sidebar_online": "1.6.0"
            }
        ]
    }
}

ximex avatar Sep 14 '16 20:09 ximex

In my opinion we should not collect as many data as we can. We should collect as many aus we really Need. So we should Filter the php data.

Fasse avatar Sep 14 '16 20:09 Fasse

if i want as many as i can it would be more ;-)

Maybe add an option to set how many informations are send.

  • 0: no data
  • 1: admidio version, php version, database type/version
  • 2: all data without version of admidio plugins and version of php extensions
  • 3: all data

ximex avatar Sep 15 '16 08:09 ximex

I would appreciate the option to decide by myself as admidio admin which information will be sent. Maybe the default setting could set to "all data" with the information for which purpose the data will be.

papierkorb27 avatar Sep 15 '16 15:09 papierkorb27

If you want to decide yourself than the default setting should be 1 because otherwise you may have already send all data to us.

But I don't like these options. There should only be send data or don't send data. It would not be useful if we have different versions of informations. Than Admin must decide another question. I like the initial idea. Maybe a flag localhost or not because data of localhost is not as important as web statistics.

It would not build up trust if we send 40 or more paramters to our server and Admidio needs trust of the people who use it.

Fasse avatar Sep 15 '16 18:09 Fasse

my vote for an ON/OFF System... the code ist good, but I think the PHP extensions should not be send (this is too much information that we don't realy need) the rest is OK from my point of view.

sirnone avatar Sep 16 '16 06:09 sirnone

@sirnone i want the extensions to get informations if we could remove things like this (or why not) https://github.com/Admidio/admidio/blob/master/adm_program/system/string.php#L12-L43

ximex avatar Sep 18 '16 12:09 ximex

@Fasse this is a must have (for me) for v3.2.

ximex avatar Sep 30 '16 21:09 ximex

Because of the release within the next few weeks we should add and test this feature within version 3.3

Fasse avatar Oct 30 '16 08:10 Fasse