Store data globally, or somewhere configurable

[eyeseast]

If I understand what's happening, data lives and is updated here: https://github.com/datadesk/cpi/blob/master/cpi/data.csv

Two things make my queasy about this (with the caveat that I haven't used this in a project yet):

• it's changing the codebase in flight, which is scary
• if I have multiple installs, they could get out of sync, or I just end up with lots of copies of the same data

One way you could avoid that is having a global, or configurable, data cache. It might be $HOME/.python-cpi/data.csv by default, with the option to configure if you needed an isolated copy somewhere. The library could pre-populate the cache, or fall back on what's included in the codebase, or warn if the data is stale.

[palewire]

I'm open to a more sophisticated solution. Just tried to keep it simple for v0.0.1.

One complicating factor is this library will need to update every month to stay in sync with the latest CPI values. You can see the StaleDataWarning I've drafted for now.

I doubt this is the first library to grapple with this kind of problem. So I'd be interested to learn how other developers have tackled the challenge. And if we can come up with a sturdy replacement for the stupid simple system I have here, that's great.

[eyeseast]

Totally understand needing to get a version out the door, especially on deadline.

I'll look around and open a PR if I find a promising pattern.

[salah93]

How about downloading the data to the global data cache after install The only catch is that it would run this before dependencies install (namely requests) so you would have to use urllib

from setuptools import setup                                                    
from setuptools.command.install import install                                  
                                                                                
from cpi.download import Downloader                                             
                                                                                
                                                                                
class PostInstallCommand(install):                                              
    """Post-installation for installation mode."""                              
    def run(self):                                                              
        Downloader().update()                                                   
        install.run(self)                                                       
                                                                                                                                                    
                                                                                
setup(                                                                          
    name='cpi',                                                                 
    version='0.0.6',                                                            
    description="Quickly adjust U.S. dollars for inflation using the Consumer Price Index (CPI)",
    author='Ben Welsh',                                                                          
    author_email='[email protected]',                                         
    url='http://www.github.com/datadesk/cpi',                                   
    license="MIT",                                                              
    packages=("cpi",),                                                          
    include_package_data=True,                                                  
    zip_safe=False,  # because we're including static files                     
    install_requires=("requests",),                                             
    cmdclass={                                                                  
    'install': PostInstallCommand,                                              
    },                                                                          
    classifiers=[                                                               
        'Development Status :: 5 - Production/Stable',                          
        'Programming Language :: Python',                                       
        'Programming Language :: Python :: 2.7',                                
        'Programming Language :: Python :: 3.4',                                
        'Programming Language :: Python :: 3.5',                                
        'Programming Language :: Python :: 3.6',                                
        'License :: OSI Approved :: MIT License',                               
    ],                                                                          
)                                          

[palewire]

Not a bad idea. Are there other libraries that do this?

[matthew-piziak]

One can't run cpi.update() when using this library with Nix, because the library gets puts into a readonly location for the reproducibility guarantees of that framework. Allowing the user to customize the download directory would allow them to specify a writeable destination.