software-submission
software-submission copied to clipboard
pyOpenSci
PsychoAnalyze
Submitting Author: Tyler Schlichenmeyer (@schlich)
All current maintainers: (@schlich)
Package Name: psychoanalyze
One-Line Description of Package: Interactive simulation & analysis for psychophysics.
Repository Link: https://github.com/psychoanalyze/psychoanalyze
Version submitted: 1.0.0a1
Editor: TBD
Reviewer 1: TBD
Reviewer 2: TBD
Archive: TBD
Version accepted: TBD
Date accepted (month/day/year): TBD
Code of Conduct & Commitment to Maintain Package
- [x] I agree to abide by pyOpenSci's Code of Conduct during the review process and in maintaining my package after should it be accepted.
- [x] I have read and will commit to package maintenance after the review as per the pyOpenSci Policies Guidelines.
Description
PsychoAnalyze is a data manipulation and visualization toolset to assist researchers in the broad domain of psychophysics and other areas of behavioral psychology and neuroscience. It provides methods and infrastructure for commonly performed analyses in the field, which includes curve fitting to the psychometric function as a model of sensory threshold. Powered by the psychoanalyze package, the project also provides a web-accessible interactive visualization dashboard built with the Dash framework and a self-hosted JupyterHub instance with executable example tutorials notebooks. Future iterations of work will incorporate more complex analyses, with a hierarchical Bayesian inferencing pipeline being the target for beta release.
Scope
-
Please indicate which category or categories. Check out our package scope page to learn more about our scope. (If you are unsure of which category you fit, we suggest you make a pre-submission inquiry):
- [ ] Data retrieval
- [] Data extraction
- [x] Data processing/munging
- [ ] Data deposition
- [] Data validation and testing
- [x] Data visualization[^1]
- [x] Workflow automation
- [ ] Citation management and bibliometrics
- [ ] Scientific software wrappers
- [x] Database interoperability
Domain Specific & Community Partnerships
- [ ] Geospatial
- [ ] Education
- [ ] Pangeo
Community Partnerships
If your package is associated with an existing community please check below:
[^1]: Please fill out a pre-submission inquiry before submitting a data visualization package.
-
For all submissions, explain how the and why the package falls under the categories you indicated above. In your explanation, please address the following points (briefly, 1-2 sentences for each):
- Data processing/munging: PsychoAnalyze transforms data from accepted schemas (e.g. data recorded in PsychoPy) into formats ready for model fitting (e.g. with PyMC). Takes the output of the results and processes them again for interactive visualization for domain-specific analyses.
- Data visualization: a fully interactive dashboard built with Dash will help researchers make appropriate online adjustments to experiments and interpret high-dimensional and longitudinal data more easily, enabling next-generation psychophysics analysis.
- Workflow automation: PsychoAnalyze is designed to be "plug-and-chug," as evidenced by the dashboard operation. However, under the hood the code base is highly modular, able to be customized without extensive software background. One-click environments are provided via Codespaces and JupyterHub.
- Database interoperability: Input/Output formats of Parquet, CSV, JSON, and SQL (DuckDB) are currently implemented with more to come, especially for formats common in laboratories such as PsychoPy format and MATLAB.
Technical checks
For details about the pyOpenSci packaging requirements, see our packaging guide. Confirm each of the following by checking the box. This package:
- [x] does not violate the Terms of Service of any service it interacts with.
- [x] uses an OSI approved license.
- [x] contains a README with instructions for installing the development version.
- [x] includes documentation with examples for all functions.
- [x] contains a tutorial with examples of its essential functions and uses.
- [x] has a test suite.
- [x] has continuous integration setup, such as GitHub Actions CircleCI, and/or others.
Publication Options
- [x] Do you wish to automatically submit to the Journal of Open Source Software? If so:
JOSS Checks
- [x] The package has an obvious research application according to JOSS's definition in their submission requirements. Be aware that completing the pyOpenSci review process does not guarantee acceptance to JOSS. Be sure to read their submission requirements (linked above) if you are interested in submitting to JOSS.
- [x] The package is not a "minor utility" as defined by JOSS's submission requirements: "Minor ‘utility’ packages, including ‘thin’ API clients, are not acceptable." pyOpenSci welcomes these packages under "Data Retrieval", but JOSS has slightly different criteria.
- [x] The package contains a
paper.mdmatching JOSS's requirements with a high-level description in the package root or ininst/. - [ ] The package is deposited in a long-term repository with the DOI:
Note: JOSS accepts our review as theirs. You will NOT need to go through another full review. JOSS will only review your paper.md file. Be sure to link to this pyOpenSci issue when a JOSS issue is opened for your package. Also be sure to tell the JOSS editor that this is a pyOpenSci reviewed package once you reach this step.
Are you OK with Reviewers Submitting Issues and/or pull requests to your Repo Directly?
This option will allow reviewers to open smaller issues that can then be linked to PR's rather than submitting a more dense text based review. It will also allow you to demonstrate addressing the issue via PR links.
- [x] Yes I am OK with reviewers submitting requested changes as issues to my repo. Reviewers will then link to the issues in their submitted review.
Confirm each of the following by checking the box.
- [x] I have read the author guide.
- [x] I expect to maintain this package for at least 2 years and can help find a replacement for the maintainer (team) if needed.
Please fill out our survey
- [x] Last but not least please fill out our pre-review survey. This helps us track submission and improve our peer review process. We will also ask our reviewers and editors to fill this out.
P.S. Have feedback/comments about our review process? Leave a comment here
Editor and Review Templates
The editor template can be found here.
The review template can be found here.
Editor in Chief checks
Hi @schlich! Welcome to pyOpenSci.
I can tell you have put a ton of work into PsychoAnalyze. Everything you have so far looks great! Without bringing my own bias into the review: I've done some work in psychometrics and I've wanted there to be a Python package developed like this for some time.
That said, there's some things we need you to address before we start the review.
Specifically, we need much more detail in the documentation.
Below I have put in the Editor in Chief checks.
You will see you have passed almost all of them with flying colors!
Except for the checks about documentation.
I have provided more detail about what we need in my comments following the checklist.
Please let me know if those are clear, and please check our Python packaging guide for more information.
Once we get those things addressed, I will be more than happy to find an editor and reviewers to help you bring PsychoAnalyze to the next level.
- [x] Installation The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (e.g. conda-forge, bioconda).
- [x] The package imports properly into a standard Python environment
import package-name. - [x] Fit The package meets criteria for fit and overlap.
- [ ] Documentation The package has sufficient online documentation to allow us to evaluate package function and scope without installing the package. This includes:
- [x] User-facing documentation that overviews how to install and start using the package.
- [ ] Short tutorials that help a user understand how to use the package and what it can do for them.
- [ ] API documentation (documentation for your code's functions, classes, methods and attributes): this includes clearly written docstrings with variables defined using a standard docstring format.
- [x] Core GitHub repository Files
- [x] README The package has a
README.mdfile with clear explanation of what the package does, instructions on how to install it, and a link to development instructions. - [x] Contributing File The package has a
CONTRIBUTING.mdfile that details how to install and contribute to the package. - [x] Code of Conduct The package has a
Code of Conductfile. - [x] License The package has an OSI approved license. NOTE: We prefer that you have development instructions in your documentation too.
- [x] README The package has a
- [x] Issue Submission Documentation All of the information is filled out in the
YAMLheader of the issue (located at the top of the issue template). - [x] Automated tests Package has a testing suite and is tested via a Continuous Integration service.
- [x] Repository The repository link resolves correctly.
- [x] Package overlap The package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci.
- [ ] Archive (JOSS only, may be post-review): The repository DOI resolves correctly.
- [ ] Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?
- [ ] Initial onboarding survey was filled out We appreciate each maintainer of the package filling out this survey individually. :raised_hands: Thank you authors in advance for setting aside five to ten minutes to do this. It truly helps our organization. :raised_hands:
Editor comments
Need to be addressed before we can review
- [ ] Documentation The package has sufficient online documentation to allow us to evaluate package function and scope without installing the package.
- There is almost no information about what psychometrics is and what features psychoanalyze offers. The very first things a potential new user should see are concrete examples--e.g., a psychometric curve fit with
psychoanalyze--and a list of features. See for example this readme for what I mean by concrete examples: https://github.com/garethjns/PsychometricCurveFitting, and the pandera docs for an example of lists of features: https://pandera.readthedocs.io/en/stable/. We need to have some initial description of what the package does, who it is for, and how it provides them what they need (features/functionality) before we start the review--doesn't have to be perfect but it has to be there. - [ ] API documentation (documentation for your code's functions, classes, methods and attributes): this includes clearly written docstrings with variables defined using a standard docstring format.
- As written, many of the functions do not have parameters and return values. We need to have these documented for all functions.
- None of the docstrings have examples. I would strongly recommend adding simple examples to your docstrings, and probably removing the suggestion that your users read your tests
- [ ] Short tutorials that help a user understand how to use the package and what it can do for them.
- The examples provided in notebooks are great! But there need to be vignettes in the static documentation that a user can read at any time, even if they don't have access to Binder, Codespaces, etc., for some reason. See the "Examples" section in the docs for pyOpenSci package pynteny: https://robaina.github.io/Pynteny/
- There is almost no information about what psychometrics is and what features psychoanalyze offers. The very first things a potential new user should see are concrete examples--e.g., a psychometric curve fit with
Suggestions, not required
- The package is available on pip, but there's no conda package -- it might be worth making a conda-forge package, e.g. with grayskull? IIUC, the package is pure Python, so this might be fairly painless
- Importing after install works, but nothing is imported into the namespace
>>> import psychoanalyze
>>> dir(psychoanalyze)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']
Is this intentional?
The package level docstring describes top-level modules here:
https://github.com/psychoanalyze/psychoanalyze/blob/0a28562dd9bbe267d62fe01608276e96d2d13127/psychoanalyze/init.py#L17
Why not import those inside psychoanalyze/__init__.py? To avoid making users import things directly, and to help them understand your intended API.
Hi @schlich just checking back. Please let me know if there's anything we can clear up about the initial checks.
Once those requirements are taken care of, I'll be happy to get started finding an editor and reviewers.
Hi Dave, thanks for the follow up, sorry for the delay! And thanks for the checks. To be honest, this submission was put out slightly earlier than anticipated due to graduation requirements for my PhD program. I'm focused on defending in the short term, but I anticipate being able to make these changes within the next few weeks-to-couple-months-ish. All great feedback, thank you! If this slight delay causes any issues with the process timeline, please let me know.
Very understood! I haven't forgot the, uh, fun of defending just yet.
Thank you for your reply -- let's keep this open for now. I will change the status to "on hold" and we'll give it a couple months.
Good luck with the PhD defense! If your maintainer chops are any indication, I think you'll be alright 🙂
hi everyone 👋🏻 i'm going through old / on hold issue where there has been no activity for over a year and closing them. It looks like this review did not fully begin. @schlich if you wish to kick off a review again in the future you are more than welcome to. I hope all is well!
