Can't set the publication date of an issue before it is published

[AhemNason]

Describe the problem you would like to solve It is not possible to create a publication date for an issue before it is published. This creates problems when using QuickSubmit to upload back content because it is not possible to set a submission's publication date in QuickSubmit (see https://github.com/pkp/pkp-lib/issues/5418).

Describe the solution you'd like It should be possible to set the published date of an issue before it is published, in order to support uploading back content. When the issue is published, it's publication date should not be overridden and it should be used to set the publication date of any assigned submissions without a date.

Who is asking for this feature? Journal managers and assistants performing migrations of back content.

Additional information Just off the top I'd like to note that this came to me specifically in relation to the QuickSubmit plugin but I've determined that it doesn't actually matter how you've uploaded the file. There are, I'd argue, sensible use cases for additional date metadata within QuickSubmit (or elsewhere) that will not be overwritten or ignored on publication of an issue. Those include:

• submission date (historical... not within OJS)
• article publication date (which OJS and QuickSubmit does have, but is overwritten by issue-level publication date)
• issue publication date

So the problem is as follows.

Let's say I'm uploading an older issue of my journal to OJS from before my adoption of the platform. We'll say the issue was originally published on the first of January, 2010 (2010-01-01). I create a "future issue" in which to assign these articles I'll be uploading. Issue metadata for a future issue does not include a publication date with the volume, issue, and year metadata.

I hop over to QuickSubmit and start working on all my uploads. When I get to the portion of QuickSubmit where I flag the status of the work, I'll select "published" and I choose which issue it was published in. I select my future issue. A date field appears. I can input when the article was published. I do this for the whole issue I'm uploading so I can preview it and make any changes before making it public.

I publish it, and I see that all the dates I entered are gone! The dates have all been overwritten by today's date, the day that I made a back issue live. I've written a guide for folks on how to avoid this (https://www.notion.so/ahemnason/Uploading-Back-Issues-in-OJS-that-Respect-Publication-Dates-2cd0fa628fe94f0888ed630c5cf8b6d5), but TL;DR the only way to avoid it is to publish your empty back issue, edit it's publication date after you hit publish, and then add your articles to it.

This doesn't change the fact that your submission dates will be wrong, but at least your publication dates will be right.

I'd make a general recommendation that "submission date" and "publication date" be accessible fields for users to input values to as metadata when needed. There are a few down-stream effects of these publication date issues. If someone doesn't catch it, citations for this back-content will be way off. Also, registration agencies like Crossref charge different fees based on the age of the content uploaded. If a journal is working on a long backrun, this could be the difference between a $1 USD fee per article deposited and a $0.15 USD fee per article deposited.

The only way to remedy this mistake with the OJS UI for users who have done this is to, article-by-article, unpublish the work, add the publication date to the "issue" section of the production workflow, and republish it.

Oh, and I confirm all this behaviour in 3.3.0.6

[NateWr]

Thanks @AhemNason. I don't think it will be too difficult to do this. Here's a list of things that need to be considered:

• [ ] Add publication date field to future issues. It should be clear that this is not usually set. Maybe the field needs a warning like "Leave this empty and it will be set automatically when the issue is publlished."
• [ ] When an issue is published, the date is assigned to submissions automatically if they don't have a previous date assigned. In such cases, use the issue publication date should be used if it has been set.

[AhemNason]

For your first point I think you mean...

Add publication date field to ~back issues~ future issues.

[AhemNason]

Another option could be an option to push issue-level publication dates to all the articles within it. So, at the very least, if you have to fix a back issue, you can just update all the article-level metadata to match issue-level metadata.

[NateWr]

Another option could be an option to push issue-level publication dates to all the articles within it.

We should avoid modifying article metadata after it is published.

[ajnyga]

ojs: https://github.com/pkp/ojs/pull/3500

When an issue is published, the date is assigned to submissions automatically if they don't have a previous date assigned. In such cases, use the issue publication date should be used if it has been set.

I am not touching article pub dates here at all. Is this something we want to handle here as well?

[asmecher]

@NateWr, could you review this? (@ajnyga, Nate is away for a bit so it won't be immediate!)

[ajnyga]

Noticed that I did not add the editor.issues.datePublished.notPublished.description translation to the pr.

That is basically the "Leave this empty and it will be set automatically when the issue is published." notification mentioned above. Let me know if you want something else there.

[NateWr]

Thanks @ajnyga, I left a couple comments.

I am not touching article pub dates here at all. Is this something we want to handle here as well?

Can you try to reproduce the problem that @AhemNason described in the original issue? I had a quick look through the code and didn't see anything that sets the submission's datePublished based on the issue. But if you can reproduce it with your changes then we want to handle that too.

[ajnyga]

1. New issue, not published, set pubDate to 2022-09-01

2. New article with quicksubmit, choose the new issue, set article Pubdate to 2022-09-10

3. After adding the article go to see article details -> The article pubdate is NULL! I also checked the db for this, publication date_published is NULL The publication status is scheduled like it should be

4. Publish the issue -> The article pubdate is set to current date!

So yes, something fishy going on here and these changes do not fix it.

[ajnyga]

When the issue is published, this is called: https://github.com/pkp/ojs/blob/main/classes/controllers/grid/issues/IssueGridHandler.php#L592

With the set pubDate being just NULL in these cases, I guess you just end up setting it to the current date.

[NateWr]

It seems like what's outlined in this comment is still relevant: https://github.com/pkp/pkp-lib/issues/7165#issuecomment-871211666. Does something more need to be done?

[ajnyga]

I think quicksubmit too needs fixing. It seems that it does not respect the publication date given by the user at all

[ajnyga]

I think we need to think about the whole palette before doing any more changes.

[ajnyga]

This is probably the source of the problem: https://github.com/pkp/ojs/blob/main/classes/publication/Repository.php#L157-L166

  protected function setStatusOnPublish(Publication $publication)
   {
       // A publication may be scheduled in a future issue. In such cases,
       // the `datePublished` should remain empty and the status should be set to
       // scheduled.
       //
       // If there is no assigned issue, the journal may be using a continuous
       // publishing model in which articles are published right away.
       $issue = Repo::issue()->get($publication->getData('issueId'));
       if ($issue && !$issue->getData('published')) {
           $publication->setData('datePublished', null);
           $publication->setData('status', Submission::STATUS_SCHEDULED);
       } else {
           $publication->setData('status', Submission::STATUS_PUBLISHED);
           if (!$publication->getData('datePublished')) {
               $publication->setData('datePublished', Core::getCurrentDate());
           }
       }
   }

If the issue is not published when you add new articles with quicksubmit, that will set the article pubdate to NULL.

[NateWr]

:thinking: if the issue is not published, the quicksubmit plugin shouldn't be trying to publish a publication, right?

[ajnyga]

Quicksubmit has two options:

1. This is not published, meaning that the submission in sent to production stage
2. This is published, meaning that you need to choose an issue (future or published) and set a pubDate for the article

If you choose 2, then this happens https://github.com/pkp/quickSubmit/blob/main/QuickSubmitForm.inc.php#L378

[ajnyga]

I would be inclined to do something like this:

        // Case 1: issue not published and a future issue - pubDate for articles set when issue published
        if ($issue && !$issue->getData('published') && !$issue->getData('datePublished')) {
            $publication->setData('datePublished', null);
            $publication->setData('status', Submission::STATUS_SCHEDULED);
        } 

        // Case 2: issue not published and a backissue - pubDate for articles set according to the issue pubDate if no article pubDate exists
        else if ($issue && !$issue->getData('published') && $issue->getData('datePublished')) {
            if (!$publication->getData('datePublished')) {
                $publication->setData('datePublished', $issue->getData('datePublished'));
            }
            $publication->setData('status', Submission::STATUS_SCHEDULED);
        }

        // Case 3: issue is published already - pubDate of articles set according to the current date (continuous publishing)
        else {
            $publication->setData('status', Submission::STATUS_PUBLISHED);
            if (!$publication->getData('datePublished')) {
                $publication->setData('datePublished', Core::getCurrentDate());
            }
        }

[ajnyga]

So publication::publish and setStatusOnPublish is in most cases called twice for each submission? First when you schedule the submission and again when you publish the issue?

So with the old code https://github.com/pkp/ojs/blob/main/classes/publication/Repository.php#L157-L166

When you scheduled the submission it would always set datePublished to NULL. Both with quicksubmit and with the usual workflow. Because when you schedule, you never have a published issue.

Then when you went on to publish the issue, you would always get the articles dates set to the current date because datePublished in all cases is NULL https://github.com/pkp/ojs/blob/main/classes/publication/Repository.php#L163-L165

Only if the issue is already published, the datePublished given to the article is respected.

[NateWr]

I think this is a holdover from the way that things worked in stable-3_3_0. There the publication date was set in lib/pkp:

https://github.com/pkp/pkp-lib/blob/stable-3_3_0/classes/services/PKPPublicationService.inc.php#L555-L557

Then a hook was called:

https://github.com/pkp/pkp-lib/blob/stable-3_3_0/classes/services/PKPPublicationService.inc.php#L581

And in OJS, a callback was registered to "reset" the date published based on the application logic (issues):

https://github.com/pkp/ojs/blob/stable-3_3_0/classes/services/PublicationService.inc.php#L252-L256

I think your proposed logic is broadly correct. The one difference is that we shouldn't call $publication->setData('datePublished', null). We should never overwrite a datePublished value if one has been manually entered in for the publication. A use-case for this is importing back issues from continuous publishing, where each article has its own published date but will be put into a future issue before the issue is published all at once.

So I would express the desired logic in pseudo-code like this:

if (!issue) {
    publication->setPublishedDate(now)
} elseif (issue && issue-published) {
    publication->setPublishedDate(issue-published-date)
} else {
    // leave the publication alone
}

If it is scheduled in a future issue, the datePublished may exist or not. Then when the issue is published, it should set the published date of its articles only for articles that don't already have a date.

[ajnyga]

but this would mean that if I have an issue already published and add a new article with a specific datePublished stamp, that will be changed to the issue-published-date?

[NateWr]

You're right. In that case, it should only use the issue published date if the publication doesn't have one yet:

if (!issue) {
    publication->setPublishedDate(now)
} elseif (issue && issue-published && !publication-published-date) {
    publication->setPublishedDate(issue-published-date)
} else {
    // leave the publication alone
}

I think that will cover all of these workflows:

1. Schedule publication without date in unpublished issue -> publish issue = publication has issue date
2. Schedule publication with date in unpublished issue -> publish issue = publication has publication date
3. Schedule publication without date in published issue = publication has issue date
4. Schedule publication with date in published issue = publication has publication date

Is that everything? I haven't accounted for QuickSubmit, but I'm guessing it results in the same flow there?

[ajnyga]

Thinking that it might cause problems if the empty publication-published-date is not set to null. I mean it might cause a db error because an empty string would not be valid for a datestamp field?

[ajnyga]

Is that everything? I haven't accounted for QuickSubmit, but I'm guessing it results in the same flow there?

Quicksubmit should be covered with that. If you choose that this is a published article, you are forced to choose an issue and add a datePublished. So it would go to the else loop.

But with the regular workflow, I wonder what happens when you schedule without a date to an unpublished issue. That would also go to the else loop and probably will throw an error because the date has to be valid or null.

if (!issue) {
    publication->setPublishedDate(now)
} elseif (issue && issue-published && !publication-published-date) {
    publication->setPublishedDate(issue-published-date)
} else {
    if (!publication-published-date) {
        publication->setPublishedDate(null)
    }
}

But I am not sure how to change of publication status will fit here?

[ajnyga]

This is edited from my initial suggestion:

      // Case 1: Issue not published yet, publication is scheduled
       if ($issue && !$issue->getData('published')) {

           // If no publication datePublished
           if (!$publication->getData('datePublished')) {
               // Use issue date published if available
               if $issue->getData('datePublished'){
                   $publication->setData('datePublished', $issue->getData('datePublished'));
               }
               // if not, set to null
               else {
                   $publication->setData('datePublished', null);
               }
           }

           $publication->setData('status', Submission::STATUS_SCHEDULED);
       }


       // Case 2: issue is published or no issue, publication is published
       else {
           // If no publication datePublished available use current date
           if (!$publication->getData('datePublished')) {
               $publication->setData('datePublished', Core::getCurrentDate());
           }
           $publication->setData('status', Submission::STATUS_PUBLISHED);
       }

[NateWr]

That would also go to the else loop and probably will throw an error because the date has to be valid or null.

This shouldn't be handled in the Repo::publication()->publish() step at all. In other words, the publication should have a valid date_published value already. The publish() function should only update this value to set it to a valid date. It should not have to set it to null during the publish step.

[ajnyga]

So are you thinking that it should be set to null in Repo::publication()->add() and Repo::publication()->version()

[NateWr]

I'm not sure I understand what you mean. Can you describe the circumstances under which datePublished should be set to null?

My understanding is that the datePublished value is set through a PUT request to the API's /submissions/<id>/publications/<id> endpoint. The only time we should set it to null is when someone makes a request to that endpoint with an empty value for the datePublished prop.

[ajnyga]

I am sure I saw fatal error regarding the missing NULL value at some point, but can not reproduce that anymore. So it was probably some other reason that produced it.

I tested the code you suggested with minor changes. Note that I do not think it is possible at the moment to publish something without an issue (case 1). Or did I misunderstand something?

        // A publication may be published without an issue. Use current date if no
        // custom `datePublished` is set. Status is set to published.

        // A publication may be published in a published issue. 
        // In such cases the `datePublished` should be what the user has entered 
        // or fall back to the issue published date. Status should be set to published.

        // A publication may also be scheduled in a future issue. In such cases,
        // the the status should be set to scheduled and `datePublished` 
        // should remain what it is.

        $issue = Repo::issue()->get($publication->getData('issueId'));

        if (!$issue) {
            if (!$publication->getData('datePublished')){
                $publication->setData('datePublished', Core::getCurrentDate());
            }
            $publication->setData('status', Submission::STATUS_PUBLISHED);
        } elseif ($issue && $issue->getData('published')) {
            if (!$publication->getData('datePublished')){
                $publication->setData('datePublished', $issue->getData('datePublished'));
            }
            $publication->setData('status', Submission::STATUS_PUBLISHED);
        } else {
            $publication->setData('status', Submission::STATUS_SCHEDULED);
        }

But anyway, I tested all these scenarios with this code

• New submission created through the regular workflow. Publish with custom article pubdate in an already published issue -> article has custom article pubdate
• New submission created through the regular workflow. Publish without preset article pubdate in an already published issue -> article has issue pubdate
• New submission created through the regular workflow. Schedule with custom article pubdate in an unpublished issue with a custom issue pubdate -> publish issue > article has custom article pubdate
• New submission created through the regular workflow. Schedule without preset article pubdate in an unpublished issue with a custom issue pubdate -> publish issue > article has issue pubdate
• New submission created through the regular workflow. Schedule without custom article pubdate in an unpublished issue without a custom issue pubdate -> publish issue > article has current pubdate
• Quicksubmit submission. Schedule with custom article pubdate in an unpublished issue with a custom issue pubdate -> publish issue > article has custom article pubdate
• Quicksubmit submission. Schedule with custom article pubdate in an unpublished issue without a custom issue pubdate -> publish issue > article has custom article pubdate

[ajnyga]

pr: https://github.com/pkp/ojs/pull/3500

[ajnyga]

OJS: https://github.com/pkp/ojs/pull/3500 pkp-lib: https://github.com/pkp/pkp-lib/pull/8505

pr's to allow setting issue published date when creating a new issue. Also includes a small tweak to DatePicker widget allowing easier access to older dates (see month & year menu option https://jqueryui.com/datepicker/#dropdown-month-year)