Media & VideoPress: Calypso Storage Space Indicator Breaking on Atomic Sites, VideoPress videos counting against storage

[JoshuaGoode]

Atomic sites that upload videos to VideoPress are having those videos count against their storage limit in Calypso. Eventually, the user may reach "100%" of 200GB while their site's wp-content directory is only a few GB. This doesn't prevent the user from uploading new videos, however.

Additionally, videos deleted from media libraries contribute to storage_used_bytes and linger (please see p9o2xV-Qo-p2 ).

Background

Recently, @jmdodd has reviewed issues with videos and VideoPress storage -- especially on Business sites and Atomic Business sites. This can be reviewed internally at p9o2xV-Qo-p2

My encounter with the issue is documented internally at p9F6qB-57O-p2

Specifically, a user's site is only using 2.1 GB on Atomic but Calypso reports "100%" utilization of their 200GB storage.

GET v1.1/sites/$site/media-storage returned the following:

max_storage_bytes: 214748364800 storage_used_bytes: 504458228723

A test video was then uploaded and deleted which increased storage_used_bytes but didn't decrease it when the video was deleted. This result was expected based on @jmdodd's findings on p9o2xV-Qo-p2

Steps to reproduce

To see the discrepancy between Calypso and Atomic, you can upload a video to an Atomic site via VideoPress, delete it, upload more..etc and compare the site's storage utilization under Site Health. Then, compare it with Calypso's storage_used_bytes.

To reproduce the situation where Calypso claims the user is out of storage space while they are not:

1. Create or observe a WordPress.com Atomic site
2. Upload 200GB worth of media to the site. Make sure a portion of that is VideoPress videos.
3. Check the site's storage utilizing in WP-Admin > Tools > Site Health > Info > Directories and Sizes or another method.
4. Check Calypso's storage utilization indicator under My Site > Site > Media
5. Use the Dev Console and run GET v1.1/sites/$site/media-storage to check the reported storage utilization that Calypso is using.
6. Delete video and check again. Review status of the video on VideoPress.

What I expected

• VideoPress videos that the user deletes are subtracted from the site's storage utilization.
• But, moreover, VideoPress videos to not count against the storage

What happened instead

• Calypso is counting VideoPress videos against the site's storage utilization.

• VideoPress videos deleted from the site's media library are not completely deleted and still attribute to storage_used_bytes

Context / Source

Internal: p9o2xV-Qo-p2 p9F6qB-57O-p2 p1587666491353000-slack-atomic 2899651-zen

#user-report #dogfooding

[lancewillett]

Removed the "High" label as this isn't a critical issue with security or data loss.

Would you be willing to clarify what you think is the next action to fix this? Is it changing a label in the UI, or something else in the backend?

[JoshuaGoode]

Thanks for checking in @lancewillett

Potential next steps:

• Adjust whatever is behind storage_used_bytes to subtract VideoPress videos from total so that the UI reflects actual usage.
• Or, don't count VideoPress uploads towards the bytes calculation at all on Atomic sites.

Another item, which may need its own issue, is regarding deletions. When a video is deleted from a media library, it should be subtracted from storage_used_bytes and completely removed.

Related internal notes: p1588166113195000-slack-triage p9o2xV-Qo-p2 p1HpG7-7W4-p2

[jmdodd]

I think it would be better to remove space-used indicators from Calypso for Atomic sites, as these do not differentiate between file space used pre-transfer and VideoPress space used post-transfer and are very confusing to the customer. WordPress.com-on-Atomic sites have accurate space indicators for their file space on Atomic in their site health pages; Jetpack sites do not seem to have an indicator for the amount of space used in VideoPress.

[github-actions[bot]]

This issue is stale because it has been 180 days with no activity. You can keep the issue open by adding a comment. If you do, please provide additional context and explain why you’d like it to remain open. You can also close the issue yourself — if you do, please add a brief explanation and apply one of relevant issue close labels.

[Robertght]

This has also been discussed below and noticed the same behavior. The video files were missing the sizes: https://github.com/Automattic/wp-calypso/pull/55560#issuecomment-905743219

Should we consider this as a bug or as an enhancement for the future? cc @jmdodd

[JoshuaGoode]

Overall, this should be resolved in the context of the original issue.

Essentially Atomic sites use Atomic site meta now for their Calypso storage indicator, not Site Upload Space Quota, max_storage_bytes, storage_used_bytes. That's only used for VideoPress uploads on Atomic.

I'd need to test, though.

[Nic-Sevic]

Tried to test this and it appears to be working. When I upload videos via calypso, they go straight to VideoPress and I'm not seeing the reported site storage drop anywhere in the dash.

[cuemarie]

Hey folks; confusion around the 0,00 B file size for videos on AT sites has resulted in a new issue, #83512 , but following some similar threads in GH have poitned back here.

Is this issue related to / open in order to look into the 0,00 B file size for AT sites? Or would that best be tracked in it's own issue, like the one linked above?

[github-actions[bot]]

Support References

This comment is automatically generated. Please do not edit it.

• [ ] 2899651-zen

[davemart-in]

Related to https://github.com/Automattic/wp-calypso/issues/83663

[jordesign]

Assigning to Lego as part of their work untangling the Media Library.

[github-actions[bot]]

This issue is stale because 180 days have passed with no activity. If you would like this issue to remain open, please provide additional context, updated reproduction steps and/or screenshots.

If the issue is not updated in another month, it will be automatically closed.