asset-share-commons icon indicating copy to clipboard operation
asset-share-commons copied to clipboard
verified publisher icon adobe

Asset Download operation fails for assets greater than 1 GB from Asset Share Page

Open rajat1020 opened this issue 6 years ago • 9 comments

User is not able to download large assets >1 GB from Asset Share Page. Getting 504 error.

Environment

  • AEM Version: AEM 6.3
  • Asset Share Commons Version: 1.8.0
  • Author, Publish or both: Both (Asset Share Common Author and Publish)

rajat1020 avatar Nov 22 '19 09:11 rajat1020

Can you please share any error.log statement related to this request?

joerghoh avatar Nov 22 '19 11:11 joerghoh 

java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
- locked <0x000000044b990048> (a java.lang.Object)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
- locked <0x000000044b990af0> (a sun.security.ssl.AppInputStream)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.read(SessionInputBufferImpl.java:198)
at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:176)
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135)
at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:72)
at com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:151)
at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:72)
at com.amazonaws.services.s3.model.S3ObjectInputStream.read(S3ObjectInputStream.java:155)
at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:72)
at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:72)
at com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:151)
at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:72)
at com.amazonaws.util.LengthCheckInputStream.read(LengthCheckInputStream.java:108)
at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:72)
at com.amazonaws.services.s3.model.S3ObjectInputStream.read(S3ObjectInputStream.java:155)
at com.amazonaws.services.s3.model.S3ObjectInputStream.read(S3ObjectInputStream.java:147)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2146)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:2102)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2123)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:2078)
at org.apache.commons.io.FileUtils.copyToFile(FileUtils.java:1554)
at org.apache.commons.io.FileUtils.copyInputStreamToFile(FileUtils.java:1528)
at org.apache.jackrabbit.oak.commons.FileIOUtils.copyInputStreamToFile(FileIOUtils.java:304)
at org.apache.jackrabbit.oak.plugins.blob.FileCache$3.load(FileCache.java:138)
at org.apache.jackrabbit.oak.plugins.blob.FileCache$3.load(FileCache.java:126)
at org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:1043)
- locked <0x00000004c7141b90> (a org.apache.jackrabbit.oak.cache.CacheLIRS$Segment)
at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:318)
at org.apache.jackrabbit.oak.plugins.blob.FileCache.get(FileCache.java:258)
at org.apache.jackrabbit.oak.plugins.blob.CompositeDataStoreCache.get(CompositeDataStoreCache.java:109)
at org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore$FileCacheDataRecord.getStream(AbstractSharedCachingDataStore.java:314)
at org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore.getStream(DataStoreBlobStore.java:583)
at org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore.getInputStream(DataStoreBlobStore.java:342)
at org.apache.jackrabbit.oak.plugins.blob.BlobStoreBlob.getNewStream(BlobStoreBlob.java:47)
at org.apache.jackrabbit.oak.segment.SegmentBlob.getNewStream(SegmentBlob.java:252)
at org.apache.jackrabbit.oak.segment.SegmentBlob.getNewStream(SegmentBlob.java:87)
at org.apache.jackrabbit.oak.plugins.value.BinaryImpl.getStream(BinaryImpl.java:53)
at org.apache.sling.jcr.resource.internal.helper.LazyInputStream.getStream(LazyInputStream.java:106)
at org.apache.sling.jcr.resource.internal.helper.LazyInputStream.read(LazyInputStream.java:65)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2146)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:2102)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2123)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:2078)
at com.mondelez.assetshare.download.servlet.MdlzAssetDownloadServlet.addZipEntry(MdlzAssetDownloadServlet.java:635)
at com.mondelez.assetshare.download.servlet.MdlzAssetDownloadServlet.buildOutputStream(MdlzAssetDownloadServlet.java:225)
at com.mondelez.assetshare.download.servlet.MdlzAssetDownloadServlet.doGet(MdlzAssetDownloadServlet.java:173)
at com.mondelez.assetshare.download.servlet.MdlzAssetDownloadServlet.doPost(MdlzAssetDownloadServlet.java:654)
at org.apache.sling.api.servlets.SlingAllMethodsServlet.mayService(SlingAllMethodsServlet.java:146)
at org.apache.sling.api.servlets.SlingSafeMethodsServlet.service(SlingSafeMethodsServlet.java:342)
at org.apache.sling.api.servlets.SlingSafeMethodsServlet.service(SlingSafeMethodsServlet.java:374)
at org.apache.sling.engine.impl.request.RequestData.service(RequestData.java:552)
at org.apache.sling.engine.impl.filter.SlingComponentFilterChain.render(SlingComponentFilterChain.java:44)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:77)
at com.day.cq.wcm.core.impl.WCMDebugFilter.doFilter(WCMDebugFilter.java:156)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.day.cq.wcm.core.impl.WCMComponentFilter.filterRootInclude(WCMComponentFilter.java:375)
at com.day.cq.wcm.core.impl.WCMComponentFilter.doFilter(WCMComponentFilter.java:190)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.day.cq.wcm.core.impl.PageLockFilter.doFilter(PageLockFilter.java:91)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.day.cq.personalization.impl.TargetComponentFilter.doFilter(TargetComponentFilter.java:96)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at org.apache.sling.engine.impl.SlingRequestProcessorImpl.processComponent(SlingRequestProcessorImpl.java:282)
at org.apache.sling.engine.impl.filter.RequestSlingFilterChain.render(RequestSlingFilterChain.java:49)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:77)
at com.day.cq.dam.core.impl.assetlinkshare.AdhocAssetShareAuthHandler.doFilter(AdhocAssetShareAuthHandler.java:437)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.day.cq.dam.core.impl.servlet.ActivityRecordHandler.doFilter(ActivityRecordHandler.java:154)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.adobe.cq.dam.webdav.impl.io.DamWebdavRequestFilter.doFilter(DamWebdavRequestFilter.java:132)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.adobe.cq.social.ugcbase.security.impl.SaferSlingPostServlet.doFilter(SaferSlingPostServlet.java:126)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.day.cq.dam.core.impl.servlet.DamContentDispositionFilter.doFilter(DamContentDispositionFilter.java:104)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.adobe.cq.dam.cfm.impl.servlets.CompatFilter.doFilter(CompatFilter.java:81)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.adobe.granite.rest.impl.servlet.ApiResourceFilter.doFilter(ApiResourceFilter.java:70)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.adobe.granite.requests.logging.impl.RequestLoggerImpl.doFilter(RequestLoggerImpl.java:134)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.adobe.granite.rest.assets.impl.AssetContentDispositionFilter.doFilter(AssetContentDispositionFilter.java:96)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.adobe.granite.csrf.impl.CSRFFilter.doFilter(CSRFFilter.java:217)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at org.apache.sling.security.impl.ContentDispositionFilter.doFilter(ContentDispositionFilter.java:180)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.day.cq.wcm.core.impl.AuthoringUIModeServiceImpl.doFilter(AuthoringUIModeServiceImpl.java:292)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.day.cq.wcm.mobile.core.impl.redirect.RedirectFilter.doFilter(RedirectFilter.java:248)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.mdlz.damui.core.filters.LoggingFilter.doFilter(LoggingFilter.java:56)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at org.apache.sling.engine.impl.debug.RequestProgressTrackerLogFilter.doFilter(RequestProgressTrackerLogFilter.java:107)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.adobe.cq.social.commons.cors.CORSAuthenticationFilter.doFilter(CORSAuthenticationFilter.java:91)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.day.cq.wcm.foundation.forms.FormsHandlingServletHelper.handleFilter(FormsHandlingServletHelper.java:221)
at com.day.cq.wcm.foundation.forms.impl.FormsHandlingServlet.doFilter(FormsHandlingServlet.java:138)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.adobe.granite.optout.impl.OptOutFilter.doFilter(OptOutFilter.java:76)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.day.cq.wcm.foundation.forms.FormsHandlingServletHelper.handleFilter(FormsHandlingServletHelper.java:221)
at com.adobe.cq.wcm.core.components.internal.servlets.CoreFormHandlingServlet.doFilter(CoreFormHandlingServlet.java:125)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.day.cq.wcm.core.impl.WCMRequestFilter.doFilter(WCMRequestFilter.java:90)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.adobe.cq.mcm.campaign.servlets.CampaignCopyTracker.doFilter(CampaignCopyTracker.java:100)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at org.apache.sling.rewriter.impl.RewriterFilter.doFilter(RewriterFilter.java:83)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at com.adobe.granite.httpcache.impl.InnerCacheFilter.doFilter(InnerCacheFilter.java:81)
at com.adobe.granite.httpcache.impl.InnerCacheFilter.doFilter(InnerCacheFilter.java:60)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at org.apache.sling.i18n.impl.I18NFilter.doFilter(I18NFilter.java:131)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:73)
at com.adobe.granite.resourceresolverhelper.impl.ResourceResolverHelperImpl.doFilter(ResourceResolverHelperImpl.java:83)
at org.apache.sling.engine.impl.filter.AbstractSlingFilterChain.doFilter(AbstractSlingFilterChain.java:68)
at org.apache.sling.engine.impl.SlingRequestProcessorImpl.doProcessRequest(SlingRequestProcessorImpl.java:151)
at org.apache.sling.engine.impl.SlingMainServlet.service(SlingMainServlet.java:234)
at org.apache.felix.http.base.internal.handler.ServletHandler.handle(ServletHandler.java:85)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:79)
at com.adobe.granite.license.impl.LicenseCheckFilter.doFilter(LicenseCheckFilter.java:308)
at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:135)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:74)
at org.apache.felix.http.sslfilter.internal.SslFilter.doFilter(SslFilter.java:96)
at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:135)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:74)
at org.apache.sling.security.impl.ReferrerFilter.doFilter(ReferrerFilter.java:295)
at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:135)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:74)
at org.apache.sling.i18n.impl.I18NFilter.doFilter(I18NFilter.java:131)
at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:135)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:74)
at org.apache.sling.featureflags.impl.FeatureManager.doFilter(FeatureManager.java:116)
at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:135)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:74)
at org.apache.sling.engine.impl.log.RequestLoggerFilter.doFilter(RequestLoggerFilter.java:72)
at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:135)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:74)
at org.apache.felix.http.base.internal.dispatch.Dispatcher.dispatch(Dispatcher.java:127)
at org.apache.felix.http.base.internal.dispatch.DispatcherServlet.service(DispatcherServlet.java:49)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)

rajat1020 avatar Nov 22 '19 12:11 rajat1020

This seems to me that the time to read the binary from S3 is causing a timeout on an AEM/network level (AEM to the browser). I don't think that there is much we can do about it. If you are using AMS you can ask your CSE to increase the network timeout on the AEM Jetty and Dispatcher level.

joerghoh avatar Nov 22 '19 12:11 joerghoh

Thanks Joe, could you please suggest the max network timeout we can put there. Right now its 180000 milliseconds by which users are able to download files upto 1 GB only.

rajat1020 avatar Nov 22 '19 13:11 rajat1020

I cannot suggest any value, because I don't know the filesizes you need to handle. Can you ask your CSE to suggest you a reasonable value?

joerghoh avatar Nov 22 '19 13:11 joerghoh

We need to download assets till 5 GB size.

rajat1020 avatar Nov 22 '19 13:11 rajat1020

than a timeout of 10 minutes might be appropriate.

joerghoh avatar Nov 22 '19 13:11 joerghoh

Thanks David.

Can i ask one more question, what is the maximum asset size which can be downloaded easily in OOTB Asset Shares Commons setup or as per the documentation.

rajat1020 avatar Dec 16 '19 14:12 rajat1020

@rajat1020 im not sure - i'd check with Adobe Support what this limit is based on your AEM setup; for example - AMS may(?) have metrics on this if you use AMS, if you are onPrem then i'd expect youd have to check with your IT team around the guaranteed rates between your AEM instance and S3.

I'd go w/ @joerghoh's suggestion - start with a 10min timeout, try a max-file size and see where it comes in and tune up/down from there.

As you can see from the stack trace you posted, there isn't any Asset Share Commons code being invoked so this sounds more like a AEM and/or Infrastructure constraint (though it does look like there much be a custom download servlet at play).

davidjgonzalez avatar Dec 16 '19 14:12 davidjgonzalez