Tom Kaitchuck

B4s should have more than 128mb... Also I should clarify, I was referring to the map stage. Because the files are split out by the mapper according to the reduce...

You are correct: https://github.com/GoogleCloudPlatform/appengine-mapreduce/blob/master/python/demo/main.py#L133 Your proposal sounds reasonable. I think the parent entity should probably be the user as that provides a rather convenient segmentation. If you'd like to take...

Actually I didn't notice, the 1261 requests. That's a lot. So clearly it is not dying right away. Are those all MR requests? or are you also serving other traffic...

Why are you running it as root? Where does it say that? If you run as non-root it should pull the dependencies into the local directory. It does an rm...

Humm. That is odd. I've added six on the list of things to delete. See if that helps. It is an old version of six, if all else fails removing...

Try getting a newer version of PIP. I had a lot of problems with the older versions.

We could cap the number of shards to prevent this sort of error. I have run 1024 successfully. In truth though, adding more shards once there are already that many...

You can manually scale if you like, it is obviously predictable. However if you do make sure you set your number of shards to be significantly more than your number...

Can you post the full stack trace? It sounds like it is simply not tolerated if the file to be deleted is missing.

dt-rush: You are generally correct that if the MR fails you will see something. The controller re-scheduling does not appear to be related to the success or failure of the...