appengine-mapreduce

appengine-mapreduce copied to clipboard

Any plan for the Go version?

15

We're planning to build some features based on appengine-mapreduce. Just wanted to know if there's any plan for the Go version. Or if it's left to the community to contribute....

adieu

"The datastore operation timed out" with many shards

1

I tried starting a mapper job with the `DatastoreInputReader` on an entity type that has about 30 million entities. I specified 30,000 shards, which causes the `/mapreduce/kickoffjob_callback` job to fail...

Erfa

Dead link in documentation

1

This link is dead in the section about Python output writers. http://leveldb.googlecode.com/svn/trunk/doc/log_format.txt

Erfa

GoogleCloudStorageLevelDbOutput doesn't cleanup temporary files

1

I am using **GoogleCloudStorageLevelDbOutput** and I see that after mapReduce job completes it doesn't cleanup temporary files, so when result file is **output-0** then there are also files like a:...

dmitriiKovchin

Java - Specifying all namespaces or one namespace

12

Hi, we have lot of namespaces in the datastore, and we want to change the schema of some KINDS. For this, we want to specify all the namespaces or one...

aswath-veersoft

see my comment here (not sure if it would be seen, so posting an issue): https://github.com/GoogleCloudPlatform/appengine-mapreduce/commit/3bcd874ebde611807f5e59e0194918fea5095243#commitcomment-20242630 because this `.Get` happens inside of the `for` loop over the filters... if I...

NickFranceschina

Create a better way to run MapReduce jobs from unit tests.

1

For the project's own unit tests this can be done by extending EndToEndTestBase. However this comes with a lot of baggage, and would need to be cleaned up. Alternately InProcessMapReduce...

tkaitchuck

I was following this example in here: https://sookocheff.com/post/appengine/mapreduce/mapreduce-yaml/ ``` from mapreduce import operation def touch(entity): """ Update the entities timestamp. """ yield op.db.Put(entity) ``` Please notice that `op` doesn't exist....

tanin47

Ant (java) build failed

1

Hi, I tried to build de java version but it failed. The same with the release. Thanks. [ivy:retrieve] :: UNRESOLVED DEPENDENCIES :: [ivy:retrieve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:retrieve] :: com.google.http-client#google-http-client;[1.19.0,2.0): not found [ivy:retrieve]...

ridiaz

DatastoreInputReader filter item in list

1

When you want to check if a list contains an item you normally check by doing `.filter("list =", item)` but DatastoreInputReader validation is expecting a list instead of an item....

RubenMattan