Java - Specifying all namespaces or one namespace

[aswath-veersoft]

Hi, we have lot of namespaces in the datastore, and we want to change the schema of some KINDS. For this, we want to specify all the namespaces or one namespace as a parameter.

Currently, if the Map has to be run for all namespaces, then we have to create n no of jobs. After this, we have to monitor, if everything was fine or not by the jobid. This is very very inconvenient.

[aswath-veersoft]

anyway you can make the mapreduce easy for namespaces by specified wildcards or passing a list of namespaces.

[aswath-veersoft]

Any update on this.

[aswath-veersoft]

Any update on this.

[aozarov]

The MR Datastore input source is based on the datastore API which does not support cross name space queries. Indeed, it could have started with a metadata query to get all the name spaces and then for each namespace create a separate query to iterate over but I am not aware of any plans to do that so far. However, doing that in the application level and tying all the individual MR jobs in one pipeline job (using the AE Pipeline API - see this example) should not be too bad and allow the namespaces to be processed in parallel.

[aswath-veersoft]

Hi, Thanks. We don't know much about the new mapreduce. We are using namespaces for 5 years, and we are lacking a good way to do schema change. More java examples with namespaces will encourage teams to use mapreduce and also adopt namespaces. The dataflow team also lacks good support for namespaces. Namespaces support and tooling is always poor and is like an after-thought process with adjustments.
My 2 cents Thanks.

[aswath-veersoft]

Hi, if there are 10k namespaces, then we should start 10k jobs and monitor them. Instead, if there is one job, that can run all the namespaces specified, that is more elegant.

Thanks

[aozarov]

@aswath-veersoft I completely agree with you. It is just a matter of priority. As this project is open-source any contribution are welcome! Until then the only viable option that I can think of is using AE pipelines (as suggested before) to tie all the jobs together problematically (and then you can use the Pipeline UI to monitor them as one pipeline).

[aswath-veersoft]

Any news of supporting multiple namespaces in the mapreduce. It should be out of the box solution.
Looking forward to Google supporting this issue and make it easy for developers to use namespaces in the appengine applications.
Without proper support from all sides, namespaces is a very bad addition, and developers are loosers.

Thanks.

[aswath-veersoft]

Any help on this? Thanks.

[ludoch]

Hi, Sorry, we have no plans working on this but we do accept contributions from the community. Thanks.

[aswath-veersoft]

It is so bad on the appengine team to ignore the easy use of namespaces in a mapreduce. You are penalizing the small developers using namespaces by asking them to learn more and write more code. Why is namespaces support ignored?

[aswath-veersoft]

Contributions? Developers are looking for help and more support with examples on this mapreduce. Don't understand, why Google makes it opensource and pushes the onus on to the community. Its hardly 4-5 years you introduced mapreduce with a big annoucements and blogs, and now Google is already retracting on its support.