Missing couchbase.repository.multibucket documentation (4.x) [DATACOUCH-570]

[spring-projects-issues]

Guillaume Durandiere opened DATACOUCH-570 and commented

There is a link about the multibucket configuration in the following documentation (4.0.1) : https://docs.spring.io/spring-data/couchbase/docs/current/reference/html/#couchbase.repository.multibucket

But there is nothing about it.

Will you add it soon ?

 

---

Affects: 4.0.1 (Neumann SR1)

[spring-projects-issues]

Michael Reiche commented

Hi Guillaume DURANDIERE - that section will be added shortly.  In the mean time, to leverage multi-bucket repositories, implement the methods below in your Config class.  The config*OperationsMapping methods configure the mapping of entity-objects to buckets.  Be careful with the method names - using a method name that is a Bean will result in the value of that bean being used instead of the result of the method. 

This example maps Person -> protected,  User -> mybucket, and everything else goes to getBucketName(). Note that this only maps calls through the Repository.

@Override 
public void configureReactiveRepositoryOperationsMapping(ReactiveRepositoryOperationsMapping baseMapping) {
 try {
  ReactiveCouchbaseTemplate personTemplate = myReactiveCouchbaseTemplate(myCouchbaseClientFactory("protected"),new MappingCouchbaseConverter());
  baseMapping.mapEntity(Person.class,  personTemplate); // Person goes in "protected" bucket
  ReactiveCouchbaseTemplate userTemplate = myReactiveCouchbaseTemplate(myCouchbaseClientFactory("mybucket"),new MappingCouchbaseConverter());
  baseMapping.mapEntity(User.class,  userTemplate); // User goes in "mybucket"
  // everything else goes in getBucketName() 
 } catch (Exception e) {
  throw e;
 }
}
@Override
public void configureRepositoryOperationsMapping(RepositoryOperationsMapping baseMapping) {
 try {
  CouchbaseTemplate personTemplate = myCouchbaseTemplate(myCouchbaseClientFactory("protected"),new MappingCouchbaseConverter());
  baseMapping.mapEntity(Person.class,  personTemplate); // Person goes in "protected" bucket
  CouchbaseTemplate userTemplate = myCouchbaseTemplate(myCouchbaseClientFactory("mybucket"),new MappingCouchbaseConverter());
  baseMapping.mapEntity(User.class,  userTemplate); // User goes in "mybucket"
  // everything else goes in getBucketName() 
 } catch (Exception e) {
  throw e;
 }
}

// do not use reactiveCouchbaseTemplate for the name of this method, otherwise the value of that bean
// will be used instead of the result of this call (the client factory arg is different)
public ReactiveCouchbaseTemplate myReactiveCouchbaseTemplate(CouchbaseClientFactory couchbaseClientFactory,
  MappingCouchbaseConverter mappingCouchbaseConverter) {
 return new ReactiveCouchbaseTemplate(couchbaseClientFactory, mappingCouchbaseConverter);
}

// do not use couchbaseTemplate for the name of this method, otherwise the value of that been 
// will be used instead of the result from this call (the client factory arg is different)
public CouchbaseTemplate myCouchbaseTemplate(CouchbaseClientFactory couchbaseClientFactory,
  MappingCouchbaseConverter mappingCouchbaseConverter) {
 return new CouchbaseTemplate(couchbaseClientFactory, mappingCouchbaseConverter);
}

// do not use couchbaseClientFactory for the name of this method, otherwise the value of that bean will
// will be used instead of this call being made ( bucketname is an arg here, instead of using bucketName() )
public CouchbaseClientFactory myCouchbaseClientFactory(String bucketName) {
 return new SimpleCouchbaseClientFactory(getConnectionString(),authenticator(), bucketName );
}

[spring-projects-issues]

Guillaume Durandiere commented

Thanks for your answer! I will try it. 

[spring-projects-issues]

ohjongsung commented

same issue documentation (4.0.2)

https://docs.spring.io/spring-data/couchbase/docs/4.0.2.RELEASE/reference/html/#couchbase.repository.multibucket

   

[spring-projects-issues]

arana3 commented

Michael Reiche

 

The instructions you provided do not work. This is because in the latest 4.x Spring Data Couchbase, the mapEntity method requires the class and an implementation of CouchbaseOperations.

 

Only the CouchbaseTemplate implements CouchbaseOperations. Therefore Reactive integrations will not be able to map other repositories to other entity classes out of the box. I am looking into an alternative in the meantime, by using CouchbaseTemplate and calling the reactive() method on it to get the associated reactive representation.

Only other option is for developers to directly access underlying SDK, but even that is not possible, as the opened buckets are defined during app setup phase

[spring-projects-issues]

Michael Reiche commented

arana3 - 

This is because in the latest 4.x Spring Data Couchbase, the mapEntity method requires the class and an implementation of CouchbaseOperations.

That is correct. That's in the solution provided. Person objects are mapped to the bucket named "protected" and User objects are mapped to the bucket named "mybucket".  Be careful not to name your method that returns the template "reactiveCouchbaseTemplate" and also be careful not to name your method that returns the client factory "couchbaseClientFactory" - there are @Beans by those names, and the value of those beans will be used instead of the values returned by your methods. This is why the example uses "myReactiveCouchbaseTemplate()" and "myCouchbaseClientFactory()" etc.  You may wish to put instrumentation (System.out.println or similar) in your method to ensure it is being used instead of the value of a bean. 

@Override 
public void configureReactiveRepositoryOperationsMapping(ReactiveRepositoryOperationsMapping baseMapping) {
 try {
  ReactiveCouchbaseTemplate personTemplate = myReactiveCouchbaseTemplate(myCouchbaseClientFactory("protected"),new MappingCouchbaseConverter());
  baseMapping.mapEntity(Person.class,  personTemplate); // Person goes in "protected" bucket
  ReactiveCouchbaseTemplate userTemplate = myReactiveCouchbaseTemplate(myCouchbaseClientFactory("mybucket"),new MappingCouchbaseConverter());
  baseMapping.mapEntity(User.class,  userTemplate); // User goes in "mybucket"
  // everything else goes in getBucketName() 
 } catch (Exception e) {
  throw e;
 }
}

https://forums.couchbase.com/t/spring-data-couchbase-4-0-0-spring-boot-2-3-0-and-configure-second-bucket-using-couchbaseclientfactory/26341/3

[mihaidi]

Hello @mikereiche one question I need to connect to multiple buckets inside same Cluster but I need to do this dynamically based on data from application.yaml also I don't need to use any Entity objects I want to query using CouchbaseTemplate and operate with json response is this even posible , thanks in advance

[mikereiche]

Mapping is by entity type only.

[mihaidi]

Mapping is by entity type only.

Well look like no , I figure out a solution and now I operate with multiple buckets without any Entity objects only with JSON string response.

[mikereiche]

"I figure out a solution and now I operate with multiple buckets without any Entity objects only with JSON string response"

Ok, so what is the issue?

[moriyahazan]

hey, i implemented mulibucket as explained in this thread, it works. thanks Now i have a question about Custom Converters it is working only for default bucket and not for all other buckets, any idea how to add convertor for all buckets? thanks in advance

[mikereiche]

Refer to the example - instead of providing a new MappingCouchbaseConverter(), provide your custom couchbase converter.

CouchbaseTemplate userTemplate = myCouchbaseTemplate(myCouchbaseClientFactory("mybucket"),new MappingCouchbaseConverter());

[moriyahazan]

@mikereiche thanks for your reply

i have following convertor:

 @Override
   public CouchbaseCustomConversions customConversions() {
       return new CouchbaseCustomConversions(Arrays.asList(
               ZonedDateTimeToEpochTimeConverter.INSTANCE,
               EpochTimeToZonedDateTimeConverter.INSTANCE));
   }


   @WritingConverter
   public enum ZonedDateTimeToEpochTimeConverter implements Converter<ZonedDateTime, CouchbaseDocument> {
       INSTANCE;

       @Override
       public CouchbaseDocument convert(ZonedDateTime zonedDateTime) {
           CouchbaseDocument cd = new CouchbaseDocument();
           cd.put(EPOC_MILLI, zonedDateTime.toInstant().toEpochMilli());
           cd.put(OFFSET_SECONDS, zonedDateTime.getOffset().getTotalSeconds());
           return cd;
       }
   }

   @ReadingConverter
   public enum EpochTimeToZonedDateTimeConverter implements Converter<CouchbaseDocument, ZonedDateTime> {
       INSTANCE;

       @Override
       public ZonedDateTime convert(CouchbaseDocument epochTime) {
           long timeMilli = Long.parseLong(epochTime.getContent().get(EPOC_MILLI).toString());
           int offsetSeconds = Integer.parseInt(epochTime.getContent().get(OFFSET_SECONDS).toString());

           ZoneOffset convertedOffset = ZoneOffset.ofTotalSeconds(offsetSeconds);
           return ZonedDateTime.ofInstant(Instant.ofEpochMilli(timeMilli), convertedOffset);
       }
   }

i set this converter inside mapping

MappingCouchbaseConverter mappingCouchbaseConverter = new MappingCouchbaseConverter();
mappingCouchbaseConverter.setCustomConversions(customConversions());
baseMapping.mapEntity(Class.forName(beanDefinition.getBeanClassName()),myCouchbaseTemplate(myCouchbaseClientFactory("mybucket"),mappingCouchbaseConverter ));

But i am getting below exception when save the doc although there is convertor from zonedatetime to couchebasedocument

2021-05-27 17:29:27.178 ERROR 18916 --- [nio-9001-exec-1] g.k.e.error.DefaultGraphQLErrorHandler   : Error executing query Exception while fetching data (/createProduct) : No converter found capable of converting from type [java.time.ZonedDateTime] to type [org.springframework.data.couchbase.core.mapping.CouchbaseDocument]

org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [java.time.ZonedDateTime] to type [org.springframework.data.couchbase.core.mapping.CouchbaseDocument]
	at org.springframework.core.convert.support.GenericConversionService.handleConverterNotFound(GenericConversionService.java:322) ~[spring-core-5.3.5.jar:5.3.5]
	at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:195) ~[spring-core-5.3.5.jar:5.3.5]
	at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:175) ~[spring-core-5.3.5.jar:5.3.5]
	at org.springframework.data.couchbase.core.convert.MappingCouchbaseConverter.lambda$getPotentiallyConvertedSimpleWrite$2(MappingCouchbaseConverter.java:784) ~[spring-data-couchbase-4.1.6.jar:4.1.6]
	at java.base/java.util.Optional.map(Optional.java:265) ~[na:na]
	at org.springframework.data.couchbase.core.convert.MappingCouchbaseConverter.getPotentiallyConvertedSimpleWrite(MappingCouchbaseConverter.java:784) ~[spring-data-couchbase-4.1.6.jar:4.1.6]
	at org.springframework.data.couchbase.core.convert.MappingCouchbaseConverter.writeSimpleInternal(MappingCouchbaseConverter.java:774) ~[spring-data-couchbase-4.1.6.jar:4.1.6]
	at org.springframework.data.couchbase.core.convert.MappingCouchbaseConverter.access$200(MappingCouchbaseConverter.java:82) ~[spring-data-couchbase-4.1.6.jar:4.1.6]
	

Can you please help?

[mikereiche]

Please create a new issue. @moriyahazan

[moriyahazan]

please see https://github.com/spring-projects/spring-data-couchbase/issues/1141

[mikereiche]

@moriyahazan - I'm not sure if this is your issue,

mappingCouchbaseConverter.setCustomConversions(customConversions());

But don't use the same name as the existing @Bean method as the spring framework replaces calls to methods with the value of matching @Bean methods.

@Bean(name = BeanNames.COUCHBASE_CUSTOM_CONVERSIONS)
public CustomConversions customConversions() {
	return new CouchbaseCustomConversions(Collections.emptyList());
}

This is why in my examples I always prefix the methods with "my"

[moriyahazan]

@mikereiche I tried to use special name for converter but issue still persist please see code in https://github.com/moriyahazan/designer-service

thanks!

[mikereiche]

Please see my response on #1141

[martinprobson]

Just an extra note as Google brought me here when I searched for multi-bucket configuration.

These instructions worked perfectly, except the Couchbase auditing support (via @EnableCouchbaseAuditing annotation) stopped working. To re-enable I just had to set the application context on the userTemplate object (see 'Extra line' below).

@Override 
public void configureReactiveRepositoryOperationsMapping(ReactiveRepositoryOperationsMapping baseMapping) {
 try {
  ReactiveCouchbaseTemplate personTemplate = myReactiveCouchbaseTemplate(myCouchbaseClientFactory("protected"),new MappingCouchbaseConverter());
  baseMapping.mapEntity(Person.class,  personTemplate); // Person goes in "protected" bucket
  ReactiveCouchbaseTemplate userTemplate = myReactiveCouchbaseTemplate(myCouchbaseClientFactory("mybucket"),new MappingCouchbaseConverter());
  userTemplate.setApplicationContect(applicationContext)  // <----- Extra line
  baseMapping.mapEntity(User.class,  userTemplate); // User goes in "mybucket"
  // everything else goes in getBucketName() 
 } catch (Exception e) {
  throw e;
 }
}

[premvarmak]

Michael Reiche Hi... I am trying to connect to multiple clusters using Spring Data Couchbase. Is there a possibility to it at this moment ? I have copied AbstractCouchbaseConfiguration and created a new class with different bean names.

With this connecttion is created to this new cluster but the repository is not getting that connection through CrudRepository. Is there a way to do it ?

[mikereiche]

The selection is made on the entity class. You'll need to make another repository interface based on the other entity class. If you still have issues, post the repository classes, the entity classes, the config class and the code that calls the repository methods so I can reproduce it.

[premvarmak]

Michael Reiche Thank you for the prompt response. Here is my requirement. I want to have one Spring Boot App which connects to multiple couchbase DBs. So basically...

Couchbase Cluster 1 (This I was able to achieve it by following your comment above) Bucket 1 Bucket 2

Now I want to have another cluster in the same app. where am facing issue. Couchbase Cluster 2 Bucket 1 Bucket 2

Here is the link to my code: https://github.com/premvarmak/spring-pvk-data-couchbase/tree/master

I have added readme file explaining the setup. But its a very basic setup which I have created following the Spring documentation.

[premvarmak]

Michael Reiche Any pointers in this direction ? In the codebase I have provided,

I have override CouchbaseTemplate to provide new cluster and bucket information and marked it under new Bean. Than I override configureRepositoryOperationsMapping. And I am able to get two couchbase instances registered but I am not able to attach this new instance configuration with repositories.

[mikereiche]

Run with the debugger and set a break-point in the implementation of CouchbaseOperations.resolve().

[mikereiche]

"I am trying to connect to multiple clusters using Spring Data Couchbase."

That is completely different than using multiple buckets of the same cluster. Please open a separate issue.

[premvarmak]

Yeah!!! I am able to connect to multiple clusters now...

Your point "The selection is made on the entity class." Based on this, I created multiple CouchbaseTemplate and mapped respective entities in RepositoryOperationsMapping. And then all is working good...

Michael Reiche Thank you for your time :)

[mikereiche]

That's great @premvarmak. Can I use your application as an example to add to the documentation? Is it in the repository posted above?

[premvarmak]

The change is my local. I will commit to the above repository and will let you know here. Thank you... I need to remove some actual DB config and commit to the repo.

[premvarmak]

Michael Reiche

https://github.com/premvarmak/spring-pvk-data-couchbase/blob/master/README.md