HikariCP
HikariCP copied to clipboard
Hibernate generated value (SEQUENCE) violates non-null constraint when saving using a statement.
To start, I am trying to follow this tutorial: https://medium.com/@wahyaumau/boost-jpa-bulk-insert-performance-by-90-3a6232d9068d
When I try to persist 50,000 records of my own using this method, I get the following stack trace:
java.sql.BatchUpdateException: Batch entry 0 INSERT INTO dummy (name) VALUES ('Name 0') was aborted: ERROR: null value in column "id" of relation "dummy" violates not-null constraint
Detail: Failing row contains (null, Name 0). Call getNextException to see other errors in the batch.
at org.postgresql.jdbc.BatchResultHandler.handleCompletion(BatchResultHandler.java:186)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:569)
at org.postgresql.jdbc.PgStatement.internalExecuteBatch(PgStatement.java:881)
at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:904)
at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1634)
at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:127)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
The code I'm using to try to replicate the tutorial:
application.yml:
spring:
jpa:
hibernate:
ddl-auto: create
database-platform: org.hibernate.dialect.PostgreSQLDialect
properties:
hibernate:
jdbc.batch_size: 30
order_inserts: true
datasource:
hikari:
connectionTimeout: 300000
maximumPoolSize: 10
Dummy.java
@Data
@Entity
@Builder
@NoArgsConstructor
@AllArgsConstructor
@Table(name = "dummy")
public class DummyEntity {
@Id
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "seqGen")
@SequenceGenerator(name = "seqGen", sequenceName = "dummy_id_seq", allocationSize = 1)
private Long id;
private String name;
}
DummyRepository.java
public interface DummyRepository extends JpaRepository<DummyEntity, Long> { }
Runner:
var dummies = IntStream.range(0, 50000).mapToObj(val -> DummyEntity.builder().name("Name " + val).build()).toList();
String query = "INSERT INTO dummy (name) VALUES (?)";
try (var conn = hikariSource.getConnection(); var stmt = conn.prepareStatement(query)) {
int counter = 0;
for (var dummy : dummies) {
stmt.clearParameters();
stmt.setString(1, dummy.getName());
stmt.addBatch();
if ((counter + 1) % batchSize == 0 || (counter + 1) == dummies.size()) {
stmt.executeBatch();
stmt.clearBatch();
}
counter++;
}
} catch (Exception e) {
e.printStackTrace();
}
This works fine if I change the generation type to IDENTITY, and it also works with either generation type if I don't use the connection/statement directly. Both of the following work with both generation types:
Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
for (int i = 0; i < entities.size(); i++) {
var entity = entities.get(i);
session.merge(entity);
if (i % 1000 == 0) {
session.flush();
session.clear();
}
}
tx.commit();
session.close();
dummyRepository.saveAll(dummies);
Is Hikari bypassing Hibernate when using a connection/statement directly or something?
It might be worth noting that I don't build the DB connection itself via the properties file, I do it programmatically using the following config class (I have a need to pull the DB password from AWS Secrets Manager in production for this project):
String connectionTemplate = "jdbc:postgresql://%s:%d/%s";
String host = "postgres";
int port = 5432;
String database = "";
String username = "";
String password = "";
// overriding username/password from secrets manager if on prod profile
String connectionString = String.format(connectionTemplate, host, port, database);
var dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.driverClassName("org.postgresql.Driver");
dataSourceBuilder.url(connectionString);
dataSourceBuilder.username(username);
dataSourceBuilder.password(password);
var dataSource = dataSourceBuilder.build();