Hibernate Lost My Update

Osipiuk·Java, Hibernate, Spring·Topic 114 min·29 Oct 2026

ACID? Sure, but at the database level, not in the tools on top. How Hibernate makes a partial commit when Spring Boot shuts down – and how to prevent it.

Czytaj po polskuIn this topic1.1In short

1.2Events that won’t go through

1.3The pod’s last logs

1.4How Hibernate writes your changes

1.5When Hibernate skips the flush at commit

1.6What happens on SIGTERM

1.7Is it just our app?

1.8The 1.2 ms window

1.9How to protect yourself

1.10Where it came from, and what’s next

1.11What should I do now?

In short

There’s a chance that during application shutdown, Hibernate won’t run the last flush() before the commit. When that happens, we end up with a commit that doesn’t contain all our changes, and nothing warns us about it (apart from a log at TRACE level).

1

@Transactional

2

public void markPaid(UUID id) {

3

var order = repo.findById(id).orElseThrow();

4

order.setStatus(PAID); // waits in memory for the flush at commit

5

longOperation.run(); // slow call: SIGTERM arrives, Spring closes the SessionFactory

6

} // commit: flush skipped, no exception, the UPDATE is never sent

How I found it and why it happens are below. If you only want the fix, skip to section 1.9.

First task at a new job: “find out why the events in the queue can’t be processed”. A great first task to get to know the company’s tools. The code in short: in one transaction we mark a parent record and its records as deleted (soft delete) + write a transactional outbox (opens in a new tab) with an event for every deleted record.

OK, so I check the logs and monitoring. First suspect – a bug in the code, or a race condition thanks to Read Committed isolation. Nope: the code looked 100% fine and hadn’t changed since that event showed up.

I noticed that the logs from that operation were among the last ones on the pod, because autoscaling was just stopping it. But that wasn’t the most interesting part. Another service had read the event about a record being deleted, and on our side that record wasn’t deleted at all… Interestingly, the last event of that transaction was about exactly the record whose change disappeared.

Since the code looked fine, I asked the database. Deleting is just a deleted flag for us, so the rows were still there. I learned that every row in Postgres has a hidden xmin column: the id of the transaction that wrote its current version. age(xmin) tells you how many transactions ago that was.

1

SELECT id, deleted, age(xmin) AS transactions_ago

2

FROM record

3

WHERE parent_id = 42;

The parent record and the deleted records – a fresh xmin, from the incident’s transaction. The record that wasn’t deleted – an xmin from hundreds of millions of transactions ago. So nobody had touched that row in a very long time. Which means the UPDATE wasn’t overwritten by another transaction – Postgres simply never got it. But in that same transaction it did get, and commit, the outbox row with the event about deleting that record.

“Connecting the dots” and ruling out a bug in the code or in the database, I started to suspect Hibernate. With some help from AI I found an interesting bit in beforeTransactionCompletion(). But one thing at a time – first: when does Hibernate send changes to the database at all?

If you know how the persistence context works and when Hibernate flushes, you can skip this one – next up is section 1.5.

In a nutshell – Hibernate (when it can) keeps changes to entities in the “persistence context” and sends them to the database only at a flush. A flush runs before a query that might read those changes, and always right before the commit (or so I thought).

Not everything waits for the flush, though. These go to the database right away1:

Everything else – changed entity fields, remove(), new entities with a sequence id – waits in memory for the flush. So if the flush at commit didn’t run, the database would end up with only what had been sent before.

For each record, our code looked roughly like this:

1

Record record = recordRepository.findActiveById(recordId);

2

record.setDeleted(true); // the UPDATE waits in memory for the flush

3

outbox.save(new RecordDeletedEvent(recordId)); // JDBC, sent right away

If the flush at commit doesn’t run, the event is saved and deleted stays false.

Why was it the last record that got lost? findActiveById() is a query on the record table, and Hibernate also flushes before a query that reads a table with something waiting to be written. So each next record pushed the previous record’s UPDATE out to the database, and at commit only the last one was still waiting in memory. Exactly what xmin showed.

findById() works differently: under the hood it’s em.find(), i.e. loading an entity by id, and that never flushes – even when it goes to the database for a SELECT. If we had used findById(), the commit would have lost the changes to all the records, not just the last one.

This is where beforeTransactionCompletion() comes back. At commit it calls managedFlush(), which checks whether the session is open. Here’s what that looks like in Hibernate 7.4.5:

1

private void managedFlush() {

2

if ( !isOpenOrWaitingForAutoClose() ) {

3

SESSION_LOGGER.skippingAutoFlushSessionClosed(); // TRACE

4

}

5

else {

6

SESSION_LOGGER.automaticallyFlushingSession();

7

fireFlush();

8

}

9

}

10

11

// inherited from AbstractSharedSessionContract

12

public boolean isOpenOrWaitingForAutoClose() {

13

return !closed && factory.isOpen()

14

|| waitingForAutoClose;

15

}

When the factory is closed, every one of its sessions counts as closed, so managedFlush() logs Skipping auto-flush since the session is closed at TRACE2 and returns, and Connection.commit() goes to Postgres, which commits everything it got. Even worse – nothing throws, the only trace is that one log line at TRACE level (!).

You don’t need Spring to reproduce it, or even a second thread. In the test below Payment has an IDENTITY id, so its INSERT goes out at persist(), while the order’s new status waits for the flush. We close the factory, commit and… the payment is there, and the order is still NEW.

GitHub — SessionFactoryClosedBeforeCommitTest.java in the repository, opens in a new tab

1

@Test

2

void closeAndCommitOnTheSameThreadAlsoSaveOnlyPartOfTheTransaction() throws Exception {

3

Session session = sessionFactory.openSession();

4

Transaction transaction = session.beginTransaction();

5

Payment payment = new Payment();

6

payment.orderId = 1L;

7

session.persist(payment);

8

session.find(Order.class, 1L).status = "PAID";

9

10

sessionFactory.close();

11

transaction.commit();

12

session.close();

13

14

assertEquals(1, count("select count(*) from payments where orderId = 1"), "the INSERT was committed");

15

assertEquals("NEW", status(), "the UPDATE was never executed, yet the transaction was committed");

16

}

I reproduced it on Hibernate 6.6.53, 6.6.59, 7.4.5 and 7.4.13 with PostgreSQL, and on 7.4.13 and 8.0.0.Beta3 with H2 (on Hibernate’s test case template). I didn’t check versions older than 6.6.

But who closes the factory in the middle of a transaction?

Remember that autoscaling was just stopping the pod? On SIGTERM, Spring Boot shuts down in two stages. First (the SmartLifecycle phase) it stops whatever can create new work: HTTP endpoints, listeners/consumers etc. + gives them a limited time to finish (by default: the Kafka container waits 10 s, RabbitMQ 5 s, and each shutdown phase in Spring Boot at most 30 s). Then it destroys beans, our EntityManagerFactory among them, and finally the DataSource.

The problem is that the second stage assumes the first one finished everything, and a handler waiting for a slow external API may not make it in time. Spring moves on to the next phase and starts destroying beans even though the thread is still running. The thread finally gets its response, leaves the @Transactional method, and Hibernate commits without a flush, because the EntityManagerFactory is already closed.

I’ll show it on a demo application (GitHub, opens in a new tab): a small shop where a single @Transactional method loads an order, saves a payment, marks the order as paid and runs a long operation (yes, normally we’d split this into several messages, but it’s a good example to show the problem with Hibernate). Below is that transaction step by step – in step 4, while the thread waits, Spring closes the factory. Leave the guard switch (more on it later) off for now.

Hibernate memory the scratchpad (persistence context)

Guard: the factory is closed, so it throws

1Order order = orders.findById(orderId).orElseThrow();

2payments.save(new Payment(order, amount));

4longOperation.run();meanwhile, another thread: Spring shuts downentityManagerFactory.close();

6// Hibernate sends COMMIT// the guard throws: ROLLBACK

Guard: the factory is closed, so it throws

sent to Postgres:BEGINSELECT … ordersINSERT … paymentsUPDATE … orders ✗ never sentCOMMITROLLBACK

✗ Partial commit, no exception, no error log

✓ Rolled back: the caller gets an exception, the message is redelivered

Step 1 of 6. The worker thread enters the @Transactional method, so Spring begins a transaction. findById() loads order #1 with a SELECT, and Hibernate keeps it in its memory: status NEW.

BackForwardStart over

Guardwithout guardoffwith guardon

What reaches Postgres right away, and what waits in Hibernate’s memory

And here’s how it plays out in time – the worker thread and Spring shutting down, side by side:

≈1.2 ms in a plain Spring Boot app

Worker thread

✗ Payment saved, order still NEW, no error

Worker thread: a message is received and a transaction begins.

BackForwardStart over

Timeline

A worker thread (e.g. a Kafka consumer) when SIGTERM arrives

Funnily enough, with SIGKILL the connection drops and Postgres rolls back. A partial commit needs exactly the “polite” (SIGTERM) shutdown.

At first I thought it was a configuration issue in our service, so I built a clean Spring Boot app (GitHub, opens in a new tab) + Spring JPA + Spring Kafka/AMQP. The result – surprising. I reproduced the bug there too, which means it can hit any application (with a bit of [bad] luck).

The repo is the same shop as in the demos above: Spring Boot 4 + Hibernate 7 + Postgres, with Spring MVC, Kafka and RabbitMQ. Each test sends a payment in through a different entry point, stops the pod in the middle of the long operation, and finally looks into the database to verify the result.3

One thing is deliberately “rigged”: a MetricsExporter bean takes 0.5 s to close and is wired so that Spring destroys it between Hibernate and the pool. That stretches the window (more on it in the next section) from ~1.2 ms to half a second, so the tests hit it every time and are deterministic. Everything else – Spring, Hibernate, HikariCP, the listener containers – runs unchanged. The test that measures the window itself works without this trick – we have that one too :)

Results (the “With the guard” column will make sense two sections from now):

(GitHub, opens in a new tab)payment saved, order NEW, success reported, no error loggedrolled back, a new pod processes the message in full@RabbitListener (GitHub, opens in a new tab)payment saved, order NEW, message counted as processedrolled back, the message returns to the queueHTTP request (GitHub, opens in a new tab)payment saved, order NEW, no response to the callernothing saved, the caller can safely retry@Async task (GitHub, opens in a new tab)payment saved, order NEW, the provider already got 202nothing half-saved, but the provider won’t resend@Scheduled job (GitHub, opens in a new tab)payment saved, order NEW, no error loggedrolled back, nothing savedPostgres’s own log and xmin (GitHub, opens in a new tab), Kafka and RabbitMQINSERT and COMMIT received, the UPDATE neverROLLBACK instead of COMMIT

To run the tests you need Docker and Java 21+:

1

$ git clone htt