Spring Batch Basics for Beginners

Introduction

Batch processing can be defined as the processing of large datasets together in a single run, rather than processing each item like we do in an API. This technique is prevalent in most industries including like finance. It's often chosen for operations that deal with significant volumes of information—such as generating reports or executing lengthy calculations and analytic tasks.

Spring Batch is a core module in Spring Framework which enables the development of robust batch applications. It offers all the core features required to efficiently handle large volumes of data, such as transaction management, job status tracking, statistics, and built-in fault tolerance. Its support for advanced scalability, including multi-threading and data partitioning, ensures that jobs can run with high performance.

Spring Batch Architecture

Core Components

JobLauncher The entry point for executing batch jobs. It receives a Job and JobParameters, then launches the job execution. The JobLauncher provides a simple interface for running jobs both synchronously and asynchronously.

Job A job is an entity that encapsulates an entire batch process. It represents a complete workflow and contains one or more Steps that execute in sequence. Each job execution is uniquely identified by a combination of the job name and job parameters, allowing the same job to be run multiple times with different parameters.

Step A step is a domain object that encapsulates an independent phase of a batch job. Each step contains exactly one ItemReader, one ItemWriter, and optionally an ItemProcessor. Steps execute sequentially within a job, and a job is only considered complete when all its steps have completed successfully.

JobRepository The persistence layer for Spring Batch metadata. It stores information about job executions, step executions, job parameters, and execution context. The JobRepository is used by JobLauncher, Job, and Step to persist and retrieve execution state, enabling restartability and tracking of batch operations. It provides the foundation for job restart capabilities and execution history.

Item Processing Components

ItemReader Responsible for reading data from a source (database, file, queue, etc.) one item at a time. The reader is invoked repeatedly until it returns null, indicating no more items are available. Common implementations include JdbcPagingItemReader, FlatFileItemReader, and JmsItemReader.

ItemProcessor (Optional) An optional component that transforms or validates items between reading and writing. The processor receives an item from the ItemReader, applies business logic, and returns a transformed item (or null to filter it out). This is where you typically perform data validation, transformation, or enrichment operations.

ItemWriter Responsible for writing processed items to a destination. It receives items in chunks (as configured by the chunk size) and writes them in batches for efficiency. Common implementations include JdbcBatchItemWriter, FlatFileItemWriter, and JmsItemWriter.

• JobLauncher → launches → Job
• Job → contains (1 to many) → Step
• Step → orchestrates → ItemReader , ItemProcessor , ItemWriter
• JobRepository ↔ maintains state for all of the above

This layered design provides separation of concerns, testability, and the flexibility to mix and match different reader/processor/writer implementations based on your specific data sources and requirements.

Batch Use Case: Creating Daily Account Balance Snapshots

Now let's implement a Spring Batch job to create account balance snapshots.

Imagine a banking system with millions of accounts and tens of millions of transactions per day. Calculating an account's current balance by summing its entire transaction history is slow and resource-intensive. A much better approach is to create a daily snapshot and then use it to calculate the balance for the next day by applying the new transactions on top of it.

Our job will implement this logic:

• For a given snapshot_date, find all accounts that have new transactions since the last snapshot.
• For each of these accounts, fetch the balance from the previous day's snapshot.
• Apply only the new transactions (those between the last snapshot and the current one) to this balance.
• Write the newly calculated balance to the snapshot table for the current snapshot_date.

This way, the batch job only processes a small, incremental subset of data each day, ensuring high performance and scalability.

Data Models and Schema

First, let's define our database schema. We need a table for live, incoming transactions and another to store our daily balance snapshots.

src/main/resources/schema.sql:

CREATE TABLE IF NOT EXISTS transactions (
    transaction_id VARCHAR(255) PRIMARY KEY,
    account_number VARCHAR(255) NOT NULL,
    amount NUMERIC NOT NULL,
    transaction_type VARCHAR(10) NOT NULL CHECK (transaction_type IN ('CREDIT', 'DEBIT')),
    transaction_date TIMESTAMP NOT NULL
);

CREATE TABLE IF NOT EXISTS account_balance_snapshot (
    account_number VARCHAR(255) NOT NULL,
    balance NUMERIC NOT NULL,
    snapshot_date DATE NOT NULL,
    PRIMARY KEY (account_number, snapshot_date)
);

Let us create some data in the database to test our job.

src/main/resources/data.sql:

DELETE FROM transactions;
DELETE FROM account_balance_snapshot;

INSERT INTO account_balance_snapshot (account_number, snapshot_date, balance)
VALUES ('ACC1001', DATE '2025-11-10', 1000.00),
       ('ACC1002', DATE '2025-11-10', 500.00);

INSERT INTO transactions (transaction_id, account_number, amount, transaction_type, transaction_date)
VALUES ('txn_001', 'ACC1001', 50.00, 'DEBIT', TIMESTAMP '2025-11-11 08:30:00'),
       ('txn_002', 'ACC1002', 150.00, 'CREDIT', TIMESTAMP '2025-11-11 09:00:00'),
       ('txn_003', 'ACC1001', 200.00, 'DEBIT', TIMESTAMP '2025-11-11 12:45:00'),
       ('txn_004', 'ACC1003', 75.00, 'CREDIT', TIMESTAMP '2025-11-11 14:00:00');

Spring Boot wires the connection to PostgreSQL and drives the schema/data initialization via the following configuration:

src/main/resources/application.properties:

spring.datasource.url=jdbc:postgresql://localhost:5433/mydatabase
spring.datasource.username=myuser
spring.datasource.password=secret
spring.datasource.driver-class-name=org.postgresql.Driver
spring.sql.init.mode=always
spring.sql.init.schema-locations=classpath:schema.sql
spring.sql.init.data-locations=classpath:data.sql
spring.batch.jdbc.initialize-schema=always
spring.batch.job.enabled=true

Our Java model will represent the snapshot we are creating.

AccountSnapshot.java:

public record AccountSnapshot(String accountNumber, double balance, LocalDate snapshotDate) {}

Implementing the Snapshot Job

1. ItemReader: Finding Accounts with New Transactions

We don't want to process every account, only those with activity. The ItemReader will select the distinct account_numbers that have transactions dated after our last snapshot.

@Bean
@StepScope
public JdbcPagingItemReader<String> activeAccountReader(DataSource dataSource,
        @Value("#{jobParameters['snapshotDate']}") String snapshotDateStr) throws Exception {
    LocalDate snapshotDate = LocalDate.parse(snapshotDateStr);
    LocalDateTime snapshotStart = snapshotDate.atStartOfDay();
    LocalDateTime snapshotEnd = snapshotDate.plusDays(1).atStartOfDay();

    Map<String, Object> parameterValues = new HashMap<>();
    parameterValues.put("snapshotStart", snapshotStart);
    parameterValues.put("snapshotEnd", snapshotEnd);

    return new JdbcPagingItemReaderBuilder<String>()
        .name("activeAccountReader")
        .dataSource(dataSource)
        .selectClause("SELECT DISTINCT account_number")
        .fromClause("FROM transactions")
        .whereClause("WHERE transaction_date >= :snapshotStart AND transaction_date < :snapshotEnd")
        .sortKeys(Map.of("account_number", Order.ASCENDING))
        .rowMapper((rs, rowNum) -> rs.getString("account_number"))
        .pageSize(100)
        .parameterValues(parameterValues)
        .build();
}

Note: The @StepScope keeps the reader parameterised per execution. Here we are using JdbcPagingItemReaderBuilder to build the reader.

2. ItemProcessor: Calculating the New Balance

This is where the magic happens. The processor receives an accountNumber, fetches its last known balance, sums up the new transactions, and returns a new AccountSnapshot.

@Component
@StepScope
public class SnapshotProcessor implements ItemProcessor<String, AccountSnapshot> {

    private final JdbcTemplate jdbcTemplate;
    private final LocalDate snapshotDate;
    private final LocalDate previousSnapshotDate;
    private final LocalDateTime snapshotStart;
    private final LocalDateTime snapshotEnd;

    public SnapshotProcessor(
        JdbcTemplate jdbcTemplate,
        @Value("#{jobParameters['snapshotDate']}") String snapshotDateStr) {
        this.jdbcTemplate = jdbcTemplate;
        this.snapshotDate = LocalDate.parse(snapshotDateStr);
        this.previousSnapshotDate = this.snapshotDate.minusDays(1);
        this.snapshotStart = this.snapshotDate.atStartOfDay();
        this.snapshotEnd = this.snapshotDate.plusDays(1).atStartOfDay();
    }

    @Override
    public AccountSnapshot process(String accountNumber) {
        MDC.put("account_number", accountNumber);
        try {
            double lastBalance = fetchPreviousBalance(accountNumber);
            double transactionDelta = fetchTransactionDelta(accountNumber);
            double newBalance = lastBalance + transactionDelta;
            return new AccountSnapshot(accountNumber, newBalance, snapshotDate);
        } finally {
            MDC.remove("account_number");
        }
    }

    private double fetchPreviousBalance(String accountNumber) {
        try {
            return jdbcTemplate.queryForObject(
                "SELECT balance FROM account_balance_snapshot WHERE account_number = ? AND snapshot_date = ?",
                Double.class,
                accountNumber,
                previousSnapshotDate);
        } catch (EmptyResultDataAccessException ex) {
            return 0.0d;
        }
    }

    private double fetchTransactionDelta(String accountNumber) {
        Double result = jdbcTemplate.queryForObject(
            """
                SELECT COALESCE(SUM(
                    CASE 
                        WHEN transaction_type = 'CREDIT' THEN amount
                        WHEN transaction_type = 'DEBIT' THEN -amount
                        ELSE 0
                    END
                ), 0.0)
                FROM transactions
                WHERE account_number = ?
                  AND transaction_date >= ?
                  AND transaction_date < ?
            """,
            Double.class,
            accountNumber,
            snapshotStart,
            snapshotEnd);
        return result != null ? result : 0.0d;
    }
}

3. ItemWriter: Saving the New Snapshots

Finally, the ItemWriter saves the newly calculated snapshots to the database. We use an UPSERT command to handle both new and existing accounts gracefully.

@Bean
public JdbcBatchItemWriter<AccountSnapshot> snapshotWriter(DataSource dataSource) {
    String sql = """
        INSERT INTO account_balance_snapshot (account_number, snapshot_date, balance)
        VALUES (:accountNumber, :snapshotDate, :balance)
        ON CONFLICT (account_number, snapshot_date)
        DO UPDATE SET balance = EXCLUDED.balance
        """;
    return new JdbcBatchItemWriterBuilder<AccountSnapshot>()
        .dataSource(dataSource)
        .sql(sql)
        .itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>())
        .build();
}

Job and Step Configuration

Now we can assemble the components into a Job.

@Bean
public Job createSnapshotJob(JobRepository jobRepository, PlatformTransactionManager transactionManager,
        ItemReader<String> reader, SnapshotProcessor processor, JdbcBatchItemWriter<AccountSnapshot> writer,
        JdbcTemplate jdbcTemplate) {
    return new JobBuilder("createSnapshotJob", jobRepository)
        .listener(new JobCompletionNotificationListener(jdbcTemplate))
        .start(new StepBuilder("createSnapshotStep", jobRepository)
            .<String, AccountSnapshot>chunk(100)
            .transactionManager(transactionManager)
            .reader(reader)
            .processor(processor)
            .writer(writer)
            .build())
        .build();
}

The listener centralises completion reporting and provides visibility into how many snapshots were produced.

JobCompletionNotificationListener.java:

public class JobCompletionNotificationListener implements JobExecutionListener {

    private static final Logger logger = LoggerFactory.getLogger(JobCompletionNotificationListener.class);
    private final JdbcTemplate jdbcTemplate;

    public JobCompletionNotificationListener(JdbcTemplate jdbcTemplate) {
        this.jdbcTemplate = jdbcTemplate;
    }

    @Override
    public void afterJob(JobExecution jobExecution) {
        String snapshotDateParameter = jobExecution.getJobParameters().getString("snapshotDate");
        BatchStatus status = jobExecution.getStatus();
        
        if (status == BatchStatus.COMPLETED) {
            LocalDate snapshotDate = LocalDate.parse(snapshotDateParameter);
            Integer snapshotCount = jdbcTemplate.queryForObject(
                "SELECT COUNT(*) FROM account_balance_snapshot WHERE snapshot_date = ?",
                Integer.class,
                java.sql.Date.valueOf(snapshotDate));
            logger.info("Snapshot job completed successfully for {} with {} account updates.", snapshotDate, snapshotCount);
        }
    }

    @Override
    public void beforeJob(JobExecution jobExecution) {
        String snapshotDateParameter = jobExecution.getJobParameters().getString("snapshotDate");
        logger.info("Job execution started for snapshotDate={}", snapshotDateParameter);
    }
}

Running the Job with Parameters

Because we used @StepScope, we need to provide the snapshotDate when launching the job. This makes the job reusable for any given day.

Running from the command line

You can pass parameters via the command line when running the fat jar:

java -jar target/spring-batch-banking-app-0.0.1-SNAPSHOT.jar snapshotDate=2025-11-11

Running from IntelliJ IDEA

If you prefer to run the job directly from your IDE:

• Create (or edit) a Run Configuration for SpringBatchBankingAppApplication.
• In Program arguments, add:

snapshotDate=2025-11-11

With spring.batch.job.enabled=true, Spring Boot’s JobLauncherApplicationRunner will automatically start createSnapshotJob using the snapshotDate parameter you passed.

Triggering via HTTP

You can also trigger the job programmatically from a controller:

@PostMapping("/trigger-job")
public void triggerJob(@RequestParam String snapshotDate) {
    JobParameters params = new JobParametersBuilder()
        .addString("snapshotDate", snapshotDate)
        .toJobParameters();
    jobLauncher.run(createSnapshotJob, params);
}

Error Handling in Spring Batch

Spring Batch provides fault-tolerance features so transient failures do not break the whole job and bad records can be isolated.

You can extend the snapshot step with retry and skip rules:

@Bean
public Job createSnapshotJob(JobRepository jobRepository, PlatformTransactionManager transactionManager,
        ItemReader<String> reader, SnapshotProcessor processor, JdbcBatchItemWriter<AccountSnapshot> writer,
        JdbcTemplate jdbcTemplate) {
    Step snapshotStep = new StepBuilder("createSnapshotStep", jobRepository)
        .<String, AccountSnapshot>chunk(100)
        .transactionManager(transactionManager)
        .reader(reader)
        .processor(processor)
        .writer(writer)
        .faultTolerant()
        .retryLimit(3)
        .retry(DeadlockLoserDataAccessException.class)
        .skipLimit(10)
        .skip(BusinessValidationException.class)
        .build();

    return new JobBuilder("createSnapshotJob", jobRepository)
        .listener(new JobCompletionNotificationListener(jdbcTemplate))
        .start(snapshotStep)
        .build();
}

• Retry: transient database issues like deadlocks are retried a few times before failing the step.
• Skip: items that throw a business exception are skipped (up to the configured limit) while the rest of the chunk continues.

Scheduling the Snapshot Job

Instead of triggering the job manually, you can schedule it with Spring’s @Scheduled support so that a new snapshot is created every day.

Enable scheduling in your main application class:

@SpringBootApplication
@EnableScheduling
public class SpringBatchBankingAppApplication {
    public static void main(String[] args) {
        SpringApplication.run(SpringBatchBankingAppApplication.class, args);
    }
}

Then create a scheduler component that launches the batch job with the current date:

@Component
public class SnapshotJobScheduler {

    private final JobLauncher jobLauncher;
    private final Job createSnapshotJob;

    public SnapshotJobScheduler(JobLauncher jobLauncher, Job createSnapshotJob) {
        this.jobLauncher = jobLauncher;
        this.createSnapshotJob = createSnapshotJob;
    }

    @Scheduled(cron = "0 0 1 * * ?") // every day at 01:00 AM
    public void runDailySnapshotJob() throws Exception {
        String snapshotDate = LocalDate.now().toString();

        JobParameters jobParameters = new JobParametersBuilder()
            .addString("snapshotDate", snapshotDate)
            .addLong("run.id", System.currentTimeMillis())
            .toJobParameters();

        jobLauncher.run(createSnapshotJob, jobParameters);
    }
}

For enterprise setups you can alternatively trigger the same job from an external scheduler (Quartz, Control-M, Kubernetes CronJobs, etc.) by calling the JobLauncher with appropriate JobParameters, similar to how we did in the controller.

Testing with Testcontainers

We will use Testcontainers to spin up a PostgreSQL container and launch the batch job with JobLauncherTestUtils to test the job.

📚 Related: To learn more about integration testing with Testcontainers, check out our blog on Integration Testing in Spring Boot with Testcontainers

SpringBatchBankingAppApplicationTests.java:

@SpringBootTest
@Testcontainers
@Import(SpringBatchBankingAppApplicationTests.BatchTestConfig.class)
class SpringBatchBankingAppApplicationTests {

    private static final LocalDate SNAPSHOT_DATE = LocalDate.of(2025, 11, 11);
    private static final Date SNAPSHOT_SQL_DATE = Date.valueOf(SNAPSHOT_DATE);

    @Container
    static final PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:16-alpine");

    @DynamicPropertySource
    static void configureDataSource(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.url", postgres::getJdbcUrl);
        registry.add("spring.datasource.username", postgres::getUsername);
        registry.add("spring.datasource.password", postgres::getPassword);
        registry.add("spring.batch.jdbc.initialize-schema", () -> "always");
    }

    @Test
    void shouldCreateSnapshotsForActiveAccounts() throws Exception {
        JobExecution jobExecution = jobLauncherTestUtils.launchJob(jobParameters());
        assertThat(jobExecution.getStatus()).isEqualTo(BatchStatus.COMPLETED);

        Double acc1001 = jdbcTemplate.queryForObject(
            "SELECT balance FROM account_balance_snapshot WHERE account_number = ? AND snapshot_date = ?",
            Double.class,
            "ACC1001",
            SNAPSHOT_SQL_DATE);

        assertThat(acc1001).isEqualTo(750.0d);
    }

    @Test
    void shouldBeIdempotentForSameSnapshotDate() throws Exception {
        jobLauncherTestUtils.launchJob(jobParameters());
        JobExecution secondExecution = jobLauncherTestUtils.launchJob(jobParameters());
        assertThat(secondExecution.getStatus()).isEqualTo(BatchStatus.COMPLETED);

        Double balanceAfterRerun = jdbcTemplate.queryForObject(
            "SELECT balance FROM account_balance_snapshot WHERE account_number = ? AND snapshot_date = ?",
            Double.class,
            "ACC1001",
            SNAPSHOT_SQL_DATE);

        assertThat(balanceAfterRerun).isEqualTo(750.0d);
    }
}

Monitoring and Observability

Below are some of the monitoring and observability features that you can use to monitor the performance of your batch job.

• Monitoring and metrics based on Micrometer You can use Micrometer to monitor the performance of your batch job and also get metrics based on your business requirements.
• Java Flight Recorder (JFR) JFR is a low-overhead, event-based profiling tool built into the Java Virtual Machine (JVM) that allows developers to collect detailed information about the performance and behavior of their applications.
• Job Listeners: Adding job listeners like JobExecutionListener can help you track the progress of your job and handle any errors that occur.
• Proper logging You can use proper logging to track the performance of your batch job and also troubleshoot any issues that occur.

Conclusion

Spring Batch is a powerful framework that can be used to build robust enterprise batch applications.

You can find the complete code for this tutorial on GitHub.

To stay updated with the latest updates in Java and Spring follow us on linked in and medium.

Happy coding!