Microservice: Circuit Breaker, Retry, and Rate Limiter with Resilience4J

Introduction#

In an architecture based on microservices, applications are built up of many small independent services that communicate over the network. This flexibility and scalability also come with potential network failure, sluggish responses, or high traffic loads to which the system must respond reliably.

To tackle these problems, Resilience4J supplies a menu of fault-tolerance options: Circuit Breakers, Retries, Rate Limiters, and others. These options protect against cascading failures while improving stability and user experience through fault handling.

• The Circuit Breaker prevents a failing service from being re-requested until it has enough time to recover.
• The Retry lets the system re-attempt an operation that did not succeed on its first attempt, thereby averting transient failures.
• Rate Limiter controls the amount of requests to prevent overloading the system.

This will be a guided introduction to implementing these resilience patterns by means of Resilience4J in a Spring Boot microservice application.

Resilience4J#

Resilience4J is an extremely lightweight and standalone Java library aimed at implementing resilience patterns in microservices-based applications conducive to graceful failure handling and system stabilization by stopping cascading failures.

Main Features of Resilience4J:#

• Retry: It automatically retries failed operations to cater to transient failures.
• Rate Limiter: It limits the number of requests for a determined amount of time to avoid overloading the system.
• Circuit Breaker: It allows monitoring of failure and thus stops sending requests to services that are in a failure state for a given amount of time to enable recovery.
• Spring Boot Integration: It provides out-of-the-box integration with Spring Boot applications through annotations and configuration.

Add Dependency (inside order-service) and reload maven.#

		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-starter-circuitbreaker-resilience4j</artifactId>
		</dependency>

Retry using Resilience4J#

Transient failures, such as temporary network glitches and intermittent service unavailability, are common in distributed systems. In its Retry mechanism, Resilience4J allows an operation to be retried a specified number of times before failing, rather than failing outright. This boosts resilience and increases reliability.

How Does Retry Work?#

• Retries failed methods automatically, based on the configuration.
• Limits attempt to avoid an infinite retry.
• Can be combined with fallbacks to handle failures gracefully.

Example of Implementation in Spring Boot#

Put @Retry on the business logic of CreateOrder() method (inside order-service). We also need a fallback method we need to create it.

package com.codingshuttle.ecommerce.order_service.services;
import io.github.resilience4j.retry.annotation.Retry;

@Service
@Slf4j
@RequiredArgsConstructor
public class OrderService {

    private final OrderRepo orderRepo;
    private final ModelMapper modelMapper;
    private final InventoryOpenFeignClient inventoryOpenFeignClient;

    @Retry(name = "inventoryRetry", fallbackMethod = "createOrderFallBack")
    public OrderRequestDto createOrder(OrderRequestDto orderRequestDto) {
        log.info("Calling the createOrder method");
        Double totalPrice = inventoryOpenFeignClient.reduceStocks(orderRequestDto);

        OrdersEntity orders = modelMapper.map(orderRequestDto,OrdersEntity.class);

        for(OrderItemsEntity orderItem: orders.getItems()){
            orderItem.setOrder(orders);
        }
        orders.setTotalPrice(totalPrice);
        orders.setOrderStatus(OrderStatus.CONFIRMED);

        OrdersEntity savedOrder = orderRepo.save(orders);
        return  modelMapper.map(savedOrder,OrderRequestDto.class);
    }

    //fallback method
    public OrderRequestDto createOrderFallBack(OrderRequestDto orderRequestDto, Throwable throwable) {
        log.error("Fallback occurred due to : {}", throwable.getMessage());
        return  new OrderRequestDto();
    }

}

• Configure in application.properties/application.yml file (order-service)

resilience4j:
  retry:
    configs:
      default:
        maxRetryAttempts: 3
        waitDuration: 10s
    instances:
      inventoryRetry:
        baseConfig: default
        waitDuration: 200ms

Explanation:#

1. Global Default Config (default)
   • maxRetryAttempts: 3 → The system will retry up to 3 times before failing.
   • waitDuration: 10s → The system will wait 10 seconds between each retry attempt.
2. Instance-Specific Config (inventoryRetry)
   • baseConfig: default → Inherits settings from the default config.
   • waitDuration: 200ms → Overrides the global setting, making the wait time 200 milliseconds for inventoryRetry.

Output#

Run all services:

After running all services (the sequence is 1. Run Eureka Server (Discovery Service) 2. Run other clients services. 3. Last run gateway service on default port 8080)

Wait for all registered client services inside Eureka:

if your services are not fully registered before another service tries to communicate with them, you may encounter a "Service Unavailable" exception.

Hit the url http://localhost:8080/api/v1/orders/core/create-order in the Browser or Postman with the request body:

After starting Eureka and all client services, test the API using a browser (for GET requests) or Postman (for all requests). If you are trying to fetch other services you can easily fetch it. Here, we are trying to fetch inventory service data by using api-gateway (port=8080). /api/v1 ignored because of using StripPrefix=2 .

Inventory Throws an exception: The Retry logic triggers. This error will come for 3times because of “maxRetryAttempts: 3”.

That’s why the createOrderFallBack() is called. It returns a new OrderRequestDto() with the value of null.

Rate Limiter using Resilience4J#

A service may get bombarded with too many requests, causing system overload, degraded performance, or even downtime when it comes to microservices. Rate Limiter of Resilience4J helps control the number of requests allowed in a defined time frame to avoid service exhaustion and permit stability.

How Rate Limiter Works#

• Limit the number of requests in a given time window, say 10 requests in a second.
• Reject any request that exceeds the defined limit.
• Apply a fallback method to deal with rejected requests gracefully.

An Implementation Example in Spring Boot#

1. Put @RateLimiter on the business logic of CreateOrder() method (inside order-service). We also need a fallback method we need to create it.

package com.codingshuttle.ecommerce.order_service.services;
import io.github.resilience4j.ratelimiter.annotation.RateLimiter;

@Service
@Slf4j
@RequiredArgsConstructor
public class OrderService {

    private final OrderRepo orderRepo;
    private final ModelMapper modelMapper;
    private final InventoryOpenFeignClient inventoryOpenFeignClient;

    @RateLimiter(name = "inventoryRateLimiter", fallbackMethod = "createOrderFallBack")
    public OrderRequestDto createOrder(OrderRequestDto orderRequestDto) {
        log.info("Calling the createOrder method");
        Double totalPrice = inventoryOpenFeignClient.reduceStocks(orderRequestDto);

        OrdersEntity orders = modelMapper.map(orderRequestDto,OrdersEntity.class);

        for(OrderItemsEntity orderItem: orders.getItems()){
            orderItem.setOrder(orders);
        }
        orders.setTotalPrice(totalPrice);
        orders.setOrderStatus(OrderStatus.CONFIRMED);

        OrdersEntity savedOrder = orderRepo.save(orders);
        return  modelMapper.map(savedOrder,OrderRequestDto.class);
    }

    //fallback method
    public OrderRequestDto createOrderFallBack(OrderRequestDto orderRequestDto, Throwable throwable) {
        log.error("Fallback occurred due to : {}", throwable.getMessage());
        return  new OrderRequestDto();
    }

}

• Configure in application.properties/application.yml file (order-service)

resilience4j:
  ratelimiter:
    instances:
      inventoryRateLimiter:
        limitRefreshPeriod: 5s
        limitForPeriod: 1
        timeoutDuration: 1s

Explanation:#

1. Instance: inventoryRateLimiter
   • limitRefreshPeriod: 5s → The limit resets every 5 seconds. This means that after every 5 seconds, a new request will be allowed.
   • limitForPeriod: 1 → Allows only 1 request per refresh period. If multiple requests come within this 5-second window, they will be blocked or delayed.
   • timeoutDuration: 1s → If a request exceeds the limit, it will wait up to 1 second before failing. If a new slot opens within 1 second (unlikely in this case), the request may go through. Otherwise, it fails immediately after 1 second.

Output#

Run all services:

After running all services (the sequence is 1. Run Eureka Server (Discovery Service) 2. Run other clients services. 3. Last run gateway service on default port 8080)

Wait for all registered client services inside Eureka:

if your services are not fully registered before another service tries to communicate with them, you may encounter a "Service Unavailable" exception.

Hit the url http://localhost:8080/api/v1/orders/core/create-order in the Browser or Postman with the request body:

After starting Eureka and all client services, test the API using a browser (for GET requests) or Postman (for all requests). If you are trying to fetch other services you can easily fetch it. Here, we are trying to fetch inventory service data by using api-gateway (port=8080). /api/v1 ignored because of using StripPrefix=2 .

• The first request is allowed and processed.
• Any additional requests within 5 seconds are blocked and handled by the fallback method.

• After 5 seconds, the limit refreshes, and a new request is allowed.
• If a request exceeds the limit, it will wait 1 second before either being processed (if allowed) or failing with a "Too many requests" response.

That’s why the createOrderFallBack() is called. It returns a new OrderRequestDto() with the value of null.

Circuit Breaker using Resilience4J#

In the microservices environment, repeated calls to a failing service can lead to cascading or domino failures, increased response times, and resource waste. The Circuit Breaker pattern of Resilience4J stops repeated calls to an unresponsive service and allows some time for recovery.

How Circuit Breakers Function#

• Closed State: There is a healthy circuit; requests are processed as normal.
• Open State: Beyond a certain threshold of failures, the circuit "opens" and blocks calls for a specified duration of cooling.
• Half-Open State: After the cooling-off period, the system allows a limited number of test requests to determine whether the service has returned to normalcy. In the event of a successful testing run, it slots into a Closed State; otherwise, it remains Open.

An Implementation Example for Spring Boot#

1. Put @CircuitBreaker on the business logic of CreateOrder() method (inside order-service). We also need a fallback method we need to create it.

package com.codingshuttle.ecommerce.order_service.services;
import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;

@Service
@Slf4j
@RequiredArgsConstructor
public class OrderService {

    private final OrderRepo orderRepo;
    private final ModelMapper modelMapper;
    private final InventoryOpenFeignClient inventoryOpenFeignClient;

    @CircuitBreaker(name = "inventoryCircuitBreaker",fallbackMethod = "createOrderFallBack")
    public OrderRequestDto createOrder(OrderRequestDto orderRequestDto) {
        log.info("Calling the createOrder method");
        Double totalPrice = inventoryOpenFeignClient.reduceStocks(orderRequestDto);

        OrdersEntity orders = modelMapper.map(orderRequestDto,OrdersEntity.class);

        for(OrderItemsEntity orderItem: orders.getItems()){
            orderItem.setOrder(orders);
        }
        orders.setTotalPrice(totalPrice);
        orders.setOrderStatus(OrderStatus.CONFIRMED);

        OrdersEntity savedOrder = orderRepo.save(orders);
        return  modelMapper.map(savedOrder,OrderRequestDto.class);
    }

    //fallback method
    public OrderRequestDto createOrderFallBack(OrderRequestDto orderRequestDto, Throwable throwable) {
        log.error("Fallback occurred due to : {}", throwable.getMessage());
        return  new OrderRequestDto();
    }

}

• Configure in application.properties/application.yml file (order-service)

resilience4j:
  circuitbreaker:
    instances:
      inventoryCircuitBreaker:
        registerHealthIndicator: true  # Expose circuit breaker health via the actuator
        slidingWindowSize: 10          # Size of the sliding window for call recording
        slidingWindowType: COUNT_BASED # Use count-based sliding window
        minimumNumberOfCalls: 10       # Minimum number of calls before calculating failure rate
        failureRateThreshold: 50       # If 50% (5 out of 10) calls fail, the circuit goes to OPEN state
        waitDurationInOpenState: 1s   # Time the circuit stays OPEN before moving to HALF_OPEN
        permittedNumberOfCallsInHalfOpenState: 3  # Number of test calls in HALF_OPEN state
        eventConsumerBufferSize: 10    # Buffer size for event logs

Explanation:#

This configuration defines a circuit breaker for inventoryCircuitBreaker using Resilience4J to handle failures in microservices. It tracks the last 10 calls and opens if 50% fail, blocking requests for 1 second before switching to half-open mode for limited test calls. If the test calls succeed, it closes and resumes normal operation; otherwise, it reopens. This prevents repeated failures and ensures system stability.

1. This configuration enables Spring Boot Actuator to monitor the Resilience4J Circuit Breaker through the /actuator/health endpoint.
   1. Add Actuator Dependency (order-service)

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-actuator</artifactId>
		</dependency>

• Configure in application.properties/application.yml file (order-service)

# enable actuator with the help of management to check circuitbreaker
management:
  health:
    circuitbreakers:
      enabled: true
  endpoints:
    web:
      exposure:
        include: "*"
  endpoint:
    health:
      show-details: always

Explanation:#

This configuration enables Spring Boot Actuator to monitor the Resilience4J Circuit Breaker via the /actuator/health endpoint. It tracks failures, transitions between states, and recovery in microservices. The circuit breaker opens if 50% of the last 10 calls fail, remains open for 1 second, then allows 3 test calls before deciding to fully restore or reopen. Actuator provides real-time insights into its status and metrics.

Why Do We Need Spring Boot Actuator? Monitoring microservices becomes challenging without Spring Boot Actuator. It provides real-time health checks, failure tracking, and performance metrics. Without it, debugging issues, tracking circuit breaker status, and integrating with monitoring tools like Prometheus or Kubernetes require extra effort. Actuator simplifies observability, making applications more resilient and easier to maintain.

Output#

Run all services:

After running all services (the sequence is 1. Run Eureka Server (Discovery Service) 2. Run other clients services. 3. Last run gateway service on default port 8080)

Wait for all registered client services inside Eureka:

if your services are not fully registered before another service tries to communicate with them, you may encounter a "Service Unavailable" exception.

Hit the url http://localhost:8080/api/v1/orders/core/create-order in the Browser or Postman with the request body:

After starting Eureka and all client services, test the API using a browser (for GET requests) or Postman (for all requests). If you are trying to fetch other services you can easily fetch it. Here, we are trying to fetch inventory service data by using api-gateway (port=8080). /api/v1 ignored because of using StripPrefix=2 .

1. We can call that api.

1. If api call fails. ( We stop inventory-service to check the fallback )

• Ide error.

Hit the url http://localhost:8080/api/v1/orders/actuator in the Browser or Postman with the request body:

1. You can see the actuator url of circuit breaker and rate limiter etc. Without Actuator, tracking the health of components like Circuit Breakers, Rate Limiters, or Database connections becomes difficult.

How to Check Circuit Breaker Status?#

1. We can call that api. Status - > Closed. FailedCall→ 0.

• If api call fails. ( We stop inventory-service to check the fallback ). Status - > Closed. FailedCall → 1.

• If api call fails. ( We stop inventory-service to check the fallback ). Status - > Open. BufferedCall = 10.

• If api call fails. ( We stop inventory-service to check the fallback ). After 1 sec. Status - > Half-Open. BufferedCall = 1 (reset calls count). FalledCall = 1 (reset calls count).

• If api call fails ( We stop inventory-service to check the fallback ) for 3 times. Status - > Open. BufferedCall = 3. FalledCall = 3.

• If api call hit successfully ( We stop inventory-service to check the fallback ). Status - > Closed. BufferedCall = 0 (reset calls count). FalledCall = 0 (reset calls count).

Conclusion#

In this article, Retry, Rate Limiter, and Circuit Breaker patterns come into play to empower microservices resilience. Retry handles transient failures, Rate Limiter prevents overwhelming the system, and Circuit Breaker prevents repeatedly invoking a failing service, thereby ensuring stability and performance within Spring Boot applications.