Resiliency Patterns with Hystrix

Overview

For improving resiliency, you need to utilize Netflix Hystrix and finish following steps:

• Set up Hystrix in Application Level
• Implement Different Resiliency Patterns
  • Circuit Breaker (熔断)
  • Fallback Processing (降级)
  • Bulkheads (隔舱／隔离)

---

Set up Hystrix in Application Level

Before implementing resiliency patterns, you need to enable Hystrix in application level.

build.gradle

dependencies {
    compile group: 'org.springframework.cloud',  name: 'spring-cloud-starter-hystrix',  version: '1.4.4.RELEASE'
    compile group: 'com.netflix.hystrix',        name: 'hystrix-javanica',              version: '1.5.9'
}

Application.java

@SpringBootApplication
@EnableCircuitBreaker        // Enable Hystrix for your service
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

---

Implement Different Resiliency Patterns

There are 3 client resilency patterns which will be implememented in there:

• Circuit Breaker (熔断)
• Fallback Processing (降级)
• Bulkheads (隔舱／隔离)

---

Circuit Breaker (熔断)

Definition: Circuit Breaker will monitor all the calls. If a call takes too long, the circuit breaker will stop and kill that call.

You can put the circuit breaker (Hysterix) into 2 places:

• Between service and database
• Between service and service

Circuit Breaker between service and database

You can annotate a function which calls the database with the @HystrixCommand annotation.

LicenseService.java

@HystrixCommand
public List<License> getLicensesByOrg(String organizationId) {
    return licenseRepository.findByOrganizationId(organizationId);
}

Circuit Breaker between service and service

You can annotate a function which calls another service with the @HystrixCommand annotation.

LicenseService.java

@HystrixCommand
public Organization getOrganization (String organizationId) {
    return organizationRestClient.getOrganization(organizationId);
}

Customize the time out of a circuit breaker

The default time out of a circuit breaker is 1 second (1000 milliseconds). You can also customize it with the parameter in the @HystrixCommand annotation.

LicenseService.java

@HystrixCommand(
    commandProperties={@HystrixProperty(
        name="execution.isolation.thread.timeoutInMilliseconds", 
        value="12000")}                                            // define the timeout as 12000 milliseconds
)
public List<License> getLicensesByOrg(String organizationId) {
    return licenseRepository.findByOrganizationId(organizationId);
}

In this code, it customized the time out to 12000 milliseconds (12 seconds).

---

Fallback Processing (降级)

Definition: When a service call fails, the alternative action will be executed.

You can add the fallbackMethod parameter into the @HystrixCommand annotation. The value of the fallbackMethod parameter is the name of the function for the fallback process.

LicenseService.java

@HystrixCommand(fallbackMethod = "buildFallbackLicenseList")              // the name of the function will be called if the call from Hystrix fails.
public List<License> getLicensesByOrg(String organizationId) {
    return licenseRepository.findByOrganizationId(organizationId);
}

private List<License> buildFallbackLicenseList(String organizationId) {   // the concrete function for the fallback
    /* omit the implementation*/
}

---

Bulkheads (隔舱／隔离)

Definition: The bulkhead pattern segregates remote resource calls in their own thread pools so that a single misbehaving service will not take down the entire application.

To achive the bulkhead, you need to add several parameters into the @HystrixCommand annotation:

• threadPoolKey
  • The signal to Hystrix that you want to set up a new thread pool.
  • The unique name of the new thread pool.
• coreSize
  • The size of the new thread pool.
• maxQueueSize
  • The size of the queue in front of the thread pool to hold the requests when the thread pool is full.
  • Different values decide different implementations of the queue:
    • If maxQueueSize = -1, the Java SynchronousQueue will be used.
    • If maxQueueSize > 1, the Java LinkedBlaockingQueue will be used.
  • The size of the queue can be dynamical.
    • Use the queueSizeRejectionThreshold attribute.
    • The queueSizeRejectionThreshold attribute can be set only if the maxQueueSize > 0.

LicenseService.java

@HystrixCommand (threadPoolKey = "licenseByOrgThreadPool",              // define the unique name of the new thread pool
                 threadPoolProperties =
              { @HystrixProperty(name = "coreSize",     value="30"),    // the size of the new thread pool
                @HystrixProperty(name = "maxQueueSize", value="10")}    // the size of the queue in front of the thread pool to hold the requests when the thread pool is full
public List<License> getLicensesByOrg(String organizationId) {
    return licenseRepository.findByOrganizationId(organizationId);
}

---

References

• 服务熔断、降级、限流、异步RPC - HyStrix