Research Question(s): How can we dynamically identify the best threading-model for a microservice under varying loads? How can we dynamically modify microservices to use the best thread-model given the current load to the service?
Key Contributions: They create a taxonomy of threading models which includes 8 different models using a combination of: 1) synchronous (one thread per request) vs asynchronous (no association between threads and requests), 2) in-line (one thread for entire request processing) vs dispatch (a thread for network processing and a seperate worker thread for handling the rpc), and 3) poll-based (a thread consistently loops polling for new requests) vs blocking (threads wait until a request arrives). They show that no model is always the best: the characteristics of services, size of thread pool, and load impact which model is best in that circumstance. They build µTune which dynamically monitors mid-tier microservices to change the threading model based on the current load. This work only focused on mid-tier microservices which propogate requests further trough the system (often to leaves that do major processing before responding) then aggregate results before responding upstream.
Opportunities for future work: Currently, the developer must choose between synchronous vs asynchronous when using µTune as their functions differ. Asynchronous will always be better than the best synchronous model, but is more difficult to program. Future work could look into automatic methods of converting synchronous code into an asynchronous version. µTune has an offline training phase where they learn the best threading model under various loads. Microservice applications are continously updated with services being changed and new services being added to the architecture. Future work could expand upon the training phase to learn the best threading model as the system is running (and potentially on only updated services).
Presenter: Darby Huye