[REF] API Design and What’s new with Open API? (Google Cloud Next ’17)

Published on Mar 8, 2017

Apigee and Google are active participants in developing the third draft of the Open API Initiative (OAI) Spec. In this video, hear what’s new with the the Open API Specification (OAS) — the vendor-neutral, portable, and open specification for providing technical metadata for REST APIs as well as latest trends in API design including topics such as Hypermedia and how the evolving OAI spec influences common patterns. Not just trends – this video is filled with real world examples.

Missed the conference? Watch all the talks here: https://goo.gl/c1Vs3h
Watch more talks about Connected Business Platform here: https://goo.gl/CtHNJm

[REF] SWIM: scalable weakly-consistent infection-style process group membership protocol

Abstract:

Several distributed peer-to-peer applications require weakly-consistent knowledge of process group membership information at all participating processes. SWIM is a generic software module that offers this service for large scale process groups. The SWIM effort is motivated by the unscalability of traditional heart-beating protocols, which either impose network loads that grow quadratically with group size, or compromise response times or false positive frequency w.r.t. detecting process crashes. This paper reports on the design, implementation and performance of the SWIM sub-system on a large cluster of commodity PCs. Unlike traditional heart beating protocols, SWIM separates the failure detection and membership update dissemination functionalities of the membership protocol. Processes are monitored through an efficient peer-to-peer periodic randomized probing protocol. Both the expected time to first detection of each process failure, and the expected message load per member do not vary with group size. Information about membership changes, such as process joins, drop-outs and failures, is propagated via piggybacking on ping messages and acknowledgments. This results in a robust and fast infection style (also epidemic or gossip-style) of dissemination. The rate of false failure detections in the SWIM system is reduced by modifying the protocol to allow group members to suspect a process before declaring it as failed – this allows the system to discover and rectify false failure detections. Finally, the protocol guarantees a deterministic time bound to detect failures. Experimental results from the SWIM prototype are presented. We discuss the extensibility of the design to a WAN-wide scale.
Date of Conference: 23-26 June 2002
Date Added to IEEE Xplore: 10 December 2002
ISBN Information:
Print ISBN: 0-7695-1597-5
Print ISBN: 0-7695-1101-5
INSPEC Accession Number: 7396105
Publisher: IEEE
Conference Location: Washington, DC, USA, USA
Reference:
Citation:
@INPROCEEDINGS{1028914,
author={A. Das and I. Gupta and A. Motivala},
booktitle={Proceedings International Conference on Dependable Systems and Networks},
title={SWIM: scalable weakly-consistent infection-style process group membership protocol},
year={2002},
pages={303-312},
keywords={computer network reliability;protocols;wide area networks;workstation clusters;PC cluster;SWIM;WAN;deterministic time bound;distributed peer-to-peer applications;experimental results;failure detection;generic software module;heart-beating protocol;membership update;network loads;performance;periodic randomized probing protocol;piggybacking;ping messages;process group membership protocol;response times;scalable weakly-consistent infection-style protocol;Application software;Computer crashes;Condition monitoring;Delay;Frequency;Large-scale systems;Peer to peer computing;Personal communication networks;Protocols;Robustness},
doi={10.1109/DSN.2002.1028914},
month={},}

Solving Service Discovery by Richard Roger, Nearform

Solving Service Discovery – Richard Roger, Nearform

The microservice architecture is a powerful way to structure large scale Node.js systems. Microservices bring many benefits, enabling rapid development, making continuous delivery easier, and making fine-grained scaling practical. But this architecture is a distributed system, and brings with it all the associated challenges of such systems, not least of which is the need for individual services to find each other. Deciding how to solving the service discovery problem is a key decision point for any architect building microservice systems.

A common approach is to use a service registry. Unfortunately service registries bring their own problems, such as ensuring consistency while scaling, or increasing deployment complexity with the need to run agents. All this could be forgiven, but for a greater sin – registries encourage strong coupling between microservices by providing a concept of serviceidentity. The very purpose of the registry, service discovery, is poisonous to one of the key benefits of microservices – that the system is dynamic and flexible.

There is a better way. Services can find each other dynamically, using gossip protocol algorithms. Highly efficient algorithms such SWIM (Scalable Weakly-consistent Infection-style Process Group Membership Protocol) mean that microservices can do away with service discovery altogether, relying instead on an constantly evolving, dynamic map of their local network. By pushing intelligence to the edges of the network, not only does this approach make failure detection orders of magnitude faster (no more waiting for heartbeats to timeout), but it also makes deployment much easier, as the network configures itself.

This talk will examine the use of the SWIM algorithm to build a Twitter clone in Node.js, one microservice at a time.

About Richard Rodger
Richard Rodger is co-CEO and co-founder of nearForm. He is an expert and thought leader in next-generation cloud and mobile technologies, with a current focus on Node.js and microservices. His book Mobile Application Development in the Cloud (Wiley, 2010) is one of the first major works on the subject.
Richard was previously CTO of FeedHenry, a mobile application platform provider that was acquired by RedHat for €63.5m in 2014.
Since Richard co-founded nearForm in 2011, the company has become the world’s largest Node.js and microservices consulting company.

Richard has long been an active member and influencer of the global open source software (OSS) community, specializing in OSS for enterprise. As CTO of nearForm, he has placed OSS at the heart of how the company works. Most recently, he created Seneca.js, a microservices tool kit for Node.js that is a key component of nearForm’s software development and delivery arsenal.
The Tao of Microservices (Manning), Richard’s new book, will be published in 2016.

Spring Cloud on AWS

Recorded at SpringOne Platform 2016.
Speaker: Agim Emruli
Slides: http://www.slideshare.net/SpringCentr…

Spring Cloud allows to implement cloud native application across different cloud platforms. Spring Cloud has also a dedicated support for running applications on the Amazon Web Service platform. This talk will introduce common patterns and best-practices to run the application the AWS cloud and how to use the platform provided services efficiently.

[REF] “The art of service discovery at scale” by Nitesh Kant

Nitesh Kant @NiteshKant / Netflix

Whether it is a simple DNS lookup or a complex dedicated solution, service discovery is the backbone of any microservices architecture and an immature solution can soon turn into an achilles’ heel.

Nitesh Kant in this talk will introduce the concept of service discovery and various use cases it solves in a complex service based architecture. He will then be introducing Netflix’s Eureka (https://github.com/Netflix/eureka); a highly-available, multi-datacenter aware service discovery solution built from scratch, it’s architecture and how it is unique in this space, by favoring Availability over Consistency in the wake of network partitions.

Nitesh Kant is an engineer in Netflix’s Cloud Platform team, working on their Inter Process Communication stack. He is the author of RxNetty (https://github.com/ReactiveX/RxNetty) which forms the core of Netflix’s IPC stack. He is also a contributor to other Open source projects like Eureka

(https://github.com/Netflix/eureka) and Karyon (https://github.com/Netflix/karyon)

[REF] Powering Microservices with Docker, Kubernetes, Kafka and MongoDB

Andrew Morgan / Principal Product Marketing Manager on MongoDB

Organisations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. This session introduces you to technologies such as Docker, Kubernetes & Kafka which are driving the microservices revolution. Learn about containers and orchestration – and most importantly how to exploit them for stateful services such as MongoDB.

[REF] Best practices for orchestrating the Cloud with Kubernetes – Google I/O 2016

Carter Morgan  / Developer Programs Engineer at Google

In the last decade, user demand for always-on applications have grown exponentially. Many developers choose application patterns to meet this need, but what about the infrastructure needed to support these ever growing demands? In this talk, we introduce you to the next level of automation using hands-on examples of industry standard tooling like Docker, a container format, and Kubernetes, a distributed automation platform. We cover the basics of modern day applications and how design patterns like microservices drive the need for more robust infrastructure. Then we cover packaging and distributing apps using Docker. Finally, we up our game to running applications on Kubernetes. By the end of this talk you’ll have the knowledge needed to excel at scale.

[REF] Microservice Based Tool Support for Business Process Modelling

The rise of micro services as architectural pattern creates a bunch of interesting opportunities for software architectures of modelling editors and additional services. Main advantages are the scalability in collaborative distributed scenarios and enhanced possibilities regarding service development and operation. Throughout this article, we will illustrate how modelling editors and additional services can be build based on micro services. Our tooling will focus on business process modelling. We will also strive to highlight how architectures of this kind can enact collaborative modelling techniques, increase reuse of utilized service components and improve their integration into lightweight user interfaces, for example in mobile devices.

Published in: Enterprise Distributed Object Computing Workshop (EDOCW), 2015 IEEE 19th International

Authors: Sascha Alpers ; FZI Forschungszentrum Inf., Karlsruhe, Germany ; Christoph Becker ; Andreas Oberweis ; Thomas Schuster
PDF Version: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7310673

[REF] Developing Fault Tolerant Distributed Systems by Refinement

Distributed systems are usually large and complex systems composed of various components. Systemcomponents are subject to various errors. These failures often require error recovery to be conducted at architectural-level. However, due to complexity of distributed systems, specifying fault tolerance mechanisms at architectural level is complex and error prone. In this paper, we propose a formal approach to specifying components and architectures of fault tolerant distributed and reactive systems. Our approach is based on refinement in the action system formalism – a framework for formal model-driven development of distributed systems. We demonstrate how to specify and refine fault tolerant components and complex distributed systems composed of them. The proposed approach provides designers with a systematic method for developing distributed fault tolerant systems.

Published in: Software Engineering Advances (ICSEA), 2010 Fifth International Conference on

Author: Elena Troubitsyna ; Dept. of IT, Abo Akademi Univ., Turku, Finland

PDF Version: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5615742

[REF] Efficient Tracing and Performance Analysis for Large Distributed Systems

Distributed systems are notoriously difficult to implement and debug. One important tool for understanding the behavior of distributed systems is tracing. Unfortunately, effective tracing for moderndistributed systems faces several challenges. First, many interesting behaviors in distributed systemsonly occur rarely, or at full production scale. Hence we need tracing mechanisms which impose minimal overhead, in order to allow always-on tracing of production instances. Second, for high-speed systems, messages can be delivered in significantly less time than the error of traditional time synchronization techniques such as network time protocol (NTP), necessitating time adjustment techniques with much higher precision. Third, distributed systems today may generate millions of events per second systemwide, resulting in traces consisting of billions of events. Such large traces can overwhelm existing trace analysis tools. These challenges make effective tracing difficult. We present techniques that address these three challenges. Our contributions include (1) a low-overhead tracing mechanism, which allows tracing of large systems without impacting their behavior or performance (0.14 ¿s/event), (2) a post hoc technique for producing highly accurate time synchronization across hosts (within 10/ts, compared to between 100 ¿s to 2 ms for NTP), and (3) incremental data processing techniques which facilitate analyzing traces containing billions of trace points on desktop systems. We have successfully applied these techniques to two distributed systems, a cooperative caching systemand a distributed storage system, and from our experience, we believe our techniques are applicable to other distributed systems.

Published in:

Modeling, Analysis & Simulation of Computer and Telecommunication Systems, 2009. MASCOTS ’09. IEEE International Symposium on

AUTHORS: Eric Anderson ; Christopher Hoover ; Xiaozhou Li ; Joseph Tucek

PDF VERSION: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=536615