AI Ops for Cloud Native Observability

Topcoder is pleased to be building a sub-community of Cloud Native application expertise for gig work and projects. This blog post and the webinar, held on May 4th, 2021 are in support of this effort.

If you are interested in growing your Cloud Native development skills and in taking on gig opportunities, please consider participating in our challenge series.

If Monitoring is knowing the known unknowns,Observability provides an ability to ponder unknown unknowns on the go.

Introduction

We are aware of Monolith Architecture with multiple layers in designing the applications. Now, we find that they are difficult to manage new modules and handle complex modules, which eventually led to architectural erosion. To circumvent complex architectural erosion problem of Monolith Architecture, Micro services architectural style emerged and were containerized.

All Cloud native applications deployed in Cloud are microservices-based and built in a container environment deliver significant business value. Elastic infrastructure on clouds makes these applications scalable and can enhance and modify applications easily.

Microservices are ephemeral by nature and the containers tightly coupled to its logs deprives a developer from viewing the logs when the Microservices are deleted, re-created, re-started, and moved from one node to another. Highly dynamic & distributed nature of Cloud native apps spread across different layers (UI, API, Service, DB and infrastructure) makes it difficult for developers to find the root-cause of problems in apps. At times, it is harder to trace a request that passes through multiple layers of Microservices applications fulfilling a client’s needs. Most often-complex problems appears at the intersections of (UI, API, Service, DB etc.) that go undetected.

Importance of Architectural concerns in Cloud Native patterns constitutes Reliability, Scaling & Observability for building best-practised containerised applications.

Observability

Simply put, Observability is way to understand complex system in detail. However, in control theory, Observability is defined as a measure of how well internal states of a system can be inferred from knowledge of its external outputs. Gartner Infrastructure Operations and Cloud Management Reports suggest through 2022, 80% of organizations adopting cloud services without using a performance-focused approach for dependency mapping will experience a decrease in service quality levels. By 2023, 40% of product and platform teams will use AIOps for automated change risk analysis in DevOps pipelines, reducing unplanned downtime by 20%.

If monitoring provides instrumentation to collect systems data and help in quickly responding to errors and issues that occur, Observability is the practice of instrumenting those systems with tools to gather actionable insights on data—which not only helps identify errors or issues but generates insights into why they occurred. This form of information is far more useful when you want to deliver applications that are more stable and reliable.

Full Stack Observability

Ensuring principles of full stack observability from front end UI to Infra, can deliver excellent customer experiences, despite complexities and the distributed nature of application and infrastructure landscape.

Full stack Observability relies on ingesting telemetry data (Metrics, Events, Logs & Traces) at a central common database. Feature engineering of the data collected and use of powerful AI algorithms establish relationship between the system’s metadata and the business context:

Metrics – the first step for full stack observability is collection of metrics and their storage in an inexpensive location, without much overhead, to measure the overall health of system with quick analysis.

Events – a higher-level of abstraction of transaction data, organization around events aids in significant point analysis. Eg: Calls made to Database & transaction events.

Logs - detailing the data flow with time stamp helps when entering debugging mode. Extracted data from log files helps in establishing business context.

Traces - provides specific insight in customer journey through the system; helps in identifying bottlenecks and errors to optimize, measures latency of individual calls in a distributed architecture.

Distributed tracing systems collect data as requests which goes from one service to another, recording each segment of the journey as span. These spans contain important details about each segment of the request, combined into one trace. Traces establish the business context correlating with centralised data and helps provide a better understanding of complex systems.

Full Stack observability with additional building blocks—Application Performance Monitoring (APM) and Real User Monitoring (RUM)—which use telemetry data to generate insights and discover hidden behaviours from unknown permutations could lead to new actions that were not envisaged.

AI Ops

With microservices finding their place in often complex and evolving Cloud-native architectures, adding new services, refactoring old ones, spinning up, and shutting down ephemeral application instances—it is next to impossible to maintain mental map and keep up with changes. When all the data is centrally located, AI Ops aids in identifying patterns, providing proactive analysis, and enabling detection of anomalies and event correlations that are not easily identifiable by humans. Parameters may change by the day, hour or minute. Application of powerful AI algorithms, made possible in organizations that acquire, aggregate, analyse and act on Metadata, enhances the ecosystem of observability and expands visibility of applications on cloud, improving reliability. AI Ops helps when situations and opportunities are present in systems that are good candidates for task automation, release automation and self-healing. This approach can engage end users on business context, provide advice, and take action on incidents that occur in the system.

Challenges

Ever increasing demands of customer expectations lead to greater complexity of Cloud native applications. Skill gaps exist (and are to be expected) in all developers; they are asked to develop multiple skills like networking concepts, data base expertise, and cloud architecture. At this stage there are too many tools in the market to consolidate and make observability work out of the box.

Benefits

Deeper understanding of system through data helps in identifying & resolving complex issues in record time thus reduce Mean time to resolution (MTTR). Earlier detection & diagnosis by predicting enhances self-heal capabilities of the systems. Access to data & failure scenarios gives an opportunity to test the robustness of the system. Developers will be able to innovate & chaos test with confidence, better understanding of systems will not let the systems break

Future

The future of Observability shall have deep integration with CI/CD pipelines, increased adoption of AI/ML models in the form of AI Ops for observability with prescriptive analytics, and unification of all observability & business analytics tools. Application Performance Monitoring products will emerge as the most preferred to enable a fully mature observability solution.

If you’re interested in learning and riding the wave of AI Ops for Cloud Native Observability into the future, take a look at our skill-builder challenges and look for gig opportunities in this area that will be coming in the weeks and months ahead!