Machine Learning and AI, Technology
Using AI In DevOps – Some Use Cases
DevOps isn’t a new idea; IT teams the world over have adopted its principles for years now. However, given the speed at which processes, technologies and tools are evolving, it’s becoming increasingly difficult to cope with effectively implementing DevOps principles. Besides, businesses are ramping up the pressure on their IT teams, demanding more continuity in integration and delivery – at the click of a button.
As CI/CD at scale in real-time becomes increasingly harder to achieve, the best solution to keep pace is Data Science. Here are some use cases where the addition of Machine LEarning t othe mikx will help the DevOps cause immensely.
Track application delivery
Activity data from DevOps tools (such as JIRA, Git, Jenkins, SonarQube, Puppet, Ansible, etc.) provides visibility into the complete application delivery process. You can use machine learning to uncover anomalies in that data – large code volumes, long build times, slow release rates, late code check-ins – to discover many of the ‘wastes’ of the software development.
Review software testing efficiency
Machine Learning can check QA results and identify novel errors by analyzing the output from testing tools. For example, ML algorithms can provide information on common or typical defects, and malfunction predictions or patterns.
Secure application delivery
You can apply Machine Learning to analyze the user behaviour of the DevOps team and identify anomalies that may represent harmful activities.
The ultimate goal of DevOps is complete automation across the project lifecycle. While complete automation is a distant reality for now, we can strive to automate as much as possible, and not just within a single phase or tool.
We use AI to remove data silos within the toolchain, which creates a conducive environment to automate analysis, log and metric data. Eg. correlation of all relevant data within a toolchain. The benefits of automation include greater speed, more accurate root cause analysis and predictive insights that are gained from the entire toolchain, rather than just one individual tool or datasource.
Collaboration is a cornerstone (and an important one, at that) of the DevOps paradigm. It is crucial for there to be a free flow of information regarding the best way to run applications and systems, between the IT, engineering and operations teams. This in turn means seamless communication and collaboration.
We can use AI for collaboration within a DevOps team by providing a single view to all project stakeholders, from which relevant toolchain data can be accessed. AI also captures knowledge as it is generated, about how systems and applications should run. ML algorithms then display this knowledge at the times they are needed, for eg. when alerts or anomalies are detected.
Documentation is difficult to maintain and must be continually updated. AI could also play a huge role in software documentation. The same kind of natural language processing used by Google to automate news writing could be used to document feature change lists, API technical details and processes used by DevOps teams.
Pattern discovery in logs provides a highly effective and automated way of discovering new knowledge in logs and thus making ultimately boring and routine logs into actions. For example, a log file might be discovered to contain a repeating pattern of connections from a seemingly diverse set of source IP addresses. When presented, this pattern might be interpreted as a new exploitation tools, covertly tried on a company network. It can then lead to a set of actions by a security team and even community at large.
Analysis of trends and summary
Summaries and trends are a common result of log analysis. A long log file might be summarized into a brief “Top 10 Attacks” or “Top Suspicious IP Addresses” or an infinite number of other useful summaries. Often, such summary view will prompt an action. For example, it might become obvious from a “Top Bandwidth Users” report that the top three users in the company utilize 90% of available bandwidth.
This can quickly lead to a disciplinary action, especially if such bandwidth is used to share files on P2P or to download non-work related materials. Similarly, an observation of a router CPU utilization log over a long period of time might reveal periods of unusually high activity, leading to an investigation possibly discovering attacker communication with a compromised system.
AI in DevOps is a new and exciting application of Data Science, and one that we’re actively tracking. Stay tuned to our blog for more!