Onboarding : Azure Monitoring and Resource Management

Topics

Scenarios

  • Monitoring resources
  • Managing resources via
    • Resources groups
    • Tags
    • Policies
    • RBAC

Extension compured to agent
Extensions have to be installed to manage agents whenever possible.
Log Analytics extension for windows and linux installs the log analytics agent on azure vm. Azure monitor dependancy extension for windows and linux installs dependancy agent on azure vm.

Key concepts

  • Azure Monitor
    • Azure disgnostic extension
    • Azure log analytics agent
    • Windows agents
    • Linux agents
    • Telegraf agent
    • Dependency agent
  • Resource Management
    • Resource group
    • Tag
    • Policy
    • RBAC

Monitoring concepts

Each project need a holistic monitor strategy.

Scenario: A financial organization is moving its systems to Azure, with a mixture of IaaS and PaaS services. In its previous environment, the organization had several instances where systems failed or issues arose. There was an extended delay to engage resources and resolve the issues. This situation affected customers’ ability to access their accounts, and it influenced satisfaction.
The organization wants to design a monitoring strategy that encompasses all the solutions that it uses. There should also be insights and alerting into the accumulated log data. The organization wants to quickly identify and minimize the impact if systems fail in the future.

Continuous monitoring strategy

  • it can improve the ability to identify issues within application
  • it can help to improve customer experience
  • to monitor performance of infrustructure and application
  • monitor security risks and suspicious activity
  • collect information on issues and analyse and then respond
  • in long run, your organization will become more productive, cost-effective, secure, and competitive.
  • why monitor applications
    • to improve application health
    • configure alert and automated response to deal with issues
    • to improve the development lifecycle, we can use monitoring, therefore we would be ready when we go to production
  • why monitor infrustructures
    • because of issues that could render the entire infrastructure unavailable
    • because of threats to security (these can have impact on productivity, financial loss, damage organization reputation )
    • risk of suspicious user account
    • malicious ip addresses
    • create automated response to alerts with playbooks and webhook.
    • learn from issues, strengthen protection, build an improved infrastructure
Availability of applications
even in development phase monitoring can help developers

Available monitoring options

  • Azure Monitor
  • Azure Security Center
  • Azure Sentinel

Azure Monitor

1.Azure Diagnostics Extension
  • An agent in azure monitor
  • Collects monitoring data from guest operating systems of azure compute resources include virtual machines.
  • We don’t pay for the extension but we pay for data ingestion
  • Data destination is azure storage account or other data sinks (additional destinations).

Scenarios

  • Collect guest metrics into azure monitor metrics (collect logs from middle-tier/vm and transfer to azure monitor)
  • Send guest logs and metrics to azure storage for archiving
  • Send guest logs and metrics to azure event hubs to send outside of azure

Data collected

Windows (WAD)Linux (LAD)
Windows event logsSyslogs
Performance countersPerformance counters
IIS logsLog files
Application logs
.Net eventsource logs
Manifest based ETW logs
Crash dumps log
File based logs
Agent disgnostic logs

Sources

2.Azure Log Analytics Agent
Azure Diagnostics ExtensionAzure Log Analytics
Only Azure VMsAzure/other clouds/on-prem VMs,
Sends data to azure storage, azure monitor metrics (only win), and event hubs.collects data to azure monitor logs.
is required for solutions, azure monitor for vms, azure security center, und…
3.Windows Agents

Coming soon…

4.Linux Agents

Coming soon…

5.Dependancy Agent

Coming soon…

Azure Monitor Logs

  • powerfull query language for joining data from multiple tables
  • with virtual presentation
  • extracts valuable information about infrastructure from log data
  • monitor health of the services
  • it’s for collecting and analyzing telemetry
  • helps to have max performance and availability for cloud application & on-prem
  • it collects data in azure monitor
  • azure monitor collects two fundamental types of data
    • metrics: tell you how the resource is performing and consuming other resources
    • logs: when the resource created and modified
Diagram of Azure Monitor's architecture displaying the sources of monitoring data, the data stores, and functions performed on the data.
high-level view of Azure Monitor[Source]
  • azure monitor is automatic system, it collects data as soon as the resource is created
  • azure monitor data can be extended
    • enabling diagnostics:
    • adding an agent: it’s for vms via installing log analytics agent and sending data to log analytics workspace.
    • custome code via Data Collector API
  • Logs (recommended for analyzing)
    • time-stamped information about changes made to resource
    • types are numeric, text, events
    • You can store metric data in logs to combine them with other monitoring data for analysis
    • You log data from Azure Monitor in a Log Analytics workspace
    • Azure provides an analysis engine and a rich query language (Kusto)
  • metrics (recommended for alerting)
    • Metrics are numerical values that describe some aspect of a system at a point in time
    • The metrics are collected at regular intervals and are useful for alerting because of their frequent sampling.
    • Metrics are stored in a time-series database
    • Metrics are suited for alerting and fast detection of issues

Source: https://docs.microsoft.com/en-us/learn/modules/analyze-infrastructure-with-azure-monitor-logs/1-introduction

Monitoring Best Practices

Sources

Azure Security Center

Source

Resource Management [source]

Scenario: Company has been moving to the cloud. This movement happened organically across different departments, and resulted in a lack of awareness of what’s already been created and where everything is. There’s no ability to easily determine who owns which resources. There’s no enforcement of standards for things like resource names, resource sizes, and geographic locations. There’s also been several instances where critical resources were inadvertently deleted, causing business-critical outages.

  • Protect business critical resources against deletion
Resource group [source]
  • logical container for resources
  • Resources can be a member of just one resource group
  • Many resources can be moved between resource groups
  • Resource group can not be nested
  • By deleting a resource group all its resources are deleted as well e.g this kind of life cycle is useful for non-production
Resource tag
Policy
Role-based access control (RBAC)

Azure Activity Log

  • activity log is an insight into subscription-level events (Retention period of 90 days)
  • each resource has its own activity log, that is a part of subscription-level activity log
  • each resource activity log can be connected to only one log analytics workspace
  • one log analytics workspace can be connected to the activity log of multiple subscriptions in a same tenant.

Connect Activity Log to Log Analytics Workspace

Select Log Analytics Workspace > Azure Activity Log / VM / Storage/.. menu > Select Resource / subscription > Connect / disconnect

Benefit of connecting Activity Log to Log Analytics
  • consolidate the activity logs of multi subscriptions in one place for analysis.
  • Store Activity Log enteries for more than 90 days.
  • Correlate Activity Log data with Azure Monitor data.
  • Use Log queries for complex analysis and get deep insight on Activity log.

Note
Each subscription has only one Activity Log.
Each Activity Log can be connected to only one Log Analytics Workspace.
One Log Analytics Workspace can be connected to the Activity Log of multiple subscription in a same tenant.

Connect an Activity Log to a Log Analytics workspace

Variante 1 (form log analytics blade)
Go to the Log Analytics Workspace > Select one of the created workspaces > From the blade > Azure Activity Log > Select one Subscription > Connect | Disconnect

Workspaces
Connect Workspaces

Variante 2 (from activity log blade)
Select a resource > Select Logs / Diagnostic settings menu > Select a Log Analytics Workspace.

Source


Resources


You owe your dreams your courage.

Koleka Putuma


%d bloggers like this: