In the Cost Management + Billing service blade > Cost analysis > there’s a possibility for filter after tag and if the environment like dev, Prod has been considered as tag. We can filter here.
Event filter after services.
Category: Azure Cloud
Topics
Related topics
Available monitoring options
- Azure Monitor: this codument go through the important aspects of it.
- Azure Security Center
- Azure Sentinel
Monitoring concepts
Each project need a holistic monitor strategy.
Scenario: A financial organization is moving its systems to Azure, with a mixture of IaaS and PaaS services. In its previous environment, the organization had several instances where systems failed or issues arose. There was an extended delay to engage resources and resolve the issues. This situation affected customers’ ability to access their accounts, and it influenced satisfaction.
The organization wants to design a monitoring strategy that encompasses all the solutions that it uses. There should also be insights and alerting into the accumulated log data. The organization wants to quickly identify and minimize the impact if systems fail in the future.
Continuous monitoring strategy
- it can improve the ability to identify issues within application
- it can help to improve customer experience
- to monitor performance of infrustructure and application
- monitor security risks and suspicious activity
- collect information on issues and analyse and then respond
- in long run, your organization will become more productive, cost-effective, secure, and competitive.
- why monitor applications
- to improve application health
- configure alert and automated response to deal with issues
- to improve the development lifecycle, we can use monitoring, therefore we would be ready when we go to production
- why monitor infrustructures
- because of issues that could render the entire infrastructure unavailable
- because of threats to security (these can have impact on productivity, financial loss, damage organization reputation )
- risk of suspicious user account
- malicious ip addresses
- create automated response to alerts with playbooks and webhook.
- learn from issues, strengthen protection, build an improved infrastructure
Azure Monitor agents
Azure Diagnostics Extension
- An agent in azure monitor
- Collects monitoring data from guest operating systems of azure compute resources include virtual machines.
- We don’t pay for the extension but we pay for data ingestion
- Data destination is azure storage account or other data sinks (additional destinations).
Scenarios
- Collect guest metrics into azure monitor metrics (collect logs from middle-tier/vm and transfer to azure monitor)
- Send guest logs and metrics to azure storage for archiving
- Send guest logs and metrics to azure event hubs to send outside of azure
Data collected
| Windows (WAD) | Linux (LAD) |
|---|---|
| Windows event logs | Syslogs |
| Performance counters | Performance counters |
| IIS logs | Log files |
| Application logs | |
| .Net eventsource logs | |
| Manifest based ETW logs | |
| Crash dumps log | |
| File based logs | |
| Agent disgnostic logs | |
Sources
Azure Log Analytics Agent
| Azure Diagnostics Extension | Azure Log Analytics |
|---|---|
| Only Azure VMs | Azure/other clouds/on-prem VMs, |
| Sends data to azure storage, azure monitor metrics (only win), and event hubs. | collects data to azure monitor logs. |
| is required for solutions, azure monitor for vms, azure security center, und… |
Windows Agents
Coming soon…
Linux Agents
Coming soon…
Dependancy Agent
Coming soon…
Azure Monitor Logs
- powerfull query language for joining data from multiple tables
- with virtual presentation
- extracts valuable information about infrastructure from log data
- monitor health of the services
- it’s for collecting and analyzing telemetry
- helps to have max performance and availability for cloud application & on-prem
- it collects data in azure monitor
- azure monitor collects two fundamental types of data
- metrics: tell you how the resource is performing and consuming other resources
- logs: when the resource created and modified
- azure monitor is automatic system, it collects data as soon as the resource is created
- azure monitor data can be extended
- enabling diagnostics:
- adding an agent: it’s for vms via installing log analytics agent and sending data to log analytics workspace.
- custome code via Data Collector API
- Logs (recommended for analyzing)
- time-stamped information about changes made to resource
- types are numeric, text, events
- You can store metric data in logs to combine them with other monitoring data for analysis
- You log data from Azure Monitor in a Log Analytics workspace
- Azure provides an analysis engine and a rich query language (Kusto)
- metrics (recommended for alerting)
- Metrics are numerical values that describe some aspect of a system at a point in time
- The metrics are collected at regular intervals and are useful for alerting because of their frequent sampling.
- Metrics are stored in a time-series database
- Metrics are suited for alerting and fast detection of issues
Monitoring Best Practices
Sources
What’s alert?
- A proactive notificaion when an important notification found in monitoring (response to incidents).
- Alert is raised before customer identifies and addresses an issue.
- Unified alerts are managed by log analytics and application insights.
- The previous type of alert are classic alerts.
- Alerts are raised for matrics and logs. Ex. Metric values, Log search queries, Activity log event, Health of the underlying azure platform, test for website availability.
Alert states
| Alert state | Description |
|---|---|
| New | Issue has been detected but has not been reviewed. |
| Acknowlaged | Administrator has reviewed the alert and started working on it. |
| Closed | Issue has been resolved. |
NOTE : The state changes are stored in alert’s history.
Alert states are independent of Monitor condition (fired or resolved).
Different type of reaction to an incident
- Alerts & alert rules & action group
- Classic alerts
- Metric alerts
- Log alerts
- Activity log alerts
- common alert schema
- Smart groups (aggregation of lalerts base on machine learning algorithms)
- Auto scale
- Change analysis
Permission and privilages
We can use the Role-based access control (RBAC) at different levels
- Subscription Level
- Resource Level
The RBACs which are available for Azure Monitor service are the following:
- Monitoring Contributor
- Monitoring Metrics Publisher
- Monitoring Reader
Ex. A user with the Monitoring contributor access for VM1 can only consume and mange the alert that have been generated for VM1.
Add new Alert
Alerts are defined via Azure Minitor Service blade.
Go to Azure portal > Monitor service> Go to Alerts section > Use add alert rule button.
According to the seleced resource there’s different signals available.
Signal types
- Metrics
- Activity logs
The alert configuration is different respectively. But without considering the signal types always we need the following items for creating an alert rule in Azure Monitor Service.
| Resource (For the scope of Alert Rule) | The scope of alert is specified in this step. – Subscription level – Resource Group level – Region – A specific resource – One Alert Rule for multiple resources with the same Resource Type is available. |
| Condition | The monitoring criteria. |
| Action Group | Collection of notifications. |
Resources
You owe your dreams your courage.
Koleka Putuma
In Azure Subscription features:
Subscription blade
How to see all available Resources Providers on Azure Portal
- Go to Azure Portal.
- Go to Subscription Resources.
- Select one of your active subscriptions.
- Subscription’s blade gets open.
- Go to Resource Providers item in the left panel.
The list of all resources providers/ all available resources is listed.
Topices
Advantages of the Azure Active Directory
- It’s suitable for the cloud.
- Support modern authentication protocols. (e.g. WS-FED, SAML Federation, OpenID Connect, OAuth)
- Azure AD will come with SDKs like ADAL and MSAL.
- Azure AD supports the scenarios like B2C, B2B, B2E (employee) and Software As a Service (SaaS) applications.
Hybrid Active Directory (On-Prem & Cloud)
- The on-premises identities can be synced to the Cloud via Azure AD Connect Tool.
- Even Multiple forests can be synced.
- The authentication can be federated to the on-premises with (ADFS).
- The identity can be manged in Azure AD but authentication can be done by on-prem AD server.
- The on-prem group policies can be synced in Azure AD.
Hybrid identities
- Create a new active directory on Azure
- Organization name: where we want to reference e.g. <MY_ORG>.local
- Initial domain: <MY_ORG>.onmicrosoft.com
- name, country, region
- Add e.g. <MY_DOMAIN_NAME>.com as a new domain name for the <MY_ORG>.onmicrosoft.com
- Verify the ownership of the domain name by adding a TXT record to the DNS Server
- Add UPN suffix to the on-prem forest. Server > Tools > Active Directory Domain and trusts > Right click Active Directory Domains > Properties > Alternative UPN Suffix > Add <MY_DOMAIN_NAME>.com
- Go to users on on-prem AD and change the domain to alternative domain or using powershell via Get-ADUser and Set-ADUser commands.
- For hybrid identities we use Azure AD Connect.
- We need a Global Admin Account in AAD.
- Download and install Azure AD Connect on the on-prem AD Server.
Provisioning a Azure Active Directory
Create Azure Active Directory (Azure Portal > All Services > Create button > the following image)
Azure Active Directory’s blade overview
Users
Groups
Enterprise applications: add new SaaS applications like Box, Dropbox,…
Devices: to mange devices.
App registration: Register application to machine to machine communication / client credential flow [more].
Application proxy: for exposing on-premises.
Azure AD Connect: for sync with on-prem Active Directory to AAD for hybrid identity.
App registration
App registration blade features are:
- New Registration
- Endpoints
- Troubleshooting
- Listing the registered applications ( All Applications | Owned Applications)
New Registration
Web APP/API is like a web site. The URL infront of Web APP/API (REST API) is the Sign-on URL. Ex. http://localhost:5000/signin-oidc
Native is like mobile app, desktop application or javascript single-page application. Redirect URL is where Azure AD directs the authenticated user’s details.
After the app registration, the following items can be important for developers. They are always available on the registered app blade.
- Application or client ID via Overview blade
- Redirect URL after successful authentication via Authentication blade
- Logout URL via Authentication blade
How to go the registered app blade
AAD > App registration> select & click the registered
Application/ Client ID
Redirect URL after successful authentication
Logout URL
App Registration usages
- For developing the Client Credential Flow / Machine to Machine communication
- For login to Azure API Management’s Developer Portal with AAD/ AAD B2C identity
Conditional Acess Policy
Licenses
Required license for conditional access policy
- Azure AD Premium P1 license
- Microsoft 365 Business Premium licenses
Conditions
The conditional access policy uses the Sign-in Risk value (it’s a probability). But it’s not the only value. Based on the policies different probability values have to be calculated [more]. Based on how conditions are configured different values are calculated and used.
Grant
In Grants either you block the users who have the conditions or you grant then if one of the checkboxes is available. Multi-factor is available only in the following licenses but with some differences [more]:
- Microsoft 365 Business Premium and EMS or Microsoft 365 E3 and E5
- Azure AD Premium P1
- Azure AD Premium P2
- All Microsoft 365 plans
- Azure AD free
Report-only mode [more]
Report-only mode is a new Conditional Access policy state that allows administrators to evaluate the impact of Conditional Access policies before enabling them in their environment. With the release of report-only mode:
- Conditional Access policies can be enabled in report-only mode.
- During sign-in, policies in report-only mode are evaluated but not enforced.
- Results are logged in the Conditional Access and Report-only tabs of the Sign-in log details.
- Customers with an Azure Monitor subscription can monitor the impact of their Conditional Access policies using the Conditional Access insights workbook.
Source : https://docs.microsoft.com/en-us/azure/active-directory/conditional-access/overview
Self-Service Password Reset
You owe your dreams your courage.
Koleka Putuma
Azure Authentication via Active Directory
| Disadvantage of Active Directory If a company has use the Active Directory of the authentication and the personals are allowed to do home office, therefore they need to use VPN Connection to authenticate to the company’s Active Directory. This isn’t so secure. |
Manage and authentication for mobile and modern devices
Classic active directory cannot manage modern devices with the following features:
- Group policies
- Kerberos or NTLM (works poorly)
- Session based security
What can help us to manage the modern devices:
- Mobile device management
- OpenID connect and OAuth
- Access token and refresh token
Forms-based Authentication
Protocols
WS-Federation
It’s a redírect-based flow. we go to a site and site says we are anonymous, and it redirects us to a authentication provider.
The user can pick an authentication provider and we provide the credential and then we get SAML post back.
SAML looks like XML and it contains what they call a SAML assertion and that establish your identity.
SAMLp
More flexible and supports more structured way to do SAML, more attributes.
OpenID Connect
OpenID Connect & OAuth are not synonymous.
OAuth is about a delegation protocol. For example I say, I’m allowing you to access my application if you match certain criteria. In this case I don’t know about the identity but if you have brown eyes and brown hairs, you are allowed to work with my software.
OpenID Connect says that you have to have minimum set of protocols that also establish your identity. OpenID is not only for web / mobile application. It can be applied to anything.
The following figure demonstrates the OpenID Connect usage for Web Application.
Insert photo here!
Single Page Application
Single Page Application is typically written in JavaScript (OAuth 2.0 Implicit Flow). Using OAuth 2.0 implicit flow and Single Page Application don’t have a secure way of storing long-lasting refresh token.
In OAuth 2.0 implicit flow, we assume that with closing the browser the user is logged out. Therefore OAuth 2.0 is suitable for Single Page Application.
Native Application
Like the applications running on a Mac OS, Linux OS or Windows OS, we use the Authorization Code Grant Flow. Here we have capability of storing long-lasting refresh tokens in a secure, encrypted manner offline.
Azure AD Authorization features
| Azure AD V1 endpoint | Authorization Code Grant Flow It has used authorization code grant flow for mobile apps and desktop applications as well. |
| Azure AD V2 endpoint | Authorization Code Grant Flow It prefers not to use authorization code grant flow for mobile app but only for desktop applications. Proof of key exchange (PKCE) flow It’s for mobile application. |
In practice’s Scenarios
| Web Browser talks to Web App It can be developed with WS-Federation, SAMLP, OpenID Connect. | Sigle Page Application talks to Web API It can be developed with OAuth to implicit flow, so ADAL.JS, MSAL.JS. | Native App talks to Wen API |
| Web Application talks to Web API It uses user credential delegated credentials, or using application’s identity. | Daemon If there’s no authentication opportunity. Daemon can call API registered in Azure AD. |
In practice Implementations
- Provision an Azure AD, Add new user
- Register your application
- Create a MVC Project with the following PowerShell code
- Perform authentication
Create a .Net core MVC project via the PowerShell.
# Create .NetCore MVC Project
$ProjectName="DotNetCorePipeline"
cd C:\YOUR PATH\AuthenticationForDevelopers
new-item -Name $ProjectName -ItemType directory
cd C:\YOUR PATH\AuthenticationForDevelopers\$ProjectName
dotnet new mvc --auth SingleOrg --client-id YOUR CLAINT ID --tenant-id YOUR TENANT ID --domain YOUR DOMAIN NAME --no-https
After creating the project go to project folder and open the project file in Visual Studio and run the project. [More Info about ID Tokens]
Business to Consumer (B2C)
for scenarios, in which the external users are the focus.
- Identities not known ahead time
- Social login may be required ( can be simple username, password authentication, with/without MFA) -> other identity provider like social accounts
- Custome user experience and brand promotion is important -> via collecting information from market
- Keep evrything secure and standard compliance.
In Practice
- Create a B2C Directory (it has two steps. First create a new one. Second assing to Subscription.)
- Register and configure an application
- Create an application that uses Azure AD B2C
Token-based authentication to SQL resources
SQL resources are the following SQL database, SQL warehouse and SQL server. The authentication is possible via AD.
Multi-Factor Authentication
Certificate-based authentication
Resources
How to change the email address of a basic user
It’s via power shell and API management Rest API possible.
Export API definition for developers
The developers can export the API definition in OpenAPI JSON format and WADL from API Management Developer Portal.
And the developers can use these files to generate client-side code by using the tools, which is adequate for them such Swagger codegen or Postman to start calling the API in a short time.
Azure Portal > API Management > Select the API Management Instance > APIs > Developer Portal Button > APIs Tab in Developer Portal > Select an API > We see the list of APIs Actions/Operations > Select an Action > API definition button -> Download the OpenAPI 3/ 2 JSON or YAML format or WADL.
Resources
Definition
Azure provides App Services through Azure Service Fabric. This abstracts the serviaces and underlying resources through the app service.
When and why we should use it?
Unique capability of each App Service
| App Service | Description |
|---|---|
| Web Apps | – is used to host web application. |
| API Apps | – is used to host APIs. |
| Mobile Apps | – is used in scenarios where mobile devices occasioonally have internet connectivity. – Backend for mobile applications. – Supporte push notifications. – And ability to work offline. |
| Logic Apps | -They orchestrate APIs into business process. – Create a complex workflow to automate a business process. – And calling APIs, which are anywhere. – |
| Function Apps | – is used to response to events. |
All the above App Services can fit together.
Resources
- Introduction to Azure App Services (Training)
- Create Azure Web App in the Azure Portal (Training)
- Try Azure App Service (Microsoft Portal)
Related words for the Service Fabric in Microservice Architecture:
- Dockre
- DC/OS
- Mesos
- Kubernetes
Microservices Development possibilities on Azure Cloud:
- Azure function
- Kubernetes Service
- Service Fabric
| Azure Function | Kubernetes Service | Azure Service Fabric |
|---|---|---|
| – They are micro-microservices. – reactes to an external change & event arrived on service bus. E.g. Blob created, message arrived on a queue or service bus queue. – They can be called as REST service from another application. – Azure function & Serverless computing is a great choice for types of applications that response to some events. – Good alternative for ASF (Azure Service Fabric) and AKS (Azure Kubernetes Service). – Doesn’t need infrastructure at all. – Suitable for background tasks with some upfront design. | – Microsoft implementation of open-source container orchestrator based on Docker => Docker Container – Container is faster => lighter virtual machine – Run Docker Container on Azure=> to manage environment (Upgrade, scale, versioning, expose network, load balance, and …) – Docker is a technology to manage and run multiple containers in production. -Azure supports Kubernetes natively & no installation need. | – Similar to containerization technologies. – Focuses on Microservices. – Kubernetes solves all the problem and developer should only develop the services-> it’s not easy to write scalable application, which runs on multiple & distributed clusters. – Container orchestration => Microserrvice Challenges: – Service Communication => service and instances of the services. – Service Discovery => How to talk to another Microservices, when there’s thousands on instances. – Monitoring Application => telemetry and collecting logs, provisioning and upgrading microservices. – Testing locally – Managing and recovering from downtime. – Scaling in & out. |
- For building a full flagged microserviceses solution.
- It has focus on business objectives and no infrastructure.
- For easy scalable architecture.
Programming model of Azure Service Fabric:
| Reliable Service: They are like windows services or Linux daemon application. It’s like console application. These services divided to sub types: – Stateless Services – Statefull Services -> for co-locate compute and data. | Reliable Actors: It uses the Virtual Actor Design Pattern and built on state-full reliable service framework. For massive amount of requests. |
| Guest executable: for existing projects without charging too much. | Containers: is like quest executable and still run on the Host OS and is completely isolated piece of deployment. |
Advantages of State-full Services:
- Reducing latency
- Provide resiliency by replication & persisting data across several different nodes.
Entry point from outside to back-end application is as follows:
| Web API – has no state (Stateless Service) – Must scale | Microservice – Independent part of the business logic and is perfect for Actor model. |
Advantages of Stateless Services:
- They are application proxies and gateway
- They are easy and chipper to scale
Start to work with the Azure Service Fabric:
- Installing Service Fabric -> is necessary for local development
- Start Visual Studio with “run as administrator” because of the “Service Fabric Cluster”. Because it needs low-privileged user called “Network Service”. The Network Service has no privileges on local system, therefore we need admin user.
Normal APP vs. Reliable APP
| Normal App | Reliable App |
|---|---|
| – An Application – Easy to write (Established framework) – Great choice of libraries – No learning curve | – A reliable service – Easy to write (established framework) – Reliable service is like an exe file that can be run without service fabric. – Great choice of libraries (x64 only) – None to little learning curve – Access ASF API for microservice scaling, health reporting, discover other services. – we can use plugable communication model. (They are Azure Service Fabric Built-in) via HTTPS, TCP, Wensocket, Custome TCP – Access to reliable storage, stateful service, low latency, high speed, local storage, replicated across machines. |
References
- My Flash cards for learning Azure Service Fabric
- ZEISS Group focuses on Azure Service Fabric for key integration platform
- Understanding the Programming Models of Azure Service Fabric (Training)
Multi Cloud Solutions 