Category: KnowHow

AWS, KnowHow

AWS : Virtual Private Network (VPC)

  • Regions
  • Availability Zones (AZs)
  • Virtual Private Clouds (VPCs)
  • AWS VPC Networking

Regions and Availability Zones allow anyone to create worldwide infrastructure with ease. They also allow for many options for creating redundancy within your platform. By properly using these components, you can create world-class level systems in terms of both scale and reach.

A Virtual Private Cloud (VPC) is a isolated private network that you control within the larger AWS network. These private networks allow you to configure your network architecture the way you desire. A VPC is region specific. You decide if your VPCs connect to each other or if you keep them independent. If you connect your VPCs, it’s up to you to configure them according to regular networking guidelines.

VPC aspects

  • Security groups are the same as firewall but not exactly
  • Two VPCs can have peering, even if they are in different regions
  • One VPC per region has automatically multi-AZ
  • AWS create a default VPC in every region but can be deleted

Services in VPC

The instance oriented fetures

  • Amazon RDS
  • Elastic cache
  • Document DB
  • Elastic search
  • EC2
  • Load balancer
  • Net Tune

Services not in VPC

Service oriented features and global services that have access to internet

  • SQS
  • S3
  • DymoDb
  • SNS
  • Cloud front
  • SCS
  • API gateway

Network Ranges

A network range is a consecutive set of IP addresses.

Network ranges are described using “CIDR” notation. CIDR notation consists of the first IP address of the network range, followed by a “slash”, followed by a number. That number describes how many consecutive address are in the range. A “/24” address has 255 addresses, while a “/16” has 65,536 addresses.

We cannot connect two VPCs with overlapping IP address ranges

Create VPCs

There are two ways in the AWS management console to create a new VPC.

You can create a very basic VPC by creating a new VPC from within the “Your VPCs” section of the VPC service. This option is best if you are an advanced VPC user and want to customize your VPC completely.

the second way is by using the VPC launch wizard.

Using the “Launch VPC Wizard,” create a new VPC. Select the “VPC with a Single Public Subnet” option. Name the VPC “Lesson-VPC” and keep the default options and create the VPC.

Review the new VPC, the routing tables that were created with it as well as the subnet and Internet Gateway.

Step1: create VPC

Step2: VPC created

VPCs List
The ‘Lesson-VPC’ is my new VPC

Step3: Create subnet

The subnets can be created in any availability zones of the VPC’s region.

Network components

ComponentDescription
Subnetsis tied to Availability Zone (AZ) and all resources created in this subnet are located in this availability zone
Route tablesis attached to one or more subnets and can be shared between subnets in different AZ.
DNS
DHCP
IPv4/6
Internet Gateway (Network routing)is represented in Route Table of the subnet and the services created in the subnet can send traffic to internet with public IP. Internet can send traffic to instances as well.
NAT Gateways (Network routing)Services can send traffic out to internet but cannot receive from internet.
Security Groupis a statefull firewall, can attach to EC2, RDS database
Network ACLsNetwork Address Control List, is a kind of stateless firewall and is applied to subnet.

Network routing

OptionsDescription
Internet Gateways
NAT Gateways
No internet connectionsis for connecting two subnets in a VPC with each other
VPN connectionsencrypted connection to connect to on-prem
DirectConnectiondatacenter to AWS

Debugging VPC Network

VPC Flow Logs

Flow logs allow you to see higher level network debugging information like the source port and source IP, and destination port and destination IP of traffic flowing within your VPC.

Traffic Mirroring

Traffic mirroring is like traditional “packet sniffing” on specific ports.

Edge cases

  • Multicast networking : is not supported in AWS
  • Penetration testing
  • Running email server

AWS networking does have some limitations that your own data center network would not.

  • You cannot use multicast in a VPC
  • You cannot put network cards into “promiscuous” mode to sniff ethernet packets.
  • There are some restrictions on opening up ports for SMTP
  • You cannot have network scans run against your account without discussing with AWS

You can connect VPCs together to enable:

  • Cross VPC connections
  • Cross region connections
  • Cross account connections

Clouds : Virtual Network and subnet

KnowHow

Visual Studio Code Online

Actually, I really like the visual studio code online!

Prerequisites

For using the Visual Studio Code Online an Azure subscription in necessary, because the IDE is being hosted on the Azure and a resource group is created for it.

Start with Visual Studio Code Online

The URL is : https://online.visualstudio.com/login

After sign-in to vs code, an environment must be created or the existing environment can be used. By creating an environment the URL to GitHub repository can be enter in the Git Repository field. The git is cloned in workspace once the IDE is hosted.

For the next time is not necessary to create an environment again, and the existing one can be used.


Experiences

I used the following tooling or extensions on the vs code online:

  • I installed the power shell 7 and developer the power shell scripts.
  • Connected the IDE to Azure and executed the power shell scripts.
  • I installed the azcli extension and developed the .azcli scipts.
  • Docker is available by default on the IDE machine.
  • I installed the docker-composer and ran the docker-composer file.
  • Installed the Azure App Service extension and deployed the docker image from local on the Azure App Service.
  • Clone the code from GitHub and work with Github.
  • Clone code from GitLab and work with GitLab.

Limitations and solutions

KnowHow

API Lifecycle

Nowadays, developers, enterprises, and organisations often create open Application Programming Interfaces (APIs) that allow other teams, developers, customers, and services to integrate to their products and services.

The following figure demonstrates the API lifecycle and after the Deploy phase the API can be shared with other API’s consumers in the Share Phase.

Today is usual to share the APIs via API Management and each cloud provider has its own API Management solution for a better API management and API monitoring. more…

API-Management-Workflow
API Lifecycle

In API Management each API has its own endpoint.

API-example-use-cases
Common API Examples
  1. Request is sent to API endpoint in API management.
  2. The above request is passed to the Backend API via API management.
  3. The request is processed in the backend API and if necessary the data layer is used and the result is generated
  4. The response, which contains the result, is sent back to the API management.
  5. The API management sends the response to the caller.

Resources:

Code Quality, KnowHow

Managing Secrets

Managing the secrets must be considered at different levels:

  • Source Code Level for example hard coded in source code.
  • Repository Level for example Git/ Git Hub.
  • Infrastructure Level for example for developing the infrastructure as code.

Source Code Level

To prevent to have secrects at code level, it’s enough not to write the users, passwords, tokens, secrets in source code but in:

  • Config file.
  • Environment variable. Two Examplpe for Python and C#

In both cases must be careful not to commit the .config and .env file to the repository.

Config file

In the case of using the Config file, must pay attention not to commit the config file into the repository.

Environment Variable

C#

Python

The environment variable in Python projects are saved in .env files.

_STORAGE_ACCOUNT_NAME=environ.get('STORAGE_ACCOUNT_NAME', 'storage account')
_STORAGE_ACCOUNT_KEY=environ.get('STORAGE_ACCOUNT_KEY', 'storage key')

Repository Levle

By preventing to write the secrets in source code and save the secrets in aconfig file, we must be careful not to push & commit the code to the repository.

Azure Cloud, KnowHow

Azure API Management

How to change the email address of a basic user

It’s via power shell and API management Rest API possible.

Export API definition for developers

The developers can export the API definition in OpenAPI JSON format and WADL from API Management Developer Portal.

And the developers can use these files to generate client-side code by using the tools, which is adequate for them such Swagger codegen or Postman to start calling the API in a short time.

Azure Portal > API Management > Select the API Management Instance > APIs > Developer Portal Button > APIs Tab in Developer Portal > Select an API > We see the list of APIs Actions/Operations > Select an Action > API definition button -> Download the OpenAPI 3/ 2 JSON or YAML format or WADL.

API definition button to download the definition for Swagger or Postman

Resources

Azure Cloud, KnowHow

Azure Service Fabric

Related words for the Service Fabric in Microservice Architecture:

  • Dockre
  • DC/OS
  • Mesos
  • Kubernetes

Microservices Development possibilities on Azure Cloud:

  • Azure function
  • Kubernetes Service
  • Service Fabric
Azure FunctionKubernetes ServiceAzure Service Fabric
– They are micro-microservices.
– reactes to an external change & event arrived on service bus. E.g. Blob created, message arrived on a queue or service bus queue.
– They can be called as REST service from another application.
– Azure function & Serverless computing is a great choice for types of applications that response to some events.
– Good alternative for ASF (Azure Service Fabric) and AKS (Azure Kubernetes Service).
– Doesn’t need infrastructure at all.
– Suitable for background tasks with some upfront design.		– Microsoft implementation of open-source container orchestrator based on Docker => Docker Container
– Container is faster => lighter virtual machine
– Run Docker Container on Azure=> to manage environment (Upgrade, scale, versioning, expose network, load balance, and …)
– Docker is a technology to manage and run multiple containers in production.
-Azure supports Kubernetes natively & no installation need.		– Similar to containerization technologies.
– Focuses on Microservices.
– Kubernetes solves all the problem and developer should only develop the services-> it’s not easy to write scalable application, which runs on multiple & distributed clusters.
– Container orchestration =>
Microserrvice Challenges:
– Service Communication => service and instances of the services.
– Service Discovery => How to talk to another Microservices, when there’s thousands on instances.
– Monitoring Application => telemetry and collecting logs, provisioning and upgrading microservices.
– Testing locally
– Managing and recovering from downtime.
– Scaling in & out.
  • For building a full flagged microserviceses solution.
  • It has focus on business objectives and no infrastructure.
  • For easy scalable architecture.

Programming model of Azure Service Fabric:

Reliable Service: They are like windows services or Linux daemon application. It’s like console application.
These services divided to sub types:
– Stateless Services
– Statefull Services -> for co-locate compute and data.		Reliable Actors: It uses the Virtual Actor Design Pattern and built on state-full reliable service framework.
For massive amount of requests.
Guest executable: for existing projects without charging too much.Containers: is like quest executable and still run on the Host OS and is completely isolated piece of deployment.

Advantages of State-full Services:

  • Reducing latency
  • Provide resiliency by replication & persisting data across several different nodes.

Entry point from outside to back-end application is as follows:

Web API
– has no state (Stateless Service)
– Must scale
Microservice
– Independent part of the business logic and is perfect for Actor model.

Advantages of Stateless Services:

  • They are application proxies and gateway
  • They are easy and chipper to scale

Start to work with the Azure Service Fabric:

  • Installing Service Fabric -> is necessary for local development
  • Start Visual Studio with “run as administrator” because of the “Service Fabric Cluster”. Because it needs low-privileged user called “Network Service”. The Network Service has no privileges on local system, therefore we need admin user.

Normal APP vs. Reliable APP

Normal AppReliable App
– An Application
– Easy to write (Established framework)
– Great choice of libraries
– No learning curve		– A reliable service
– Easy to write (established framework)
– Reliable service is like an exe file that can be run without service fabric.
– Great choice of libraries (x64 only)
– None to little learning curve
– Access ASF API for microservice scaling, health reporting, discover other services.
– we can use plugable communication model. (They are Azure Service Fabric Built-in) via HTTPS, TCP, Wensocket, Custome TCP
– Access to reliable storage, stateful service, low latency, high speed, local storage, replicated across machines.

References

Code Quality, KnowHow

Security aspects in code

There are different aspects which must be considered in source code for a better code quality and security. Some of them from the past time that we didn’t upload our source code in GitHub or other code repositories like:

  • Data injection
  • Database connection string

Some other aspects belong to nowadays that we upload code to repositories and using cloud:

  • Database connection string
  • Client Credentials such as Client Secrets
  • Access Keys to external APIs
  • Certificates
  • Encryption Keys

And of course there’s solution for each of the issues.

In the following link I’ll explain how we can solve the above issues via Azure Key Vault.

Azure Key Valut