Archives de catégorie : English posts

Azure Site Recovery for Azure Virtual machines (Preview)

I’ve been a big fan of Azure Site Recovery service. The product has evolved a lot since initial release (only capable to orchestrate a failover between two on-premises sites), introducing replication of on-premises workloads to Azure and many more features. Until theses last days, we had to comply with some limitations (Max 64 disks of 1Tb each per protected workload) and a major scenario we could not address. Note that Larger disk sizes just been announced in preview, up to 4Tb per Disk.

Since introduction of ASR replication capabilities (for Hyper-V, VMware or physical), we were not able to protect an Azure Virtual machine with Azure Site Recovery. ASR Team, just announced a preview of Azure Site Recovery for Azure Virtual machines. That was one of the top customer requests about Azure Site Recovery. Introduction of this new Azure Site capability allow to respond to the following scenarios:

  • Can survive to a major Azure Region failure by replicating virtual machines to another region
  • Migrate virtual machines between Azure regions
  • Comply with regulations requesting a Disaster Recovery strategy
  • Ability to clone an application based on Virtual Machines to validate

Feature is now available for public preview. Let’s see how to protect an Azure Virtual machine and develop a Disaster Recovery Plan. Note that even if this preview is available in the Azure portal, it’s still a preview. For example, support for PowerShell and CLI is not yet available.

Prerequisites

  • An instance of the Azure Recovery Site service: We will need to have an instance of the service in the Azure region in which we want to replicate with. This can be strange. If you remember Azure backup limitations, Azure Backup instance service must be in the same region as virtual machines we wish to protect.
  • A resource group in the target Azure region to store
  • A virtual network (and subnet) to connect the future Network interface. It’s not mandatory if you plan to establish RDP between two paired Azure regions (Azure virtual Network span between paired Azure regions).
  • A storage account created in the target Azure region to store virtual machines
  • A storage account created in the source Azure region to cache content to be replicated to the Target storage account
  • An availability set depending of the destination Azure region if protected workloads in the source Azure regions are linked to an existing availability set.

 

Setup

This blog post aim to describe the setup Itself, but also:

  • The Failover of an Azure Virtual machine to the target Azure Region
  • The Rollback of an Azure Virtual machine to its initial Azure region

Enabling and managing this new feature is so simple and very attractive from a financial point of view (no extra-cost for compute), that would be a crime to do not use it.

If you know the Azure Site Recovery portal experience, you should discover Azure as a source of a protected workload. Note that service is now in preview. For this blog post, I will be having a workload to protect located in Central US Azure region. My goal is to use the West Europe Azure region for DRP plan.

clip_image001

Azure Site Recovery service instance cannot be in the same Azure region as workloads to be protected. It’s logic. On this first stage, we only select the source Azure region, the deployment model and the resource group containing the workloads you wish to protect. Azure Site Recovery instance must depend of the same Azure subscription (and of course Azure environment). While writing this blog-post, it was not yet possible to protect workloads from different subscriptions. That’s a scenario that Azure CSP vendors are waiting to offer services. At this stage UX experience does not change so much from protecting other workloads:

clip_image002

That’s here that many UX changes are introduced. For this blog post, I choose to select a target Azure region that is not paired with the source Azure Region. We are not limited to the paired Azure regions. Azure Site Recovery portal make some proposal for some resources:

clip_image003

 

If you click on the « Customize » link in the upper part of the UX, you will be able to change proposal for required resources:

  • A resource Group in the target Azure region
  • A virtual Network in the Target Azure region

 

For each Virtual machine to protect we can customize:

  • The Storage account to be used to store VHD to be used by future replicated virtual machines
  • The Storage account to be used for ASR cache purpose

 

For these two storage Accounts, you don’t need more than a Local Redundant Storage.

clip_image004

 

If your virtual machines to be protected is linked to an Availability set, you will need to select an availability Set in the destination Azure region to offer the same SLA for your virtual machine.

Each Azure Virtual machine to be protected must be associated to a replication Policy.

we can customize the following parameters:

  • Retention policy for the recovery points
  • Frequency for app-consistent snapshot

clip_image005

 

Initial configuration is now over. We can enable replication.

clip_image006

 

Once replication is enabled we can follow the Azure Site Recovery Jobs. Bellow we have all jobs related to the setup of the protection of your Azure virtual machine.

clip_image007

 

If we take a closer look at the jobs related to replication, we will discover that ASR deploy a virtual machine extension in our workload.

clip_image008

 

Watch out, it’s not because initial replication is considered as enabled that its completed. Initial replication will take some times.

clip_image009

 

In my case (standard Windows Virtual machine without any data disks), initial replication took around fifty minutes.

clip_image010

Once initial replication is terminated, we can consider that our workloads is protected.

clip_image011

 

If we take a closer look at virtual machine information page, we discover two type of recovery points:

  • Crash-consistent
  • App-Consistent

 

In DRP situation, we can choose to prefer RTO versus RPO.

clip_image012

If you are familiar with Azure SQL, you should be familiar with that map. Now I have an Azure Virtual machine located in Central US Azure region with a replica in East US Azure region.

clip_image013

 

Now you have a DRP, you can test it. Microsoft recommendation is to perform a test failover from times to times to validate our DRP plan. You can choose to ignore the recommendation is you wish but that’s a good idea to test a DRP.

clip_image014

 

We can select the recovery point using:

  • The latest recovery point (for a lowest RPO)
  • The latest processed (for the best recovery time)
  • The last app-consistent

 

For this test, recommendation is to select a dedicated Virtual Network that is not connected to the source Virtual Network (Virtual Network Gateway or Peering). In the jobs, we can track the progress.

clip_image015

 

At this stage of the test we have two virtual machines running: The original and the test. Because the second one is connected to an isolated network you can check the health of the application and validate the DR plan.

clip_image016

 

Test is now complete we can cleanup used resources.

clip_image017

 

Our workload is now protected. Let’s see how to initiate a « clean failover ».

 

The « Clean Failover »

We will be considering that failover is initiated manually. In fact, using Log Analytics we can:

  • Detect an unhealthy virtual machine and initiate a single failover
  • Detect a major outage in Azure Activity and initiate a complete relocation

 

When initiating the failover process, we need decide if fast service recovery is more important than full service restauration.

clip_image018

 

Because we have both App-Consistent snapshots and Recovery points (up to 24 hours with the default policy), custom recovery point allows us to choose a specific version of my application. That’s great because, nowadays, applications are distributed among many servers. In DR situation, we will need to restore many servers.

clip_image019

 

In Azure Site Recovery Jobs view we can notice that failover was initiated.

clip_image020

 

If we look at the details, we see that the virtual machine located in the source Azure region was shutdown properly and a new virtual machine was provisioned in the target Azure region. In my case, complete failover operation took five minutes and a half (I choose to minimize the Recovery Point objective).

clip_image021

 

Once terminated Job appear as completed in the Azure Site Recovery Jobs view. From a virtual machine point of view we now have two virtual machines but only the one located in the target Azure region is running.

clip_image022

 

Once failover is completed we need to commit the operation. Once commit is performed we select another recovery point.

clip_image023

 

The clean Rollback

Rollback process is named « re-protect ». In fact, it’s the same process but we switched the target and source parameters.

clip_image024

 

By default, ASR will use the same parameters but you can override them if necessary.

clip_image025

 

It’s a little bit longer than on the protection phase, but remain an acceptable

clip_image026

 

Disable replication

That’s the bonus. When failover is committed, you have the choice to disable replication. This option was designed to move a workload from an Azure region to another.

 

Additional readings

To have more information about the feature, I would recommend the following readings :

 

Conclusion

What to say. Even if it’s a preview, it’s almost feature complete (CLI & PowerShell support are missing, just like Managed disks). Once feature will be Generally Available, we will have first-class Disaster Recovery service we cannot not implement.

 

BenoitS – Simple and Secure by design but Business compliant (with disruptive flag enabled)

 

DirectAccess now supported in Azure virtual machines

Note : This post was updated on 11 of november 2016 due to recent change : DirectAccess No Longer Supported in Microsoft Azure

I was expecting this news for a while. Until a few days, DirectAccess was considered as unsupported scenario in Azure according to KB2721672. On the 14th of July this year, this KB was updated. The Remote Access feature is not longer listed as unsupported and appear in the supported features list But the DirectAccess Feature remain an unsupported scenario.

clip_image001

 

Hosting a DirectAccess infrastructure in Azure introduce a new way to build IT. Everyday that pass, the need to a local IT infrastructure is reducing. Soon, we could consider building a complete IT infrastructure hosted in Azure and connect computers to it using DirectAccess.

Dear, I have other wishes about DirectAccess :

  • ARM template to build a DirectAccess infrastructure in Azure (using a certificate located in Azure Keyvault of course)
  • Having DirectAccess as a VPN gateway scenario choice

 

Thank you for the news Richard and for the update.

­

BenoîtS – Simple and secure by design but Business compliant (with disruptive flag enabled)

Breaking the myth of DirectAccess end-to-end scenario – Part 6

After enabling the IPsec logging on our internal server we will have new security events logged on this server. When VIP user logged on a DirectAccess client located on Internet try to access sensitive data, we will see that in the Windows Firewall with Advanced Security console.

clip_image001

It’s our new IPsec Tunnel initiated from our DirectAccess client. In the security event log we will found events like this one, proving computer authentication:

clip_image002

And also user authentication.

clip_image003

Bonus, what would happened if we change the security group with something like « NT AUTHORITY\Organization Certificate » as filtering security group as illustrated bellow:

clip_image004

My VIP will require to logon with his smartcard to initialize IPSEC tunnel with the internal server, not because required certificate is on the smartcard but because when we log with a smartcard, we are automatically member of this group.

clip_image005

When user see this message on DirectAccess client-side, we have the following event on our internal server

clip_image006

So my VIP need his smartcard.

clip_image007

That’s the end of this blog-post series. Hope you enjoyed. It’s time for me to go back to clouds.

BenoîtS – Simple and secure by design but Business compliant (with disruptive flag enabled)

Breaking the myth of DirectAccess end-to-end scenario – Part 5

Connection Security Rules are now in place on both sides. We just need some additional magic trick. First, I recommand to excempt ICMP messages from IPsec tunnels that end on our internal server. It’s for debuging purpose. It’s configurable at the Windows Firewall with advanced security, in the IPsec Settings tags.

clip_image001

 

Remember, it’s just for debugging purpose and helped me a lot to understand what happened under the hood. Last configuration, we need to configure the authorization model. It’s not at the Connection Security Rule level but at the Windows Firewall with Advanced Security node, in the IPsec settings tab and my recommendation would be to do not configure this setting in a GPO but directly on each server. With this approach, our server-side GPO remain generic.

clip_image002

 

Now it’s time for the big magic trick. You can test now if you want, you are not able to access internal server with sensitive data. It took me a long time (and a lot of aspirin) to understand why until I noticed this fucking checkbox in the infrastructure IPsec tunnel configuration on DirectAccess gateway.

clip_image003

 

In our scenario, our new Connection Security Rule is never used even if destination match because another Connection Security Rule is already applicable. That’s our case. On the DirectAccess gateway, default configuration includes two Connection Security Rules:

  • DirectAccess Policy-ClientToInfra (Infrastructure IPsec tunnel)
  • DirectAccess Policy-ClientToCorp (User IPsec tunnel)

Problem is located in this tunnel. If we have a look at tunnel endpoints, it include the 2002:836b:a:1::/64 prefix.

clip_image004

 

And my internal IPv6 destination is 2002:836B:a:1:0:5efe:192.168.1.101. It means that for this destination, traffic will go through the IPsec infrastructure tunnel. We have the same problem on the DirectAccess client-side. To solve this problem, we need to reconfigure default configuration for the DirectAccess Policy-ClientToCorp specify that network traffic that matches another IPsec connection security rule does not go through the IPsec tunnel.

That’s a big problem. If we use GPMC to reconfigure the required parameters (in both GPO), we risk to things:

 

Because I spend now most of my time in clouds, I wrote a small PowerShell script that perform the following actions:

  1. Check for execution prerequisites (GPMC, RemoteAccess, …)
  2. Search a writable domain controller (PDCE in fact)
  3. Perform a Backup of Server-side and client-side GPO
  4. Create a GPOSession object to update required parameters in both GPO
  5. Update the required parameter in the GPOSession object
  6. Update GPO with the GPOSession

 

PowerShell script is available here.

Here an example of the script output with the STATE parameter. Script found DirectAccess GPOs and checked we have the default configuration.

clip_image005

And now the same script with the ENABLE parameter that enable our required parameter in both GPOs.

clip_image006

At this point, you are no longer supported by Microsoft as you just customized GPO configuration of DirectAccess. That’s why script alse have the DISABLE parameter to revert to initial configuration.

clip_image007

 

In a standard DirectAccess infrastructure, all IPsec tunnels terminate at the DirectAccess Gateway witch have an IPsec DOS protection layer. As our additional IPsec tunnel won’t terminate on the DirectAccess Gateway, we won’t be able to use extra security layer. If I have a look at Server-side GPO, now the checkbox is enabled on our DirectAccess gateway.

clip_image008

 

And on the DirectAccess client-side GPO configuration, it’s the same.

clip_image009

 

Now our new IPsec tunnel is operational. We will see that in the next blog post.

 

BenoîtS – Simple and secure by design but Business compliant (with disruptive flag enabled)

Breaking the myth of DirectAccess end-to-end scenario – Part 4

If we have a client-side configuration of the tunnel, we need a remote-side. This configuration will be applied by GPO on our internal server with sensitive data.

clip_image001

It’s logic, we choose the oposite choice on remote side of the IPSEC tunnel.

clip_image002

Our internal Server will be the remote endpoint of the IPSEC tunnel. We must enforce authentication for inbound and ountbound communications.

clip_image003

If we require authentication we must also enforce authorization on the local tunnel endpoint. But wait a minute where is the IPv6 address of our internal server? You can include it yes. But if you do so, you will generate as much as GPO as internal server to secure. With my approach, we generate a generic GPO, applicable to a large number of internal server. You AD admin team will appreciate.

clip_image004

Because we are requesting authentication for inbound and outbound communications, we must have the same configuration on both sides of the IPsec tunnel. Our authentication will be composed of primary and a secondary authentication method.

clip_image005

It’s exactly the same configuration as on DirectAccess client-side. That’s my choice, to be able to filter on a computer or user criteria.

clip_image006

But one important point here. This configuration will only work if you have a certificate delivered to your internal server and this certificate must be delivered by the same Public Key infrastructure as the one used for DirectAccess clients. This new Connection Security Rule will only apply on Domain Windows Firewall profile.

clip_image007

This Connection Security rule will be name « SECURED ZONE ».

clip_image008

Now it’s time for a magic trick. Our new Connection Security Rule apply to all domain traffic, including both internal and external access (IE DirectAccess). Problem, I do not want to enforce IPSEC for internal access (not yet). That’s why I need an exception to manage this case. We will introduce a new Connection Security Rule on GPO targeting our server with sensitive data. This will be an Authentication exception.

clip_image009

This exception will cover my complete IPv4 internal IP range. If you already have IPv6 capable systems on your internal network that need to communicate with our secured server, I would recommend you to add these IP to the list, unless you want to establish a secure communication with IPsec tunnels.

clip_image010

This exception will only apply to the Domain Windows Firewall profile.

clip_image011

This Connection Security Rule will be named « LAN EXCEPTIONS ».

clip_image012

OK, prerequisites are in place. It’s time for the big magic trick. Why? Because at this stage, it does not work. That’s the most interesting part, unless you want to send your time hitting your head against the walls to find the solution.

BenoîtS – Simple and secure by design but Business compliant (with disruptive flag enabled)

Breaking the myth of DirectAccess end-to-end scenario – Part 3

Let the fun begin and create new IPsec tunnel. We will be starting with the client-side. We will introduce this GPO configuration on a dedicated GPO applicable to our DirectAccess clients. Remember, it’s not supported to customize DirectAccess configuration (We will break that rule later, …).

clip_image001

We are in a client-to-gateway configuration and we to need to ensure that if network traffic match this rule we use this tunnel.

clip_image002

During IPsec Initialization, DirectAccess client send authentication information. These information will be required for filtering purpose on the DirectAccess gateway side.

clip_image003

On client-side of the tunnel our new tunnel must only be used if we try to access our server with sensitive information. IPv6 address of this server will be configured as a remote endpoint.

clip_image004

IPsec authentication is a negotiation process (Show me who you are and show me who are you).

I wanted to provide the same security level as default DirectAccess IPsec tunnels, so I choose to use a primary and a secondary authentications methods.

clip_image005

My choice, using Computer certificate plus Computer Kerberos ticket as primary authentication methods and only user Kerberos ticket for secondary authentication methods. With this choice I will be able to filter access based on computer account group membership or user account membership.

clip_image006

At last, this new IPSEC tunnel must only apply on DirectAccess scenarios, so limited to private and public firewall profiles.

clip_image007

This Connection Security rule will be name « SECURED ZONE ».

clip_image008

That’s all on DirectAccess client-side. On DirectAccess gateway-side we will follow the same logic. So until now it’s not so complicated. Not yet. Don’t worry headache is on the way.

BenoîtS – Simple and secure by design but Business compliant (with disruptive flag enabled)

Breaking the myth of DirectAccess end-to-end scenario – Part 2

To make it clear. Every time we deployed DirectAccess, we always skip the optional configuration part. I’m talking about this screen.

clip_image001

 

First reason, application server must be IPv6 ready. IPv6 ready can be ISATAP or native IPv6. That’s the first challenge. Because it’s only a demonstration, environment built for this blog post will, I will be using ISATAP, not native IPv6 for internal network. I wanted to make the content more digest. In real production environment ISATAP would not be supported for multiple reasons but the same approach can be used.

So my first step will be to configure my internal server with sensitive datas as ISATAP client of my ISATAP router. By Default ISATAP DNS record is blocked for name resolution because it’s on the DNS Query Block List. Not a problem we will be configuring ISATAP client locally with a single Powershell command : Set-NetIsatapConfiguration -Router srv.directaccesslab.lan -PolicyStore PersistentStore :

clip_image002

 

Now in DNS I have two records for this server.

clip_image003

 

But on my DirectAccess client, name resolution does not provide the expected result. It’s not because of ISATAP, we would have the same problem with native IPv6. Where does this address come from?

clip_image004

 

If you are a DirectAccess expert, you immediatly recognize a NAT64 prefix and you know why if you are familiar to this old blog post of mine : DNS64 behavior change in Windows Server 2012. By default, the DirectAccess configuration no longer resolve IPv6 host names since Windows Server 2012. If we take a closer look at the DNS64 configuration, we will discover that in default configuration, only A records are resolved.

clip_image005

 

To enforce IPv6 DNS name resolution, we will reconfigure DNS64 configuration with the following command: Set-NetDNSTransitionConfiguration -OnlySendAQuery $false

clip_image006

 

After DNS64 reconfiguration, name resolution is better on DirectAccess client-side.

clip_image007

 

Even if DNS64 provide IPv6 addressing, it’s not IPv6 traffic than reach the internal Server. NAT64 act as a translation device between IPv6 on the ourtide and IPv4 on the internal network. Would be difficult to establish an IPsec tunnel in such situation.

Now we have IPv6 addressing from end to end, we can discuss about establishing new IPSEC tunnels. By default DirectAccess configuration generate two types of tunnels :

  • Infrastructure tunnel (IPSEC Tunnel mode)
  • User tunnel (IPSEC Tunnel mode)

If we use the approach proposed by Tom Shinder, this lead to a new IPSEC Transport. Too complicated to secure and not challenging enought. I already covered such scenario in this blog post series

So we will introduce a new IPsec Tunnel, starting in next post.

 

BenoîtS – Simple and secure by design but Business compliant (with disruptive flag enabled)

Breaking the myth of DirectAccess end-to-end scenario – Part 1

It’s been a while since i published content on DirectAccess, that’s also why this post is in English. As I spend more and more of my time with Cloud Business (for Blue team), I do not spend much more time on DirectAccess. These days I had time to explore my OneNote on DirectAccess and rediscovered a subject we are not so much to deal with: The end-to end-mode of DirectAccess, the Myth of end to end security. That will be the subject of this blob post-series. Do you remember if the early days of DirectAccess, we used to present such diagram?

clip_image001

In such scenario a DirectAccess infrastructure could be configured to offer an extra security layer for VIP trying to access sensitive data. VIP would be invited to use his smartcard to authenticate to initialize the application tunnel. For many people, it was a myth. In fact not really some smart people published some content about this : A Short Introduction to UAG DirectAccess End to End Security. It was very helpful to be to start thinking. It was UAG based but we are able to do it with Windows 2012 R2, nothing changed.

clip_image002

But one point is missing in Tom Shinder article (Sorry). How do we filter connections and add our extra security layer (enforce smartcard authentication to access servers with sensitive data? That the pain point. In fact, it’s possible with Tom Shinder approach but complex. Why? Because it relies on IPsec transports, not IPsec tunnels. This means that we must secure each incoming protocol at the Windows Firewall level of our servers to enforce a specific option

clip_image003

It’s possible from a technical point of view but complex to setup and maintain. Would it be possible to secure all communications? Yes, but it would rely on IPsec tunnels, not IPsec Transports. This is a hudge change that require to customize default DirectAccess configuration, what is not supported by Microsoft.

It will be a long path to achieve this objective, so multiple blogs posts will be needed:

BenoîtS – Simple and secure by design but Business compliant (with disruptive flag enabled)

MS-OTPCE – One-Time Password Certificate Enrollment Protocol

I had many questions these days about DirectAccess OTP authentication mechanism. I already published multiple blog posts on the subject:

All these blog post are useful to understand and troubleshoot DirectAccess OTP scenario but on resource was missing : the Windows protocol definition for [MS-OTPCE]. Now you have all information to clearly understand the authentication process.

 

BenoîtS – Simple and secure by design but Business compliant (with disruptive flag enabled)

Renewed as MVP Enterprise security, seven in a row

One more time. Seven years as an Enterprise Security MVP. I spend most of my year to work on Windows Azure Pack. This year, I will spend my time on the following subjects :

  • Windows Server 2016
  • Windows Azure Stack
  • Security features of Windows 10
  • Security features of Windows Server 2016

 

BenoîtS – Simple and secure by design but Business compliant (with disruptive flag enabled)