Google
 

Saturday, February 18, 2017

Abuse of Story Points

Relative estimates are usually recommended in Agile teams. However nothing mandates a specific sizing units like story points or T-shirt sizing. I believe that - used correctly - relative estimation is a powerful and flexible tool.
I usually prefer T-shirt sizing for road-mapping to determine which features will be included in which releases. When epics are too large and subject to may changes, it makes sense to use an estimation technique that is quick and fun and doesn't give a false indication of accuracy.
On the release level, estimating backlog items using story points helps planning and creating a shared understanding between all team members. However used incorrectly, the team can get really frustrated and might try to avoid story points in favor of another estimation technique.

In a team I'm working with, one of the team members suggested during a sprint retrospective to change the estimation technique from story points to T-shirt sizing. The reasons were:
  • Velocity (measured by story points achieved in a sprint) are sometimes used to compare the performance of different teams.
  • Story points are used as a tool to force the team to do a specific amount of work during a sprint.
Both reasons make a good case against the use of story points.

The first one clearly contradicts with the relative nature of story points as each team has different capacity and baseline for their estimates. Also the fact that some teams use velocity as a primary success metric is a sign of a crappy agile implementation.
The second point is also a bad indicator. The reason is that you simply get what you ask for: If the PO/SM/Manager wants higher velocity then inflated estimates is what (s)he gets. Quite similar to the Observer effect.

Fortunately in our case both of these concerns were based on observations from other teams. Both the Product Owner and Scrum Master were knowledgeable enough to avoid these pitfalls and they explained how our team is using velocity just as a planning tool. However, the fact that some team members might get affected by the surrounding atmosphere in the organization is interesting and brings into attention the importance of having consistent level of maturity and education.

What is your experience with using story points or any other estimation technique? What worked for you and what didn’t? Share your thoughts in a comment below.

Wednesday, November 9, 2016

Nano Server on AWS: Step by Step

Windows server 2016 comes in many flavors. Nano server is the new addition that is optimized to be lightweight and with smaller attack surface. It has much less memory and disk footprint and much faster boot time than Windows Core and the full windows server. These characteristics make Nano a perfect OS for the cloud and similar scenarios.
However, being a headless (no GUI) OS means that no RDP connection can be made to administer the server. Also since only the very core bits are included by default means that configuring the server features is a different story than what we have in the full windows server.
In this post I'll explain how to launch and connect to a Nano instance on AWS. And then use the package management features to install IIS.

Launching an EC2 Nano server instance:

  • In the AWS console under the EC2 section, click "Launch Instance"
  • Select the "Microsoft Windows Server 2016 Base Nano" AMI.


  • In the "Choose an Instance Type" page, select "t2.nano" instance type. This instance type has 0.5GB of RAM. Yes! this will be more than enough for this experiment.
  • Use the default VPC and use the default 8GB storage.
  • In the "Configure Security Group" page things will start to be a bit different from the usual full windows server. Create a new security group and select these two inbound rules: 
    • WinRM-HTTP: Port 5985. This will be used for the remote administration.
    • HTTP: Port 80. To test IIS from our local browser.

  • Note that AWS console gives a warning regarding port 3389 which is used for RDP. We can safely ignore this rule as we'll use WinRM. RDP is not an option with Nano server.
  • Continue as usual and use an existing key pair or let AWS generate a new key pair to be used for windows password retrieval.

 

Connecting to the Nano server instance:

After the instance status becomes "running" and all status checks pass, observe the public IP of the instance. To manage this server, we'll use WinRM (Windows Remote Management) over HTTP. To be able to connect the machine, we need to add it to the trusted hosts as follows:
  • Open PowerShell in administrator mode
  • Enter the following commands to add the server : (assuming the public IP is 52.59.253.247)
$ip = "52.59.253.247"
Set-Item WSMan:\localhost\Client\TrustedHosts "$ip" -Concatenate -Force

Now we're ready to connect to the Nano server:
Enter-PSSession
-ComputerName $ip -Credential "~\Administrator"


PowerShell will ask for the password which you can retrieve from AWS console using the "Get Windows Password" menu option and uploading your key pair you saved on your local machine.

If everything goes well, all PowerShell commands you'll enter from now on will be executed on the remote server. So now let's reset the administrator password for the Nano instance:
$pass = ConvertTo-SecureString -String "MyNewPass" -AsPlainText -Force
Set-LocalUser -Name Administrator -Password $pass
Exit 

This will change the password and disconnect. To connect again, we can use the following commands and use the new password:
$session = New-PSSession -ComputerName $ip -Credential "~\Administrator"
Enter-PSSession $session



Installing IIS:

As Nano is a "Just Enough" OS. Feature binaries are not included by default. We'll use external package repositories to install other features like IIS, Containers, Clustering, etc. This is very similar to apt-get and yum tools in the Linux world and the windows alternative is OneGet. The NanoServerPackage repository has instructions regarding adding the Nano server package source which depends on the Nano server version. We know that the AWS AMI is based on the released version, but it doesn't harm to do a quick check:
Get-CimInstance win32_operatingsystem | Select-Object Version

The version in my case is 10.0.14393. So to install the provider, we'll run the following:
Save-Module -Path "$env:programfiles\WindowsPowerShell\Modules\" -Name NanoServerPackage -minimumVersion 1.0.1.0
Import-PackageProvider NanoServerPackage

Now let's explore the available packages using:
Find-NanoServerPackage
or the more generic command:
Find-Package -ProviderName NanoServerPackage


We'll find the highlighted IIS package. So let's install it and start the required services:
Install-Package -ProviderName NanoServerPackage -Name Microsoft-NanoServer-IIS-Package
Start-Service WAS
Start-Service W3SVC


Now let's point our browser to the IP address of the server. And here is our beloved IIS default page:


Uploading a basic HTML page:

Just for fun, create a basic HTML page on your local machine using your favorite tool and let's upload it and try accessing it. First enter the exit command to exit the remote management session and get back to the local computer. Note that in a previous step, we had the result of the New-PSSession in the $session variable so we'll use it to copy the HTML page to the remote server over the management session:
Copy-Item "C:\start.html"  -ToSession $session -Destination C:\inetpub\wwwroot\

Navigate to http://nanoserverip/start.html to verify the successful copy of the file.


Conclusion:

Nano server is a huge step forward to enable higher density of infrastructure and applications especially on the cloud. However it requires adopting a new mindset and a set of tools to get the best of it.
In this post I just scratched the surface of using Nano Server on AWS. In future posts we'll explore deploying applications on it to get real benefits.

Saturday, June 25, 2016

Agile and Continuous Delivery Awareness Session


This is a recording of a talk that I and Mona Radwan from http://www.agilearena.net/ gave at the Greek Campus in Cairo.
My part was focusing on the value of Continuous Delivery from a business perspective and the related technical practices required to achieve it.

Friday, May 20, 2016

Introduction to AWS video [Arabic]

My video "Introduction to AWS [Arabic]" on Youtube.

Saturday, February 27, 2016

AWS Elastic Load Balancing session stickiness - Part 2

In my previous post "AWS Elastic Load Balancing session stickiness" I demonstrated the use of AWS ELB  Load Balancer Generated Cookie Stickiness. In this post we'll use application generated cookie to control session stickiness.
To demonstrate this feature, I created a simple ASP.NET MVC application that just displays some instance details to test the load balancing.

Starting from the default ASP.NET MVC web application template, I modified the Index action of the HomeController:



Similar to what I've done in the previous posts using Linux shell scripts, this time I'm using C# code to request instance metadata from the http://169.254.169.254/latest/meta-data/ URL then I store the host name and IP address in the ViewBag object and display them in the view:


I deployed the application to two EC2 Windows 2012 R2 instances. As expected, using the default ELB settings, requests will be routed randomly to one of the instances. This can be tested by looking at the host name and IP displayed in the response.

Looking to the request and response cookies, we can find the asp.net session cookie added:

To configure stickiness based on the ASP.NET_SessionId cookie, edit the stickiness configuration and enter the cookie name:


Checking the cookies, we find that ELB generates a cookie named "AWSELB". As documented: "The load balancer only inserts a new stickiness cookie if the application response includes a new application cookie."


Now the browser will send back both the session and ELB cookies:

Still my preference for maintaining session state is to use a distributed cache service like Redis or even SQL server. Because in case an instance goes down or is removed from an auto-scaling the user will lose his session data in case it's stored in memory.

Saturday, February 6, 2016

Introduction to AWS presentation

My Introduction to AWS presentation that I presented at the Architecture Titans technical club.

Monday, January 4, 2016

AWS Elastic Load Balancing session stickiness

In a previous post "Configuring and testing AWS Elastic Load Balancer" I described how to configure AWS ELB to distribute load on multiple web servers.
We observed that the same client might get routed to a different EC2 instance. Some applications require the user to always be directed to the same instance during his session. This is the case when an in-memory session state is used or any other application specific reason. This requirements is often referred to as session stickiness.
AWS ELB offers two ways to provide session stickiness: using a cookie provided by the application, or using a cookie generated by ELB.

Load Balancer Generated Cookie Stickiness

Using an Expiring cookie
Using the same configuration as the previous post, the load balancer will have the stickiness configuration set to "Disabled". 
To change this behavior:
  1. Click "Edit" link
  2. Select "Enable Load Balancer Generated Cookie Stickiness" option.
  3. As a testing value, enter 60 as an expiration period.

Editing stickiness properties of ELB

Now, let's start testing the effect of this new configuration. Open the test url (for example: http://test-elb-834781956.eu-west-1.elb.amazonaws.com/cgi-bin/metadata.sh, check the previous post for more details). Using fiddler or network tab in developer tools of your favorite browser, you can observe that the response includes this header:

Set-Cookie: AWSELB=A703B168326729FE0B7D2675641656C1889E580D7525169B4BE36A819D9F2A18BE64415E6B0C90F5F2AC8CB0CFAF8DABE929DB27D5077D6FF616065A5BAF81DDB430BE92;PATH=/;MAX-AGE=60

As we see, it's a cookie named "AWSELB" with max-age of 60 seconds. and it applies to the whole site.

AWSELB cookie in in the response as appears in Chrome dev tools

If you refresh the page, you'll find that the browser sends the cookie as expected:

Cookie: AWSELB=A703B168326729FE0B7D2675641656C1889E580D7525169B4BE36A819D9F2A18BE64415E6B0C90F5F2AC8CB0CFAF8DABE929DB27D5077D6FF616065A5BAF81DDB430BE92


The cookie is sent by the browser, and the response does not include a new cookie

But the response does not include the cookie. So it will expire after 60 seconds and the browser will not send it after expiration. Refreshing the browser several times will direct the traffic to the same EC2 instance, we can verify this by examining the response which looks like:

Host name:
ec2-52-30-170-211.eu-west-1.compute.amazonaws.com
Public IP:
52.30.170.211

As soon as the cookie is still active, the request is directed to the same instance. But what happens after the max-age passes?
A new cookie will be generated and you might be directed to one of the other web servers, and a new cookie with another value will be generated by ELB:

Set-Cookie:AWSELB=A703B168326729FE0B7D2675641656C1889E580D7525169B4BE36A819D9F2A18BE64415E6B0C90F5F2AC8CB0CFAF8DABE929DB27D5077D6FF616065A5BAF81DDB430BE92;PATH=/;MAX-AGE=60

Notice that the value of the cookie has changed. And after this cookie expires, a new cookie might be generated using the old value pointing to the first instance.


Using an ELB cookie without expiration:
If the expiration values is left blank, then the behavior differs. The cookie will be generated without the max-age value. And the browser will send the same cookie until it's closed.

Set-Cookie:AWSELB=A703B168326729FE0B7D2675641656C1889E580D7525169B4BE36A819D9F2A18BE64415E6B0C90F5F2AC8CB0CFAF8DABE929DB27D5077D6FF616065A5BAF81DDB430BE92;PATH=/


What happens when a server goes down?
To try this scenario, let's shut down the server which is getting the requests and refresh the browser. This time ELB generates a new cookie pointing to a healthy instance.


Summary:
ELB has a built-in mechanism to support session stickiness with no code changes from the application side.
Using an expiring cookie might not be the best option to guarantee session affinity as the cookie is not renewed and it seems that there is no way to achieve a sliding expiration window for it. So you might prefer to go with a cookie without expiration.

In the next post, we'll use the other method available for session stickiness: using Application Generated Cookie Stickiness.