Where have I been lately

Since July 2018, I have been working at the same Company as a Software Developer. After 9 years in IT (of which last 6 years were as DevOps/Systems Programmer), I wondered where do you I go from here.

While AWS keeps churning out amazing features and services and the DevOps community as a whole is only getting stronger, I wanted to see how I can utilize what I have learnt so far and how to stand out (in terms of experience and skillset) in near future.

I realized that back during my last year at University and then throughout my career (now 10 years), there has been one constant – I love to code. Regardless what language or what product, I just love to code. And this begs the question, how I can translate this into something that would give me yet bigger dividends in terms of my employment and in terms of my own interests.

And, as a side note, while modern day DevOps and Cloud automation development offers opportunities to code, as a DevOps Engineer, if you are only working on the Systems side of things, your coding and toolset (e.g. Python, Shell Scripting, Ansible, etc.) would always be done from systems point of view.

BUT…

There are two sides to a complete (or if I may humbly say..’full stack‘) DevOps Engineer: Systems side AND Development Side (thus the word ‘DevOps’). While I have been involved in OPS side mainly for 6 years (2012 to 2018), I felt now I should switch gears and get in on the Development track. This moves allows me to bulk up / level up my coding skills by giving me chance to not only learn how to produce production grade software using languages such as Java/Kotlin/SQL/JavaScript, etc. It also gives me opportunities to learn of interesting frameworks such as Spring and React and also allows me to become familiar with Databases (stored procs, triggers, ETL, etc.).

So what now for myself?

I will still continue down the DevOps or SRE side of things in near future as I want to further build on my experiences.

What have I learned?

There are few things that have happened with my move into development role:

1) I have proved it that I can switch gears and go from OPS/DevOps role into Development role and back.

2) Most of the DevOps folks belong to one of two camps: Those who came from OPS background and those who came from Development background and while both are super intelligent, they tend to have hard time picking up skills and experience from other side – in my case, this experience will make me a ‘full stack DevOps Engineer‘.

3) Since I do love coding and now I am becoming more experienced with full stack application development, I can see myself getting into a Site Reliability Engineering (SRE) role in near future.

4) Working again as a Software Developer after spending years working as a DevOps Engineer has given me appreciation and awareness of the systems side of things and even things such as build scripting, and the whole CI/CD pipeline (which does give me advantages during my projects and tasks).

4) This is for my personal self: While there are awesome folks who have 10+ years experience working as a Software Developer, there is a fairly large percentage of Developers who after few years into their careers switch into some other roles where some code less and less while others get into non-coding positions (I respect both choices). However, I have gone into Application Development other way around (after about 9 years into my Career and after 6 years of pure DevOps roles). This journey of mine has been interesting and eye opening (so much has changed technologically for the better!).

Advertisements

AWS virtualization type and volume snapshot

So couple of days back, I took a snapshot of volume that was attached to an EC2 instance running ubuntu 16.04 LTS for backup purposes.

It so happened that I needed to restore that instance using that snapshot and therefore, I used that volume snapshot, created AMI out of it and launched the instance. However, the instance failed status checks and in the system log, I could see error messages indicating ‘kernel panic’.

After thinking about it, it clicked in my mind: when I was creating AMI out of my volume’s snapshot, I picked the virtualization type and I had selected the default dropdown value of ‘paravirtual’. However, the AMI I had selected from the AWS Marketplace indicated that the ubuntu 16.04 LTS image I used was of type ‘hvm’ virtualization.

Therefore, I recreated another AMI out of my snapshot and this time, I selected ‘hvm’ as the virtualization type and then launched my EC2 instance and success!

I then did some research online and turns out, others had also ran into same issue due to mistake made in selecting incorrect virtualization type!

Here is a good explanation of Hardware virtual machine (HVM) and ParaVirtual (PV) from AWS:

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html

Installing GUI Desktop for Amazon Linux

This is regarding installing GUI Desktop for Amazon Linux (AWS offered Linux distro).

One of the Developers wanted to have GUI Desktop enabled on his Amazon Linux EC2 instance running on AWS. The guides I found on internet were geared towards enabling GUI for either RHEL or ubuntu on AWS but not for Amazon Linux (for those who aren’t familiar – Amazon Linux is Amazon’s very own Linux distro. which is based on RHEL so among different things, it uses for example the yum package manager).

That led me to talk with AWS Support and there the answer was clear: Amazon Linux (as in the distro created by Amazon) does not support GUI. However, AWS has published guides to install and enable GUI Desktop for ubuntu (14.04 and 16.04), centOS 7, RHEL 7.3.

Luckily, the Developer was more than happy to switch his choice to ubuntu!

Lesson learned: Chocolatey

Lately, I have been using Ansible to provision an EC2 instance using the latest Windows 2012 R2 Server AMI and Windows 2016 Server AMI on AWS.

Life is easy when provisioning a Linux server as you can use the distro’s package manager (apt-get, yum for example) to download and install software packages. However, when it comes to Windows OS, it’s a bit different.

I understand that starting Windows 10 (not sure about latest MS Windows server OS),  Microsoft has started to include a software package manager in which you can hook in other windows-oriented software package manager.

However, since I am trying to provisioning a Windows 2012 Server, that is not an option for me. Therefore, I am utilizing ‘Chocolatey’ directly to download and install Windows software packages.

Sure, there are caveats involved (namely security of packages/public repo. if you are using the public repo., and having to utilize a private repo. for Production usage) but nonetheless, it is a solid choice for my use case.

One Key lesson learned though: Some of the Windows software packages take forever to install. Packages such as ‘sql-server-cmdlineutils’ and ‘sql-server-management-studio’ take in access of 10 minutes (in case of ‘sql-server-cmdlineutils’ it took me close to 70 minutes). And this too on a relatively powerful EC2 instance type – t2.xlarge!

Some Tips about AWS Organizations

Just had a great conversation with AWS Support Rep. Today.

Some interesting points:

1. You can can’t attach a budget to OU (Organizational Unit) –> budget are associated using account ID only.

2. Say you have a budget and have 2 accounts added to that budget — and for sake of conversation, say budget = $1000/monthly –> this means they both have the same $1000/monthly (not $1000 for Account A and $1000 for Account B).

3. Say you create SCP policy to whitelist services XYZ on a DEV account –> those restrictions will apply to *all* users/roles (even admin/root).

Personally,

AWS Organization is a must for any Account at least for disabling those AWS services which a Company will not be using in foreseeable future (for example, An Insurance Company might want to disable IOT related service).

Some Terraform limitations encountered

While working with Terraform (against AWS), I have recently run into following 2 roadblocks:

For this time, I ended up using boto3 python library to bypass these issues. I guess, time to  start using Cloudformation Templates again!

AWS CSA (Associate) Exam Tips

Hello,

I passed my AWS CSA (Associate) exam on Thursday, Nov. 16, 2017. Here are some tips:

– Just reading the official study guide is not enough – max 50% of the exam was based on the book’s reading material.

– Make sure to at least do the exercises in the book – believe me, some of the minute details (such as route 53 allows which type of record to have a TTL value? — This you can only get if you did the hands-on exercises).

– The book (2016 edition) does not talk about ECS yet it appears on the exam – what type of Load balancer allows for dynamic port mapping when used with ECS?

– Take additional time to study – don’t rush it. I literally crammed the book from start to finish in 7 days … DO NOT do this!

– I would strongly, strongly suggest doing hands-on exercises (if possible, by doing both from book and from linux academy). Just knowing the theory won’t help you pass.

– I just read the book and the only thing that helped me pass was my work experience with AWS. Therefore … do the exercises!!!

– Topics to focus on: autoscaling, VPC, networking, security groups, NACL, s3, cloudtrail, vpc flow logs, kinesis, cloud front, autoscaling groups.

– There were 55 questions and 80 min.

I would strongly recommend everybody who works with AWS (even if you don’t code) to get this certification. This cert. has given me understanding at higher level how services in AWS are connected to each other and now at least I can think in terms of AWS services when architecting a complexsolution.

Finally, this cert. allows me to talk and understand about AWS on any given topic.

Github Enterprise Setup (AWS)

Tool: Github Enterprise

Target Environment / Platform: AWS

Deployment type: PaaS (Github provided machine image — AMI)

Use case: Provide developers with a Source Code Management (SCM) tool.

Synopsis:

We first looked at utilizing github.com as our SCM to allow us for SaaS based Github offering. However, as of this writing, Github.com does not provide ActiveDirectory (SSO) integration – which meant users had to use local github.com userIDs that is why we decided to go with github enterprise.

Continue reading “Github Enterprise Setup (AWS)”