Tags

, , , , , , , , , , , , , , , , , , , , , ,

Well, here we are again, as of Dec 2020 on another journey I have undertaken – attempting to move over from AWS Cloud Solutions Architect and into an AWS Specialist Solutions Architect Big Data Engineer role (AWS has rebranded it to Data Analytics).

Since starting on my Cloud journey, I have gained AWS experience but never had a chance to gain Big Data engineering experience. And because I do not have hands on Data Analytics Engineering (not Data Scientist) work experience, I decided I was not too proud to reach out and ask others for pointers. Instead of flailing about, the best course was to ask for pointers on what I needed in order to enjoy a smoother path into Data Analytics Engineering.

But when I joined and reached out to different Big Data & Data Analytics Engineering and FinTech groups for pointers – I completely struck out. It seemed as if no one wanted to provide any kind of pointers.

I had stated the following to create some kind of feedback impetus:

a) MBA (Strategy, Innovation & Tech Management)
b) MS Cybersecurity
c) AWS Cloud Solutions Architect Pro cert (& a lapsed AWS DevOps Associate cert)
d) Azure 900 cert

And working on:
— 2 different AWS Data Analytics (Big Data) courses
— since no two instructors teach the same material and some are better than others

Along with a DVD course from the Great Courses, topic – Big Data: How Data Analytics is Transforming the World.

Plus, from my previous college work, which is somewhat rusty, I have had two semesters of Statistics and other advanced math like – Multivariable Calculus; Differential Equations and of course, Physics. But, no idea at this point if those areas are necessary for a Data Analytics engineer. (If it is, I have material at home to visit and refresh some of that knowledge….)

No idea as to why no one wanted to step up (even privately) to provide any pointers, it is most interesting. And I am the type to listen to everyone, junior or experienced – everyone may have some solid nuggets of information to glean useful knowledge from. Because of not knowing who knows what, I listen up. And if you are smart or purport to be – you ‘better’ listen to those around you.

  • Could it be that I am just not in the correct forums?
  • Everyone is too busy, especially with CoViD still taking place?

But in joining Big Data and FinTech groups, those ‘should’ be the right places to go to gain some solid info.

In this scenario, the next best bet is to do my research and forge ahead, and “hope” I do not end up flailing about…

And as I journey, I will provide more content for others in the same boat. Not having hands on experience really does slow down any tech job, so the only recourse is to learn as much as you can, as fast as you can. And try to gain as many credible certifications as you can.

However, what I already knew, but wanted to highlight it again here for others – be careful of rabbit holes. When I study, I may come across a point where I need to stop and gain more information or another perspective elsewhere. So, while I do enjoy learning new material, I do the digging but have to make myself stop any further rabbit hole traversals….

For example, I was not a Hive or Hadoop or YARN person until I started these Big Data courses. In those moments when I come across content like that, I may need corroboration so I start checking other sites such as: an AWS Blog site, or Stackoverflow site or somewhere else. Then, once on those sites and during the reading, they might refer to another site. At this point, you need to rein it in and go back to your course work, which I do.  Or else, I will keep going down one tangential path after another to learn more on EMR, EMRFS, HDFS, etc…

My courses are from:

  • ACloudGuru and
  • CloudAcademy

and any other site that has yumptious tid-bits on any Big Data content that I can devour.

Because of my insatiable knowledge of more, more, ‘more’ – I could have finished one of the courses two weeks ago and be nearly finished with the second one at this point.

If you are anything like me, you do not just take a course without doing more digging on a topic to gain a better understanding of what you are learning.  Take HUE for example, that is Hadoop User Experience and what I see is a no-brainer.  Why use anything else, HUE does most of the work you need, whether is doing any kind of SQL work in Hive/Hadoop or on AWS S3. Creating tables, populating those tables, etc….

Then there is Parquet – I never knew anything about that but discovered it is a fantastic Hadoop tool to do compression and columnar formatting to and from JSON and .CSV files. To do columnar specific queries instead of doing row queries across an entire ultra-massive database is a no-brainer. Pair this with SQL and you have a winner.

But Parquet and HUE are just a pair of good tools you can use for Data Analytics.

Python is another area where I only have basic knowledge of, but – ya gotta learn it.

Then, there is the matter of having a computer background, which may either:

  • be a bonus and speed you along in your Big Data engineer journey or
  • it may hinder and hurt you (not likely though)

It could be a bonus because you WILL have a foundation to work from, in all the different aspects of computing. But! It still depends on what kind of computing background you have. Is it:

  • Mainframe based, PC network based or Cloud (AWS, Microsoft, Google, etc.)
  • networking (data) and routing
  • administration of a network

And with a computer background/foundation, it will depend on how flexible you are in adapting to the Big Data (Data Analytics) world. If you are a brute force type of person, it could take longer to get up to speed in this area.

What I mean is, if you are strictly a linear thinking individual, you may not see beneficial adjuncts or tangential uses of various software and tools. This is where being a Lateral thinker comes in to play – this kind of thinking outside the box; going from A to C and then to K with the desired successful outcome – without causing harm/damage and meeting all requirements – means a faster transition with possibly considerable cost savings (in whatever you are working on).

If you do ‘not’ have a computer background, other than college, you “might” have a fresh clear perspective with no prejudices to work from. You could also potentially have a leg up on others without college (or very recent college) because you ‘may’ have gotten into courses that involved cutting edge software/tools.

Or, it may hobble you because you do not have that broader base to work from. On this aspect, it means you ‘may’ have some catching up to, especially if you did not major in computer science in college and have no hands-on work experience.

Having an extensive computer background allows you to do more extensive troubleshooting on your own. Being able to ask other probing questions to get to the root cause of some issue.

But, human nature being as fickle as it is, there are winners in both scenarios – those with a computer background and those with only a college education.  Bottom line – you simply cannot block anyone with drive, an open mind and able to adapt to anything on the fly. If one is open to listening to others, no matter their background or title or any of the other societal issues (discrimination due to racism, sexism, ageism; inequalities) – then you have it made in growing in Big Data or any other field you choose to work in.

Being a techie all my adult life, moving from AWS Cloud Solutions Architect and becoming an AWS Specialist Solution Architect Big Data engineer only makes sense for me. I enjoy data. I enjoy what can be done in the cloud. Combine these two together, and you have me.

Because of not gaining any hands-on work experience in Big Data to date, we (in this case – me) may have to gain a few certifications to make the leap/transition and possibly need to take an initial pay cut as well.

As a result, I am looking to make an expeditious move – NOW. My mode is – move on to somewhere else.  In Big Data/Data Analytics.

There will be more to follow as I make time to write more content on this topic/journey.