I’ve been frustrated many times when one’s capabilities don’t match with what’s been advertised or promised or promoted. Competing for job positions in the Linkedin era have made it worse. This behavior is mostly visible in the Data Science field where it suffers from bloated and exaggerated skills set.
My views on Data Science
DS will be in much better shape, when all industries agree (by the current widely-known general definition) DS is actually Business Intelligence++ or Data Analysis++. This change has already happened years ago in major tech companies like Amazon, Facebook, Lyft and industries need BI and DA to get insight of their growing data, build dashboards and visualizations, present the results to stake holders. But I’d stop right there. Expanding it to Data Engineering and correctly doing ML from Development to Production need (a lot of ) Software Engineering skills. These are kind of skills that no number of bootcamps or MOOC is sufficient. They’re good for getting started but often would give rise to (isolated) Expert Beginners if one is not careful because they fail to estimate how much knowledge is out there that they don’t know.
SDE skills like other skills need deliberate practice building, breaking and maintaining non-trivial systems. There are Data Scientists who have transitioned into SDE either directly or indirectly where I’d call it Machine Learning Engineer (MLE for short) or some would call it Full-Stack DS.
This distinction is critical when it comes to the right implementation of building a product on data. I echo the similar opinions as written in here and here that a correct implementation of building a product on data should be end-to-end as much as possible and BI++ are included in the end-to-end approach but should not be responsible for the end-to-end system simply because they haven’t been trained for and don’t have the skills to build the end-to-end approach.
How all this relates to the title of this blog?
The above explanations capture my motivations of writing this blog but I want to generalize (like a Math enthusiast would do). I think what fuels some of my frustrations and other are also coming from inability to have an honest self-evaluation of one’s self where there’s no recruiter, no Linkedin, no job application, no resume in the picture. It’s this honest self-evaluation that gives and directs meaning to growth.
Here are my suggestions
Realize expertise is a fallacy
The Dreyfus model of skill acquisition is not sufficient for one’s self to do an honest self-evaluation. True self-evaluation doesn’t stop at expertise but enriches it with constant learning and (the famous quote of those who made the most number of mistakes) to make less mistakes and make better decisions. I’d argue that a true “expert” doesn’t call themselves an expert because they’re aware of how much knowledge is out there and how much they don’t know. Also one’s age or the number of years being in a profession are not sufficient at showing necessarily one has reached the status of an advanced user who makes good decisions and/or can estimate and plan for the consequences.
Practice deliberately the hard-way
Have the beginner’s mindset. Put time and effort to practice deliberately. Read a lot and practice a lot by building non-trivial stuff. Realize that no number of blog posts, books, MOOCs are sufficient if not accompanied by the first hand experience of building and creating.
Have someone(s) to look up to and aspire to be like them
At any stage, find capable people in your profession and ask for mentorship directly or indirectly by observing and grasping their work. Indirect mentorship at more intermediate to advanced level could also mean learning from collective efforts of a group of people. For example, in the Software world, it’s easy to find high quality projects and follow their maintainers or core developers and study (real deliberate study) their codes.
Hope this post serves you well. Keep learning, practicing and building non-trivial stuff.