“He’s still s***’in whether he’s on salary, paid hourly
Until he bows out or he s***s his b****** out of him
Whichever comes first, for better or worse
He’s married to the game…” – Eminem, “Not Afraid”
What do data analysts and Eminem have in common? They’re married to the game, except that in the former case, the game keeps changing. Luckily for Eminem, he’s in a game where he can still be the GOAT (Greatest Of All Time), even in 2025.
I’ve had my share of 5 am mornings, especially when my morning commute to the office took 60-90 minutes depending on traffic. But this one was different. I stumbled out of bed to log onto my professor’s Welcome Webinar – the only direct contact we would have with him in our 8-week executive education class – and what did I hear? That in creative testing, A|B tests were systemically being replaced by deep learning algos that could pair infinite creatives to infinite consumers and predict engagement metrics before we even sent any ad creatives out into the wild.
Welp. At the place where I’d “grown up” in tech through the 2010s, A|B testing rigour had been the holy grail. We’d religiously watched test-and-learn dashes that measured statistical significance and called winners and losers. In my last-held FTE job, I’d been trying to translate that framework into the complexity of Paid Social creative testing, developing taxonomy to translate the intangibility of creative style into measurable building blocks.
Barely four weeks before this, I’d been reworking my resume to pitch myself as a freelance performance marketing consultant. And Claude (yes, Anthropic’s Claude) was patting me on the back.
Claude: You ARCHITECTED a test-and-learn framework! You’re a principal-level executive!
Me: Umm… Execs are VPs, aren’t they? I’ve never been a full VP, only an Assistant VP in a non-tech previous life 15 years ago. I’m not sure people will appreciate me saying I’m a thing I’m not.
Claude: But you are! You ARCHITECTED stuff! Roadmaps are STRATEGY! You’re an exec and nobody will know it until you sell yourself as one!
Apparently, not. Later that morning, the last of my illusions came crashing down when I watched my prof’s recorded video for the module of that week. He confirmed what he had already intimated at the live session. A|B testing is going the way of the dodo because it takes too long to go to statistical significance, and it restricts us to testing only two things at a time: Option A and Option B. Why plug away at this when Gen AI can create hundreds of variants and pushing them out into a platform-based deep learning algo can create hundreds of outcomes, all in the blink of an eye?
The Upwork Wake-Up Call
I’d already received a huge wake-up call when I logged back into my Upwork profile, which was ossified in 2020. For five years of FTE work, I’d neglected to update it. So, my Senior-level business analyst self was showing up in the algo that sent me recommended gigs. To my abject horror, the hourly rates for Google Sheets data visualization work, which was still my bread-and-butter as an FTE, went for as low as $20. In Seattle, that was minimum wage.
Note: Things got slightly better by the time I took this screenshot. In early September 2025, data viz gigs went for a strict $20 but the range expanded to $20-45 by October 2025. I guess people realized that LLMs might not be the universal data-viz panacea that they were touted to be for the last 6-12 months.
“Ahem,” I thought, “if I’m going to earn $20 an hour doing analytics, I’d much rather earn it being a barista at one of the Eastside’s boutique cafes. At least I’ll meet people and have fun!”
After gathering the data points to post my findings, I closed my Upwork account. I’ll probably re-open it someday with a new account (or shift to Toptal), but I knew my profile needed a complete purge and reboot.
What Are We Married To, Really? Being The Smartest Person in the Room
Taking the choice to rebrand my Upwork positioning forced me to face a hard question: why was I clinging to pitching myself as “a data person” when I thought it would be more fun to be a barista if there was equal pay?
It wasn’t all about the money. When I decided to join the gig market, I’d been prepared to do tactical gigs at $30-$40 / hour to grasp the fundamentals of SEO and CRM, less than half of what I’d earned as an FTE. This eventually morphed into full-time content creation at no pay. And I made that shift with no regrets at all. I’m having more fun and feeling more intellectually stimulated than I’ve been in 25 years.
But for over a decade, I’d been conditioned to expect that data work was worth more, not only financially but societally, than those other things I was intrinsically more motivated to do. The pay reflected the belief that data people were the smartest people in the room.
And so, I believed I had to do data to make the best use of my STEM education. That was what kept me married to the game for so long – I believed that STEM was for driving economic progress whereas writing and video creation were for fun. That by doing data, I was choosing the serious work instead of the frivolous work. In short, I was married to the illusion of being the smartest person in the room.
Perhaps it wasn’t always an illusion. The term “big data” was first coined in the 1990s, but at the time when my first tech employer embraced it, the industry-wide upward curve still hadn’t taken off (and wouldn’t do so for another 5+ years). Scrappy, business-savvy people who had strong enough mathematical foundations to learn SQL on the job were the backbone on which the early adopters of big data laid the bricks for scalable, automated reporting solutions and data-driven insights. At that time, perhaps we might indeed have been the smartest people in the room – or at least, the most data-equipped people. The flavour of all our talk about AGI today shows that we haven’t moved away from equating access to data with intelligence.
At my peak earning capacity, I received over $235K total compensation per annum. And I walked away from it to be greeted by gigs at minimum wage. Whether on salary or paid hourly, is 2025 the time for me to stop being married to the game of being a data analyst?
The Commoditization of Data Science: Capacity Scales, Certifications Blossom
When there is demand, there comes supply. And so, as “big data” became more ubiquitous in the latter half of the 2010’s, the education sector stepped up to fill the need for such professionals.
As shown in the table below, over the one-and-a-half decades from 2010 to 2025, the number of universities granting data science or data engineering-related degrees has ballooned from just six in 2010 to over 100 by the end of the decade and is still growing.
Meanwhile, the volume of data in businesses is ballooning with the emergence of hyperscale infrastructure and the ability to process natural language. Scrappy wielders of SQL from the early 2010s, like me, are being left far behind in the dust. Perhaps none of us are the smartest people in the room anymore when deep neural networks mean that machines can process huge quantities of data faster than any of us can.
| Period | Technology | Qualifications & Labour Supply |
| Mid-2000s to early 2010s | Relational databases stored on-premises using SQL Server, IBM DB2, Teradata, Oracle | In 2010, only 6 universities granted degrees in data science, data engineering or business analytics. Experienced professionals trained on-the-job filled the demand gap. SQL was primary skill set to extract business insights. |
| Early or mid-2010s | Hadoop allowed bigger volume and complexity of data. Mix of on-prem (on-premises) and cloud solutions emerged. Beginnings of the data lake taking over from structured databases. | By 2014, 20 universities were granting degrees in data science, data engineering and business analytics. Emergence of data science bootcamps for mid-career professionals. Advanced statistics and machine learning knowledge needed to design, test and deploy predictive modelling. |
| Late 2010’s to 2020’s | Data moves into the cloud. AWS, Google BigQuery, Microsoft Azure emerge as top solutions. Data lakes become the norm. | By 2019, 101 universities granted degrees in data science, data engineering and business analytics. With increasing volume of data, especially fuelled by LLMs and GenAI, deep learning and neural networks can achieve more nuanced outcomes than traditional machine learning. |
With Huge Labour and Infrastructure Supply, Tacit Knowledge May Still Have Value
The same forces enabling the “Shein-ization” of Paid Social are the ones driving traditional data wrangling techniques to the wayside, crumbling the institutions that formed entire disciplines of expertise during my heyday as a data-driven marketing professional in the 2010’s.
As the scale of computing infrastructure continues to grow, it will put immense power into the few hands that can afford it. I think so, because for half of my exec education course, I’ve sat bug-eyed asking myself the question, “How many companies have the financial reserves – or the richness of data – to even afford this?”
Amidst the doom and gloom of watching my past career crumble in front of my very eyes on that mid-October morning, I found one bright spark. Even if creatives are going to become the cookies which we feed into an unfathomable cookie monster while we wait for what it spits out, somebody must train the monster to be a connoisseur. And the taxonomy which I’d spent half a year coming up with? Perhaps it could be the beginnings of the new gold because it identified the features of creatives that I thought would be important for driving performance.
In Part 8, I’m going to talk about how data analysis can be both a science and an art. Even if machines own the “science” part, we humans still have a role to play in the art of guiding the machine towards answering the right questions and turning data into insights and outcomes.
References:
- History and popularization of the term “big data” – https://fsulib.com/what-is-big-data/
- Governors State University paper on data migration from relational databases to Hadoop, 2016 – https://opus.govst.edu/cgi/viewcontent.cgi?article=1180&context=capstones
- University of Wisconsin 2017: A Modern History of Data Science – https://uwex.wisconsin.edu/stories-news/history-of-data-science/
- American Statistical Association blog on the emergence of Master’s degrees in analytics & data science (last updated 2022) – https://community.amstat.org/blogs/steve-pierson/2016/08/12/the-emergence-of-masters-in-analytics-programs
- RTInsights: How Hyperscale Innovations Have Changed The Cloud Migration Calculus – https://www.rtinsights.com/how-hyperscale-innovations-have-changed-the-cloud-migration-calculus/
- DeepAudio and Email Open Rate Prediction, University of Miami 2025 – https://scholarship.miami.edu/esploro/outputs/doctoral/AI-for-Content-Management/991032598405902976