From The Trenches Part 8: Rebundled, Or Realizing I Wasn’t As Much of a Commodity as I Thought

By /

Last week, I spent a ton of time unpacking the “why” behind my experience with long-term unemployment in 2020. And perhaps rather unfortunately, the part of that saga that travelled the farthest was the sight of me, five years later, still bursting into tears on camera at the recollection of my last day at a company and job that I had loved for seven years.

But really, I didn’t want to dwell on how miserable an experience unemployment was. It does a number on your sense of self-worth, and the cognitive dissonance between getting an award in my 2nd year (the first year of being promoted from Mid to Senior), versus being redundant and unable to land even a Junior-level analyst job five years later, was why it took 18 months for me to achieve full emotional closure.

Sending hundreds of resumes out and expanding into increasingly junior roles when I didn’t land any Senior-level interviews convinced me that I was a commodity.  One of the things I said in my podcast was that the Year 2 award went to my head, and the problem was that I’d allowed myself to think I was a superstar for five years when I wasn’t really one.

But what if both things could be true? That I was indeed a superstar in Year 2, and still genuinely redundant in Year 7?

Unbundled, Then Rebundled: My Journey Back Into Employment

Year 2 of my business analytics career was 2014. In the early 2010’s, large corporations were still unlocking the growth potential from data-driven insights, while racing to integrate multiple on-prem databases into unified infrastructure. Cloud-based data lakes were still nascent.

I was unlocking immense value by being able to do two things: 1) relate the differences between syntax and data schemas between multiple on-prem legacy data sources to combine insights, and 2) transcend data asks to dig into the underlying business questions and answer them.

And the reason why I could do that was because I had a technical Master’s degree, so learning code came relatively easily to me, plus 10 years’ professional experience in business-adjacent government roles, culminating at the cusp of Director.

I was a scarce solution for the constraint at the time – fragmentation of data, with the need to do synthesis between data findings and business context, and not many people able to bridge that gap. The award reflected my scarcity.

But I wasn’t scalable. The market couldn’t get the hundreds (or thousands) of business analysts it needed by enticing mid-career professionals to ditch Director-level positions and start again at Mid IC level. I was willing to do it because I was seeking a new life in a new country and making a public-to-private sector transition. That was a rare, once-in-a-lifetime circumstance.

The answer to the shortage of Analytics professionals was that more universities started offering specialized Master’s degrees. And the Agile framework, already heavily in use for software development, brought business stakeholders and data analysts together in a scalable and structured way. By setting requirements, business departments could frame the question and the problem, and the ticket pipeline gave everyone visibility over what analysis work was needed and the status of request fulfilment.

With that transition, my unique edge of bringing business and data together was eroded. Meanwhile, I was lagging fresh graduates for the Analytics IC positions in the market (which skewed heavily towards Junior and Mid-levels due to where the talent pool was) because they fit the profile that could satisfy market demand at volume, whereas I didn’t.

I honestly thought my career was over. With six-figure sums spent on tuition for a Bachelor’s and two Master’s degrees, I was devastated that my education would be wasted if I’d never work again at age 42 with my career stalled permanently at Senior IC, not even ten years after getting a top global MBA. It felt like an indictment of my professional capability. After nine months of abject agony and desperation, my career got a new lease of life through rebundling. When packaged as a combination of analytics, performance marketing, and people leadership, my skill set earned an immediate pay bump of 20% with increased responsibility, learning, and growth.

Shifting Constraints in the Data World

The past two years haven’t been kind to data professionals. Part of that is coming from vendors and corporations experimenting with Gen AI and Agentic AI to create solutions where a business leader can ask AI a data-related question and get a fully formatted visualization in seconds, coupled with concise commentary.

I tried building an agent to automate business commentary earlier this year. It didn’t work, but that was because at the time, I didn’t understand that LLMs aren’t wired to do arithmetic. That’s beside the point, because there are providers and experts who can build these agents in a way where they could work with reasonable reliability after some iteration and testing.

What saddened me deeply was to see that the current mismatch in the market for data expertise – high demand but still the perception of oversupply – has been affecting compatriots with far more technical depth than I ever had.

I believe a part of this is simply that the generation of data professionals who wrangled messy on-prem datasets into scalable structures, two or three times over, have spent most of their careers solving constraints that are stabilized for now.

Over 12 years of doing analytics, I’ve worked with datasets in IBM DB2, Teradata with Toad, Hadoop with Qubole over AWS S3, Google BigQuery, and finally, AWS with Snowflake. The Big Three of today, namely AWS, Microsoft, and Google, are probably the players that will scale into the foreseeable future because that’s where the bulk of cloud capacity investments are happening.

A generation of professionals’ transformational work has turned the problem of messy and siloed data into neater, more scalable solutions. But work done for now will never be work done forever, because new businesses will start and conditions will change.

The skills of the 2010s’ data professionals are still as relevant in the next wave of AI-driven development as before. Perhaps even more so, as businesses get more data driven. But the market needs to find a new constraint for them to solve.

Tufte and Science as Art

While I was unemployed in 2020, I decided that it wouldn’t be practical to pursue yet another Master’s degree to keep myself employable in analytics, not from a personal budgeting nor a career trajectory perspective. It would leave me out-of-pocket with a huge tuition bill and send me into another regression back to Mid or Junior level at age 45, coming hot on the heels of my last Mid-level career restart only seven years prior.

But what I could do was to sit down and plough through the reading list that our data scientists had recommended for the rest of the team. The books were very technical and academic, but I allocated focus time to do all the examples personally. I also added one more book, not on the list, to my self-designed curriculum: Edward Tufte’s “The Visual Display of Quantitative Information”.

This book had been written well before data analysis became cool. From the examples – good or bad – you’ll find that even in the 19th century, people were already doing data visualization in ways that were both deeply informative and aesthetically beautiful. What’s more, “chartjunk” – the equivalent of today’s social media scroll-stoppers – existed in the 1980s, in the form of attention-grabbing data visuals for magazines.

The reason why I am mentioning this book, which was first published in 1997 and had a second edition in 2001, is because the line, column, and pie charts we use in day-to-day business might be easily automated with AI, but it takes humans to create imaginative visualizations that translate quantitative information into compelling visual soundbites to make a point. This, I feel, could be the new direction for data storytelling to turn BI professionals into artisans of data.

How I’d Do Agile Differently if I Had Another Chance

Agile got widely adopted in analytics at around the time when I went to the other side of the table, becoming the stakeholder setting the requirements and filing tickets.

In my early days as a Mid-level analyst, data requests were conversations. Sure, especially at the beginning, there were plenty of asks for “I want a report that shows Metrics X, Y and Z”, but for the most part, stakeholders would outline the broader problem they wanted to solve, leaving some latitude for me to exercise judgement and be a partner in curating and synthesizing the data to answer business questions.

My stakeholder team saw me as part of them, rather than as a support function or a service provider. In fact, when I relocated from the Geneva office to the headquarters in Bellevue, they sent me an edible fruit hamper as a housewarming gift, and later on, a voucher for a local artisan fabric shop (garment sewing is one of my hobbies) as a congratulatory gift when I secured a lateral transfer to pivot into performance marketing.

As Agile enabled the volume of requests to go up, that level of partnership got lost. I got lost, too, in the language of “requirements”. It became easy to be too prescriptive and task oriented. Sometimes, I got overly wrapped up in dictating methodology because I focused on getting problems solved at the quickest possible speed, rather than on ensuring the most collaborative outcomes.  In short, I was using my ability to roll my sleeves up and get hands-on in the wrong way, taking opportunities to show business acumen and ownership away from my Analytics partners.

It’s easy to blame Agile, but I think just as much as systems influence our behaviour, we have agency to decide how we want to use systems. My Analytics colleagues were concerned that the volume of analytical work that I’d been taking on, in an area that had previously been underserved, would cascade to them after my resignation. But I pointed out that there was a good deal of agency on both sides to flex the volume of work based on the business questions that needed to be answered. And that because I’d leaned in to ensure that the most critical questions for now had been thoroughly addressed and documented, the workload might come back after some time if market or platform conditions changed, but for now with everything stabilized, I would anticipate minimal increase to their workload in the short run.

Reframing the work from a pipeline of requests, over which the Analytics team had little to no control, to a living situation where both sides could collectively evaluate what business questions need to be asked, which ones are the most actionable, and then hand ownership to Analytics, was a thing I should have done earlier. And if I were to go back into tech someday, that’s the first thing I would do differently, to grow the next generation of hybrid professionals who can flex from analytics into business roles.

What Could Our Next Rebundling Look Like?

We’re still searching for what the next bundle of data leadership skills will look like. Will it be building, supervising, evaluating, and reconfiguring business analysis agents? Will it be setting up scalable cloud-based data infrastructure natively in new growth companies? Or will it be defining test-and-learn playbooks, optimization KPIs, and feature definitions for a new generation of deep-learning based marketing optimization initiatives?

I’m probably not the right person to forecast this crystal ball, but I’m trying to prepare for it anyway.  Up on my “to-learn” list will be coding and no-code ways of building custom agents for data analysis, after I finish my current executive education credential in marketing at the end of November. I do hope that those of us in the data world who are still seeking new areas of purpose and relevance will be able to find it in time to come!

In Part 9, to be posted at the end of this week (yes, I’m increasing to 2X weekly frequency to free up my time in December for hands-on practice with agents), I’ll share my perspective on how advertisers’ ad spend optimization is social platforms’ pricing and revenue optimization – and how that impacts the balance of power in the performance marketing ecosystem. Stay tuned!

References:

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Scroll to Top