Prologue
It’s 8:37 a.m. on a Thursday. I’m settling into my zone in my home office at Downtown Toronto — hot chocolate in hand, quiet workspace, phone about to go silent. Then, an unexpected phone call. It’s my internet provider — let’s call them “SomeRandomNetwork” — but one of the big 3 in Canada 🙂
Caller — “Hey, we’ve noticed your Wi-Fi tends to go off around 10 p.m. every night. Just checking — any issue on your end?”
I paused, taken aback.
Then I replied politely, “No, that’s intentional. I physically disconnect the router (due to 5G signal) before bed — for my sleep.”
The rep laughed, and so did I. But as I hung up, something about that call lingered. They had been watching my Wi-Fi habits within the intimate walls of my home. Tracking me. Even in my little corner of the world, data flowed silently — collected, analyzed, questioned.
I couldn’t shake the thought: My house, my devices… They’re all part of a massive data-generating system. I’d spent years designing analytics platforms — harvesting, modelling, transforming data for clients — and here I was, realizing that my home life mirrored those very systems. The irony wasn’t lost on me.
Even choices meant to disconnect, like unplugging my router, left traces — what I now understood as data exhaust. Every action, intentional or not, painted a digital portrait. My Wi-Fi habits were just one of many invisible footprints. My offline self might embrace silence, but my digital counterpart? It never slept.
That phone call marked a turning point. It made me think about the pervasive influence of Big Data — how it observes, shapes, and sometimes manipulates the world we live in. And I realized: understanding its biases and limitations isn’t just optional anymore; it’s essential.
Introduction
It’s noon, and I’m cruising down the 401 toward a client meeting in Ontario. At the back of my mind, I can’t help but think about how many apps have already harvested my data, and the day’s barely halfway through. Back home, my thermostat adjusts itself based on my usual wake-up time. My ride-share app knows my weekday routines better than I do. My smart assistant logs subtle patterns — volume changes, time triggers, even tone.
“timestamping my route with precision”
In the car, my GPS records every turn, precisely timestamping my route. Music App? It’s learning my tempo preference, mood shifts, and even time-of-day listening habits to serve a perfectly curated playlist.
Even if I went completely off-grid, there would still be traces: GPS pings, medical records, banking activity, streaming history. Every micro-interaction is silently recorded, analyzed, and monetized.
In today’s world, modern living is deeply entangled with data, from how we shop, commute, and scroll, to how we sleep.
Chapter 1 — What Is Big Data?
It’s easy to think of data as rows and columns in a spreadsheet. But in reality, data today is sprawling, chaotic, and persistent. Big Data isn’t just big — it’s beyond what your everyday laptop can comfortably handle. It’s too fast, too vast, and too varied for traditional tools. And what’s considered “big” is always shifting, depending on how powerful our machines become.
But that’s just the surface.
Life now unfolds as an interconnected web of data points: structured like a warehouse, flexible like a lakehouse, but intimate like a journal. Every swipe, every search, every GPS ping… creates a digital exhaust. Some of it clean and structured. Some of it messy and unstructured. But all of it — valuable.
And this is where Big Data Analytics enters.
The Why Behind Big Data Analytics
We don’t collect data for the sake of collection. We analyze it to find meaning. To spot patterns. To anticipate behaviours. From credit fraud alerts to app recommendations, the engine behind the scenes is always the same: take massive data, process it fast, and make sense of the noise.
As a developer, I live in this pipeline — building systems that integrate raw input from everywhere: sensors, mobile apps, online transactions, ride-share pings, fridge alerts, insurance forms, you name it.
Sometimes I wait and batch the data — great for dashboards and weekly reporting. Other times, the business can’t wait. So Stream processing lets us analyze data seconds after it’s created — critical in finance, healthcare, ride-sharing, even customer service chatbots.
But before any analysis can happen, the data needs to be cleaned. Trust me, dirty data lies. Like I said in my Data Quality article, garbage in, garbage out. Cleaning isn’t glamorous, but it’s where good insights are born.
From Big Data to Big Insights
Once cleaned, that data becomes gold. You can mine it, model it, and train it. You can predict weather, optimize delivery routes, detect disease patterns, or detect churn before it happens. Tools like deep learning, predictive models, and classification algorithms allow us to surface insights we never saw coming.
Big Data brings big value — and big risks.
Security gaps. Privacy breaches. Ethical gray zones. It’s one thing to analyze data. It’s another to be trusted with it. Because in the end, Big Data isn’t just about machines. It’s about people. And the digital lives we’ve chosen — or unknowingly created.
Chapter 2 — The 4 Vs and the Digital Crumbs
Finally, pull into the office parking lot for my client meeting. Swipe my card. Take a ticket. Another crumb.
12 pm (Noon)— From the moment I left home, data’s been following me like a shadow.
8:37 am — My smart thermostat logged the hour I got out of bed.
12:19 pm — My GPS tracked my route down the 401.
12:31 pm — My Apple Watch kept an eye on my heart rate as I changed lanes.
12:44 pm — A highway cam likely caught my license plate, maybe even my speed.
1:03 pm — By the time I parked, my digital footprint was miles ahead of me.
And it doesn’t stop there.
1:17 pm — At my downtown office, I badge in. The system logs my check-in time.
My productivity apps track every click, scroll, and idle second.
2 pm to 5 pm — My messages, meetings, lunch order, even the time I spend reviewing dashboards — it’s all recorded somewhere.
Even my intentions are data. I mention I need a new couch, and suddenly, my phone, TV, and browser serve up matching furniture ads like mind readers. That’s not magic. That’s tracking.
I wont get too technical how all these work, but majority of apps / devices collect personal data in some way — there’s a good and bad element to it.
The Everyday Sources of Data
Smartphones. Browsers. Social media. Fitness trackers. Surveillance cameras. Iot sensors. Streaming services. Credit card swipes. Loyalty programs. Your entire routine is a goldmine for data collectors.
And to make sense of this flood, we turn to a foundational framework in Big Data Analytics: The 4 Vs.
Volume
Data is big. It’s not megabytes or even gigabytes anymore — we’re talking zettabytes of data created every single year. That’s 1 billion terabytes. From personal messages to surveillance footage, the sheer size of what’s stored is almost unfathomable.
Velocity
Data doesn’t just exist. It moves fast. My location updates in real-time. Spotify adjusts my recommendations before the next song. Mobile Apps know what I’ll like next before I do. Data is created, collected, and processed instantly across millions of apps and sensors.
Variety
Forget spreadsheets. Today’s data comes in every shape and form:
- Structured: Spreadsheets, database tables
- Semi-structured: JSON files, XML from APIS
- Unstructured: Voice notes, Instagram videos, screenshots, even GIFS of me reacting to those couch ads
But note that each format has its quirks, storage needs, and processing challenges.
Veracity
Not all data is clean or truthful. Noise, duplicates, and missing values are constant threats.
If my Apple Watch thinks I’m jogging when I’m driving, bad data.
If my GPS pin drops in the wrong plaza, bad data. The reliability of data — its veracity — is crucial, or we risk making bad decisions from bad inputs.
The bottom line?
Whether I’m asleep in bed or stuck in traffic, my life has become a streaming service of data points — playing on loop, 24/7. And in the world of analytics, everything counts.
Press enter or click to view image in full size 
Chapter 3 — GPS Told Me to Turn Here: Bias in Big Data
I once read online about a guy who drove straight into a lake because his GPS told him to. At first, I laughed. Then I thought about how often I trust my apps without question, no matter how off they feel. That’s not just convenience… that’s automation bias in action.
Bias is everywhere. We talk about it in hiring, media, politics — but few stop to realize: bias lives in data too. Big data may sound objective, but it’s created, collected, cleaned, and coded by humans. And we? We are the most biased variables in the system.
Bias in Data Collection — The Convenience Trap
Let’s say you want to analyze workout patterns in a city. You pull data from smartwatches and fitness apps. But guess what? You’re missing people who can’t afford wearables, or simply don’t use them. That’s selection bias.
Worse, your data might reflect historical bias — inequities baked into past decisions. Policing, credit scores, and healthcare — all these domains carry the weight of societal bias. And when we collect that data, we carry that weight forward.
Suggestion? Widen the net. Use diverse sources. Don’t chase only clean or easy datasets. And always ask: who’s missing from this picture?
Bias in Algorithms — Code Doesn’t Equal Neutral
Facial recognition once misidentified dark-skinned women at a much higher rate than white men. Not by accident — but by design. The algorithms were trained on biased data, built by biased teams.
This is algorithmic bias — when machine logic learns human prejudice and scales it like wildfire.
Sugesstion? Transparency. Use representative training data like the PPB dataset. Audit models regularly. Be loud when things go wrong.
Bias in Interpreting Results — We See What We Want
Even when the math is sound, our minds bend the results. Ever cherry-picked a stat to back up your hunch? That’s confirmation bias. Ever stretched insights from one group to everyone else? That’s overgeneralization. Ignored results that didn’t fit your story? Reporting bias.
Sugesstion ? Define goals early. Document every assumption. Share failures, not just wins.
Big Data, Big Responsibility
The GPS is helpful, but it doesn’t replace your eyes. Data is powerful, but it doesn’t replace your judgment. Bias isn’t just a glitch in the matrix; it is the matrix. And if we want to build better models, we first have to debug ourselves.
Chapter 4— The Entire History of You
Before I wrap up, let’s showcase what a “FICTIONAL” analytical model will look like. You don’t need a giant enterprise to run a data warehouse (This section was inspired by a Netflix TV Series).
You are the warehouse. In my head; “A married couple is just a FULL OUTER JOIN operation between two datasets (bodies).”
Your life, whether complex or quiet, generates operational logs, which flow into fact tables, enriched by dimension tables, all eventually visualized as a personal KPI dashboard.
Every moment is recorded. Every movement, a metric. Every decision, a datapoint.
Analytical Model: Life, Logged
Your Operational Event Log (Streaming in Real Time)
These are the raw signals your devices generate before they’re transformed into structured data. Think of it as a Kafka stream of you.
{
"event_time": "2025-04-25T07:31:45Z",
"event_type": "transit_card_tap",
"location": "Union Station",
"device": "PrestoReader_002",
"user_id": "me_001",
"lat": 43.6453,
"lon": -79.3806
}
Above, this is your operational data store (ODS) — where raw, messy life gets staged before it’s cleaned and modelled. Before you know it, it’s part of a fact table.
Fact Tables — The Highlights of Living
These are time-based, measurable events in your life.
Dimension Tables — Okay, let’s give our life a bit of context
These give your fact tables meaning — the who, what, where, when, and why.
Arranging Your Life In Data Marts below:
Your KPIS — The Dashboard of You
You don’t need Tableau or Power BI to know your story, but if you did…well
Aggregated Data
Life Modelled. Life Measured. Life Lived.
You don’t need a smartwatch to know you’ve been moving. But when that motion becomes a metric, it gains memory.
Your life leaves a data trail — one that’s human, habitual, and analytical all at once. And whether you analyze it or not, that trail speaks. This final chapter looks like something lifted from an Episode of Black Mirror. I know.
Conclusion
In trying to unplug, I discovered I was still connected.
Even silence had a signal.
Big Data wasn’t just watching — it was learning, predicting, shaping.
And in that realization came clarity:
“To live freely in a digital world, we must first learn to read the footprints we leave behind.”
Thanks for reading!
