Over 71,000 tech jobs disappeared in Q1 2026 where Oracle alone cut 30,000, twelve thousand of them in India. The reason given, every time, was some version of the same line: AI changes what we need.

But here's the part that gets less attention: in the same quarter, data engineering roles continued to grow. Demand for big data specialists is projected to more than double by 2030. Companies building AI systems are hiring data engineers faster than they were last year.

So the story is that two completely different versions of the data engineer now exist and the market is rapidly choosing one over the other.

The ones getting hired aren't necessarily better at SQL or Spark or Airflow. They're better at working with AI. They build data platforms that AI systems consume. They use AI tools to work at a level that would've been senior-architect territory three years ago. They think about data as the foundation for intelligence.

The ones getting let go, in many cases, are still excellent at a version of the job that's fading. They maintain pipelines, run batch ETL and hand off clean tables to analysts. That work still exists, but it's increasingly the kind of thing that AI tooling can either automate or drastically simplify which means companies need fewer people doing it, and they're willing to pay less for it.

This is the split. And it's going to define data engineering careers for the next decade.

We've watched this split from the hiring side. Ideas2IT has been bringing on data engineers and data architects through this entire cycle, and the pattern is consistent, the people we hire, and the people who thrive once they're here, are the ones who've already crossed over to the AI-fluent side of this line. Murali Vivekanandan, our founder, touched on this in a recent India Today piece on the layoff wave: the companies hiring right now aren't doing it despite the uncertainty. They're doing it because they're clear about what AI actually requires.

This piece is about what that clarity looks like in practice, what the AI-fluent data engineer actually does differently, and why the environment they work in matters as much as the skills they carry.

How has the data engineer's role changed in 2026?

The data engineer's role in 2026 has shifted from pipeline builder to platform architect, designing the data infrastructure that AI systems consume and not just the pipelines that feed dashboards and reports. Most "data engineering trends" content  hands you a skills checklist, Python, SQL, cloud certification, Spark that was useful in 2021 but misses where the real shift happened: not in the tools, but in who the work is for.

Two years ago, a data engineer's main consumers were analysts and BI tools. You built pipelines, cleaned data, modelled it for dashboards, maybe set up a warehouse. The feedback loop was long and forgiving, if a batch job ran overnight and populated a report by 9 AM, you'd done your job well.

Today, the consumer list looks completely different. RAG pipelines need chunked, embedded, well-attributed data served with low latency. Feature stores need versioned datasets with clean lineage. ML training loops need reproducible snapshots of data states from specific points in time. Agentic AI systems need context-rich, governed data delivered in formats that traditional batch architectures were never designed to handle.

And here's the part that makes the divide so sharp: AI tools have simultaneously made the old version of the job dramatically easier. LLMs can write SQL, scaffold DAGs, generate transformation logic, and even debug pipeline failures. The "pipeline builder" skillset is the thing many data engineers built their careers on and is getting commoditised in real time.

What AI can't do is the harder stuff: understanding which business process depends on which data contract, anticipating how a schema change will cascade across a dozen downstream consumers, designing governance into a system instead of bolting it on later, making cost-performance tradeoffs that require understanding both the cloud bill and the business context. That's the part of data engineering that's becoming more valuable.

‍

The split, in a sentence: the "data" part of data engineering is getting automated. The "engineering" part is becoming the whole game.

What does AI fluency actually mean for data engineers ?

AI fluency for data engineers means two things happening simultaneously: building data platforms that AI systems depend on (feature stores, vector indexing, training data versioning) and using AI tools to operate at a higher architectural level than was previously possible. It is the combination of these two capabilities building for AI and building with AI that increasingly separates the data engineers getting hired from the ones getting let go.

Every company now claims to care about AI fluency. For most, it means "we'd like candidates who've used ChatGPT." That's just awareness.

Zapier recently published something more honest. They created a formal rubric four levels, from Unacceptable to Transformative that they apply to every single hire. The minimum bar isn't "have you tried AI." It's "show us a repeatable system you've built with it, and the measurable impact it had." Their CEO was direct: if you're not meaningfully improving your work with AI, you don't clear the bar.

We think about this differently at Ideas2IT our context is different, since we're building data and AI systems for enterprise clients rather than optimising internal SaaS workflows. But the underlying principle holds: AI fluency doesn't work as a top-down mandate or a side initiative. It has to be in the infrastructure, the workflows, the daily rhythm of how engineers learn and build.

At our end, that's meant embedding AI into the SDLC across projects, running an internal AI Academy focused on building production systems rather than running prompt experiments, and pursuing external validation like AWS GenAI Competency. The numbers bear it out roughly 40% improvement in per-developer productivity so far, and compounding as teams build shared patterns.

Why does the org-level stuff matter for an individual data engineer weighing their options? Because a single AI-fluent person at a company that doesn't get it is swimming against the current every day. They're fighting for tooling, justifying time spent on AI-assisted work, explaining to managers why the old way is slower. An AI-fluent engineer inside an organisation that's already built the muscle the shared tooling, the workflows, the culture of building with AI as a default that person is compounding. The environment is a multiplier, and it's one of the most underrated factors in career decisions right now.

For data engineers specifically, AI fluency shows up in two directions at once.

You're building the platforms that AI systems depend on. Feature stores, training data versioning, vector indexing, metadata tagging, data quality gates the entire substrate that ML and LLM systems rely on. You don't build the models instead you build the thing that makes the models useful or useless.

And at the same time, you're using AI to do your own work differently. The boilerplate generation, the initial pipeline scaffolding, the routine transformation logic, AI handles that now. Which frees you to spend your time on system design, governance architecture, cost optimisation, the decisions that determine whether a platform scales or buckles. The data engineers who resist this trade are spending most of their week on work a well-configured AI session can do in minutes. The ones who embrace it are doing the most interesting work of their careers.

That intersection building for AI, building with AI is what defines the AI-fluent data engineer. Your definition is not a certification or a tool on your resume. It's a way of working that compounds over time, and it's increasingly what separates the people getting hired from the people getting cut.

What does an AI-fluent data engineering team look like in practice?

An AI-fluent data engineering team operates in an environment where AI is embedded in the development lifecycle by default where engineers design platforms from scratch for AI-native workloads rather than maintaining legacy systems. The org-level infrastructure matters as much as individual skill, because fluency compounds when the tooling, workflows, and institutional knowledge all reinforce it.

The perspective in this piece is shaped by what we see at Ideas2IT, so it's worth being transparent about what that environment looks like.

Our data engineers design platforms from scratch across industries healthcare data infrastructure, financial services, location intelligence, clinical trial platforms. The work rotates, the tech stacks vary, and the problems tend not to repeat. We've also built six of our own products through IdeaLabs like LegacyLeap, Anticlock.ai, Migratix, Explayn and others.

On the stability side: 16 years, zero layoffs, low attrition. There's a stated principle "Employees > Clients" that's been acted on, including walking away from engagements that weren't good for the team. That context matters when you're reading a blog about a market where people are being let go through automated emails.

Data engineer and data architect roles at Ideas2IT

We're looking for Data Engineers and Data Architects in Chennai. If what you've read here matches how you already think about the work, or how you want to be thinking about it, it's worth a conversation.

‍Reach out here or write to recruitment@ideas2it.com.

‍