Reader Settings

Font Size20px
20px

Blog Post

How It Is to Be a Data Scientist

Vishesh Yadav
7/28/2026
5 min read
How It Is to Be a Data Scientist

Being a data scientist at IISc sounds impressive mainly because people do not know what either of those words means.

“Data scientist” sounds like you are studying data in a laboratory.

You wear a white coat, put a CSV file under a microscope, and announce:

“Interesting. The third column is suffering from missing values.”

And IISc sounds like a place where intelligent people are constantly discovering things.

Which is technically true.

Every day, somebody discovers that the server is down.

Then somebody else discovers that the person who knows why is on leave.

Then a third person discovers that the server was never down. You were connected to the wrong Wi-Fi.

This is research.

The Introduction Problem

When you tell someone you work as a data scientist at IISc, they usually react in three stages.

First, they become impressed.

Second, they become insecure.

Third, they ask you to fix their laptop.

Apparently, working with machine learning means you are responsible for every machine.

Printer not working?

Machine learning.

Phone storage full?

Artificial intelligence.

Microsoft Excel has changed the date format?

Please use deep learning.

Nobody asks a civil engineer to repair their water bottle. But once you say “data scientist,” every electronic object in the room becomes your child.

Even your family starts introducing you incorrectly.

“This is our son. He works in AI at Indian Institute of Science.”

Then they pause, allowing the other person to understand that they have won.

Indian parents do not introduce children. They publish benchmark results.

The Campus

IISc is beautiful.

There are trees, old buildings, birds, quiet roads, monkeys, laboratories and people walking around as if they are carrying information that could change humanity.

Sometimes they are.

Sometimes they are carrying a charger.

The campus creates a very specific psychological effect.

You are surrounded by so much intelligence that even being confused starts feeling academic.

Outside IISc, when you do not understand something, you say:

“I don’t know.”

Inside IISc, you say:

“This requires further investigation.”

You have not failed.

You have identified a research gap.

This is the first important skill you learn.

The second is finding an empty meeting room.

What a Data Scientist Actually Does

People imagine data science as a clean process.

You collect data.

You train a model.

The model becomes intelligent.

You publish a paper.

Science moves forward.

In reality, the process is closer to this:

You collect data.

The data is wrong.

You clean the data.

The labels are wrong.

You fix the labels.

The timestamps are wrong.

You fix the timestamps.

The server runs out of memory.

You reduce the batch size.

The model trains.

The results look excellent.

You become happy.

Then you discover data leakage.

Now the model was not intelligent.

It was cheating.

Which means the model has successfully adapted to the education system.

You spend three months preventing an algorithm from copying the answers, even though the algorithm was trained entirely by researchers who survived by copying assignments.

The machine is learning from us faster than expected.

Data Is Never Just Data

In theory, data is information.

In practice, data is the final form of several people misunderstanding each other.

One person designed the sensor.

Another person configured it.

A third person recorded the output.

A fourth person renamed the columns.

A fifth person sent you the file.

By the time it reaches you, “time” is stored as frequency, “frequency” is stored as text, and one column contains the value “final_final_updated_2.”

You ask what it means.

Nobody knows.

But everyone agrees that it is important.

Research datasets often have the personality of inherited property.

Nobody knows exactly where they came from, several people claim ownership, and touching one column can start a dispute.

Then you are told to build a robust model.

A robust model means a model that continues working despite the data, the hardware, the assumptions, the documentation and occasionally the researcher.

Meetings

Research meetings are interesting because everyone enters with a different understanding of why the meeting exists.

One person wants a progress update.

One person wants a technical discussion.

One person wants to know why the experiment has not finished.

One person has joined because the calendar invite looked compulsory.

The meeting begins with:

“So, where are we?”

This is a dangerous question because there are many possible answers.

Physically, we are in Room 308.

Technically, we are debugging preprocessing.

Scientifically, we are exploring representation learning.

Emotionally, we are finished.

But you cannot say all this.

So you say:

“We are seeing some promising initial results.”

This sentence is the backbone of Indian research.

“Promising” means the graph exists.

“Initial” means do not ask why it changes every time.

“Results” means the code ran once.

Then someone asks:

“Can we improve this?”

Of course we can improve it.

We can improve anything.

We can improve the model, the data, the pipeline, the evaluation, the hardware and the person who asked the question.

But research has a tradition of improving everything except the deadline.

The Server

A data scientist’s relationship with a server is similar to a toxic relationship.

When it works, you forget every bad thing it has done.

When it stops working, you blame yourself.

Maybe I used too much memory.

Maybe I launched too many processes.

Maybe I should have checked the GPU.

Maybe the server needs space.

The server never apologises.

It only says:

CUDA out of memory

This is not an error message.

It is a personality.

Servers at research institutions also have names, which makes the suffering personal.

You are not just unable to access a machine.

Catanzaro has rejected you.

Euclid is not responding.

Ramanujan has killed your process.

You start speaking about them like difficult colleagues.

“Catanzaro was fine yesterday.”

“Euclid is behaving strangely.”

“Someone has occupied all the GPUs.”

There is always someone.

Nobody has seen this person.

But all four GPUs are occupied, 700 GB of disk space is missing, and a process has been running since the previous government.

This person is the dark matter of scientific computing.

We cannot observe them directly, but their effects are everywhere.

GPU Politics

In ordinary offices, people fight over credit.

In AI labs, people fight over memory.

Not human memory.

That is already gone.

GPU memory.

You start judging experiments morally by how much VRAM they use.

A small model is disciplined.

A large model is ambitious.

A model using two GPUs is collaborative.

A model using all available GPUs is management.

You check nvidia-smi more often than your bank balance.

At least the bank balance is predictable.

It is low.

The GPU situation changes every second.

When a GPU becomes free, you must act immediately.

There is no time to think about whether the experiment is ready.

The GPU is free now.

This is how many research decisions are made.

Not by scientific importance.

By availability.

Human civilisation may eventually achieve artificial general intelligence because somebody noticed GPU 2 was free at 3:17 AM.

Papers

Reading research papers is another major part of the job.

A paper begins with a simple problem.

“We want to estimate the position of an object.”

Reasonable.

Then, within two pages, the object has entered a probabilistic graphical model with latent variables, Lie groups, Bayesian assumptions and twelve Greek letters.

The paper says:

“Clearly.”

Nothing is clear.

“Clearly” in a research paper means the authors have decided the reader’s confidence is now a personal problem.

You read the same paragraph six times.

Then you open another paper that explains the first paper.

That paper cites a third paper.

The third paper is from 1987 and is available only as a scanned PDF tilted fourteen degrees to the left.

At this point, you are no longer learning a method.

You are doing archaeology.

And after two days, someone in the lab explains the entire concept in one sentence.

You understand it immediately.

This creates a new research question:

Why was the paper twenty-eight pages?

Coding for Research

Research code is different from production code.

Production code must be clean, tested and maintainable.

Research code must produce Figure 4 before Friday.

You begin with good intentions.

You create folders:

data

models

experiments

utils

After three months, the main script is called:

final_working_latest_new_corrected_v7.py

There are four commented versions of the same function.

One line says:

# do not change this

Nobody knows why.

You try changing it.

The accuracy drops by 18%.

You put it back.

Science has spoken.

A research codebase is not built.

It accumulates.

Every bug fix becomes architecture.

Every temporary solution becomes permanent.

Every notebook begins as exploration and ends as infrastructure.

Then somebody asks you to make the pipeline reproducible.

You reproduce it.

The result changes.

This proves the pipeline has free will.

Model Accuracy

Model accuracy is emotionally dangerous.

When the accuracy is low, you are sad.

When the accuracy is high, you become suspicious.

An accuracy of 72% is disappointing but believable.

An accuracy of 99.8% means something has gone terribly wrong.

This is one of the few jobs where success is treated as evidence of failure.

You show your result to the team.

“Ninety-nine per cent.”

Nobody celebrates.

Everyone starts an investigation.

“Did you split the data correctly?”

“Is there leakage?”

“Are the labels encoded in the filename?”

“Did you test on the training set?”

A scientist is a person who spends months trying to improve results and becomes frightened the moment they improve too much.

Eventually, you fix the problem and accuracy falls to 91%.

Everyone relaxes.

Now the model is scientifically honest.

The Difference Between Research and Engineering

Research asks:

“Can this work?”

Engineering asks:

“Can this work repeatedly?”

Management asks:

“Can you demonstrate it tomorrow?”

The data scientist stands between all three, holding a notebook that only runs on one machine.

In research, a method can be successful even if it works under twelve conditions.

In engineering, users will immediately create a thirteenth condition.

The paper says the model is robust to noise.

The real world sends a corrupted packet, missing timestamp, unusual emitter, wrong unit, disconnected sensor and a user who clicked the button twice.

Then the model fails.

This is called deployment.

Being Surrounded by Intelligent People

Working at IISc means meeting people who are genuinely brilliant.

This is inspiring for approximately four minutes.

Then somebody casually uses a mathematical concept you have been trying to understand for two weeks.

They say:

“It’s quite straightforward.”

You nod.

There are only two respectable responses inside a research institution:

“That makes sense.”

And:

“Let me think about it.”

The second response means it does not make sense.

But with dignity.

The strange thing is that everyone is insecure.

The student thinks the senior researcher knows everything.

The senior researcher thinks the professor knows everything.

The professor knows the field is too large for anyone to know everything.

The field itself is waiting for the reviewer.

Intelligence does not remove confusion.

It gives confusion better vocabulary.

You are not lost.

You are navigating a high-dimensional search space.

Professors

Professors possess a special ability to ask one sentence that creates three months of work.

“What if we try this on real data?”

A simple question.

Almost friendly.

But the current dataset is synthetic.

The real data has a different format.

The sensor is unavailable.

The documentation is incomplete.

The machine recording the data uses an old library.

The old library does not compile.

The person who wrote the library has graduated.

His email now bounces.

But the question remains:

“What if we try this on real data?”

You cannot say no.

Research begins where reasonable objections end.

Deadlines

Research deadlines are flexible in the same way metal is flexible.

Technically true, under sufficient force.

At the beginning, the deadline is far away.

This is the planning phase.

In the middle, there is still time.

This is the denial phase.

Near the end, the entire team suddenly becomes productive.

This is science.

Experiments that took four days now complete overnight because fear has improved the optimisation algorithm.

You stop asking whether the code is elegant.

You ask whether the graph has labels.

Then, after submission, somebody notices a typo in the title.

The model survived noise.

The humans did not.

The Prestige

There is prestige attached to working at IISc.

Prestige is useful because it helps other people misunderstand your life positively.

They imagine you are doing important work every minute.

Sometimes you are.

Sometimes you are installing a package.

Sometimes you spend six hours discovering that two libraries require different versions of Python.

This is not shown in films about scientists.

Nobody makes a dramatic montage where the researcher creates a Conda environment.

But this is where much of science happens.

Not at the whiteboard.

Inside dependency conflicts.

You may be working on radar, drones, machine learning, state estimation or signal processing.

But eventually, all scientific fields converge on one common problem:

Why is this package not installing?

The Emotional Experience

Being a data scientist at IISc is exciting because difficult problems are everywhere.

It is frustrating for exactly the same reason.

You may spend weeks on an experiment that fails.

Then one small change makes it work.

You feel intelligent.

The next day, it stops working.

Balance is restored.

There are moments when everything connects:

The data makes sense.

The model improves.

The visualisation looks clean.

The theory matches the experiment.

For several minutes, you feel that human knowledge has moved forward.

Then the Wi-Fi disconnects.

This is useful.

It prevents arrogance.

So, What Is It Really Like?

Being a data scientist at IISc means living between impressive words and ordinary problems.

You work on artificial intelligence but forget your own passwords.

You study robust systems using a pipeline that breaks when a folder is renamed.

You build models that detect hidden patterns while spending twenty minutes searching for your charging cable.

You are surrounded by people trying to understand uncertainty, while everyone is uncertain about funding, publication, deadlines, careers and whether the meeting is in Room 208 or Room 308.

And yet, beneath all the confusion, something real is happening.

People are asking difficult questions.

They are building things that did not exist.

They are testing ideas that may fail.

They are trying to understand signals, machines, movement, language, intelligence and the world itself.

Most days, progress does not look like discovery.

It looks like debugging.

It looks like cleaning one more file.

It looks like rerunning one failed experiment.

It looks like admitting the previous result was wrong.

It looks like saying:

“Let us try once more.”

Which may be the least glamorous and most honest definition of science.

So yes, being a data scientist at IISc is prestigious.

But mostly from a safe distance.

From inside, it is data, doubt, GPUs, meetings, broken environments, strange filenames, promising initial results and the constant suspicion that the model knows something you do not.

And sometimes, after weeks of work, the experiment succeeds.

You look at the result.

You feel proud.

Then you realise you used the wrong dataset.

Further investigation is required.