Why Data Science Needs to Embrace Part-Time Analysts

You’ve started using a data science tool that’s supposed to “empower users.” And for some features, that’s true; it’s really easy to get some things done. But as soon as you need to take one step beyond those features — which almost always happens — it’s bang-head-against-wall time.

But that’s ok. You’re a data analyst. You know the drill. Spend enough time with a tool and eventually you’ll get it. In a few months, the weirdnesses will be second nature to you.

But there’s a big if: only if you spend a lot of your time as a data analyst.

That’s not true for a lot of people who need to crunch data. They probably have a few weekly/monthly reports or analyses that are critical to their work. But they only tweak these reports once or twice a year.

Two to three times a year, they do get to spend more time on analysis. For example, at the beginning of the year they may set up some reports to track their team or department’s new goals, and they analyze the results at the end of the year. They may also have a quarterly report they tweak every once in a while. But they’re not spending time every week or even every month immersed in the tool.

And that is going to bite them on the ass. Even if initially they can carve out enough time to figure out the bizarre commands needed to get something done, six months from now will they remember what they did and why? Not likely.

It’s not that these analysts don’t want to spend more time crunching data. They can see the potential of what they could do with the data they have if they only could spare the time. But it’s only a small slice of the work on their plate.

Ironically, if it was quicker & easier to do some work in data science, they might be able to muck around more frequently. Right now, it’s just not worth the hassle given all the other work on a typical part-time data analyst’s plate.

As AI technology improves, there will be even more part-time analysts who are struggling with this challenge. IBM, Microsoft, and hundreds of startups are trying to figure out how to automate as much of the work involved in using machine learning and other complex techniques. The closer they get to putting these techniques in the hands of Excel power users, the more likely the world of data science will include lots of people who are actively flexing their data science muscles infrequently.

Most of data science is built around the implicit assumption that the people who do it will either be working full time or part time on it. That assumption is understandable: in the world of coding, it’s largely true. But for data science to reach its full potential, it’s going to need to embrace users who don’t or can’t spend anywhere near that kind of time.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s