Data Science Overview 📖

MATH/COSC 3570 Introduction to Data Science

Dr. Cheng-Han Yu
Department of Mathematical and Statistical Sciences
Marquette University

What is Data Science or a Data Scientist?

A Little History of Data Science

• In 1985, Dr. Jeff Wu used the term Data Science for the first time as an alternative name for Statistics.

• The first question is What is DS? Actually this is a really hard question to answer, at least at this moment.

• 1985. Originally, DS was proposed to replace statistics. DS is just another name or maybe better name for statistics.
• 1997. But still at that time, nobody cares. Statisticians are weird guys in math departments who deal with proofs and theorems and sometimes real data, and they are stubborn, not willing to change. But if you wiki the definition of statistics, statistics is a science of data, and you tell me what is the difference of science of data and data science?
• 2001. Using this definition, DS becomes a broader term than statistics. Anything related to data can be a part of data sicence, while statistics now becomes a narrower subject focusing on inference and data analysis part.

• Here shows the google trend of DS. When I was in college or beginning of PHD studies, very few people used DS as a term. And all of sudden, DS become more and more popular, and everybody is talking about it.

Source: https://www.reddit.com/r/meme/comments/floq3q/reality_behind_data_science/

Source: https://br.ifunny.co/picture/we-will-work-together-statistics-computer-science-please-teach-now-h4hdtthT9?s=cl

😕 Still what on earth is data science?

Battle of the Data Science Venn Diagrams

• 2010 Drew Conway

• 2012 Brendan Tierney

• Starting 2010, people tried to use venn diagrams to define DS, what should be include, what should be not. It’s a subset of something, or superset of something.

• We have to do DS on some subject/field.

• (left) ML use CS and STAT but doesn’t care about domain knowledge.

• If you apply CS to substantive expertise without solid math and stats, you are doing bullshit basically.

• (right) DS is a subset of KDD - Knowledge Discovery in Data

Battle of the Data Science Venn Diagrams

• 2013 Ulrich Matter

• 2014 Michael Malak

Battle of the Data Science Venn Diagrams

• 2015 Stephan Kolassa

• 2016 Gregory Piatetsky-Shapiro

Shall We Continue?

• 2017 Longbing Cao

• totally-not-a-Venn-diagram by Calvin Andrus

• You probably get the idea. There are so many ways to define data science.

What is Data Scientist

• 💰 💰 Dr. Grant (Statistics Education Research Journal, 2017): Data science is the field of people who decide to print “Data Scientist” on their business cards to get a salary bump! 💵 💵

What is Data Scientist

Source: https://www.instagram.com/data_science_learn/

Source: https://www.instagram.com/data_science_dojo/

What Wiki Defines

GenAI Prompting

• Make AI your thinking partner and tutor!

• Copy and paste the paragraph into your GenAI chat as the initial prompt.

I am a college student in an introductory data science course. Be my thinking partner and tutor, not just an answer generator. Explain everything at an introductory level in plain language and avoid advanced jargon unless I ask. Walk through problems step by step and show your reasoning, and ask up to two quick questions if you need missing details. Use a tiny example when it helps. If code is needed, use either R (tidyverse) or Python (pandas, scikit-learn) based on what I say I am using, and keep the code minimal, well commented, and explained line by line. If my approach is wrong, tell me clearly what is wrong and how to fix it. End with one quick check for understanding and a one sentence summary.

GenAI Prompting

• Why should we care about and learn data science?

> How does data science show up in everyday decisions I already care about?

> What mistakes happen when decisions are made without data or with bad data?

> How is data science used in my major: [environmental science, computer science, etc]?

• Share your AI’s responses and your thoughts!~

Goal: Answer the student’s implicit question: Why should I care? These prompts connect data science to lived experience and decision making, not abstract tools. They also reveal that data science is not just for programmers or specialists.

Data Science in This Course

• Data science is an discipline that allows us to turn raw data into understanding, insight, and knowledge.

• We’re going to learn to do this in a tidy way – more on that later!

• This is a introductory data science course with an emphasis on important tools of R/Python that help us do data science.

A Data Science Project

Data Science Workflow

• Import: Take data stored somewhere and load it into your workspace.
• Tidy: Storing data in a consistent rectangular form, i.e., a data matrix.
• Transform: Narrowing in on observations of interest, creating new variables, calculating statistics.

• To do DS, we need data, so the first step is to import data (——).

• But usually the raw data are very messy, so we need tidy the data by storing data in a consistent rectangular form (data frame in R), which is easy for modeling and analysis.

• Depending on our research goal, we may just want to focus on some variables , or observations in the data, or we may want to create some variables for example, the data use feet, but we want to use meter instead. This data transformation further cleans the data, so that we can extract any information we need from data more easily and efficiently. In other words, we improve the quality of data.

Data Matrix

• Each row corresponds to a unique case or observational unit.

• Each column represents a characteristic or variable.

• This structure allows new cases to be added as rows or new variables as new columns.

• The process of importing, tidying and transforming data is called data wrangling or data munging.

• Visualisation: A good visualisation shows you things that you did not expect or raise new questions about the data.

• Once the data are ready, it’s time to explore our data.

• My recommendation is that always plot your data before building any model or algorithm or doing analysis because good visualization let you understand more about your data, and may provide some hints about how to build your statistical model, machine learning algorithms or how to do analysis.

• And good visualization shows you things that you did not expect or raise new questions about the data. (outliers, missing data)

• So after visualizing your data, you may have better idea of how to transforming your data so that you can be better prepared for modeling and answer your research questions. (nonlinear relationship <-> linear relationship)

mpg |> ggplot(aes(x = displ, y = hwy)) +
    geom_point(aes(color = class)) + 
    geom_smooth() + 
    theme_bw()

• Model: Models are complementary tools to visualisation. Once you have made your questions sufficiently precise, you can use a model to answer them.

• The next step is to use or build a rigorous statistical or machine learning model to answer our research questions. For example, forecasting stock prices, classifying images.

• And transform-visualize-model becomes a loop of understanding or learning from data. We keep transforming, visualizing and modeling our data until our questions are properly answered by our data and model.

library(tidymodels)
linear_reg() |>  
    set_engine("lm") |> 
    fit(hwy ~ displ, data = mpg)

parsnip model object


Call:
stats::lm(formula = hwy ~ displ, data = data)

Coefficients:
(Intercept)        displ  
      35.70        -3.53  

• Communication: It doesn’t matter how well your models and visualization have led you to understand the data unless you can also communicate your results to others.

• Finally we have to share our analysis results with others.

• At this stage, visualization plays an important role as well.

• Figures/tables speak louder than words.

• We can have different types of deployment. In addition to papers or reports, we can create a website, a software package, a mobile app, dashboard and so on.

• Programming: Surrounding all these tools is programming.

• Of course, every aspect of DS is done by programming. DS cannot be done by pieces of paper and pens. It has to be done using computers.

R for Data Science

Source: https://teachdatascience.com/tidyverse/

• If you are ready, let’s dive in and learn all these tools. Well not all, but some of these tools.

• By the way, you see these hex stickers? It kind of becomes a trend that when we create an R package, or software, we also create a hex logo associated with it, so it works as product or brand logo. It may be easier for you to promote your package if you have a cute or beautiful hex logo.

Python for Data Science

Source: https://www.e2enetworks.com/blog/9-python-libraries-for-data-science-and-artificial-intelligence

GenAI Prompting

• Making Data Science Concrete Through Examples

> Walk through a simple data science problem from question to conclusion.

• Share your example! Does the workflow generally follow what we discussed?